Title: Default for 22.1 Basic concepts
Course: HELM Workbooks 2023-2024

Initial concepts

Determinants reminder

A square matrix possesses an associated determinant. Unlike a matrix, which is an array of numbers, a determinant has a single value.

A two by two matrix \(C=\left[\begin{array}{ll}c_{11}&c_{12}\\ c_{21}&c_{22}\end{array}\right]\) has an associated determinant

\[\mathrm{det}(C)=\begin{vmatrix}c_{11}&c_{12}\\ c_{21}&c_{22}\end{vmatrix} = c_{11}c_{22}-c_{21}c_{12}\]

(Note square or round brackets denote a matrix, straight vertical lines denote a determinant.)

A three by three matrix has an associated determinant

\[\det (C)=\begin{vmatrix}c_{11}&c_{12}&c_{13}\\ c_{21}&c_{22}&c_{23}\\ c_{31}&c_{32}&c_{33}\end{vmatrix}\]

Among other ways, this determinant can be evaluated by an “expansion about the top row”:

\[\det(C)=c_{11}\begin{vmatrix}c_{22}&c_{23}\\ c_{32}&c_{33}\end{vmatrix} - c_{12}\begin{vmatrix}c_{21}&c_{23}\\ c_{31}&c_{33}\end{vmatrix} + c_{13}\begin{vmatrix}c_{21}&c_{22}\\ c_{31}&c_{32}\end{vmatrix}\]

Note the minus sign in the second term.

A matrix such as \(A=\left[\begin{array}{cc}4&8\\ 1&2\end{array}\right]\)  which has zero determinant is called a singular matrix. Matrices with a non-zero determinant are non-singular. The key factor to be aware of is as follows:

Key Point

Any non-singular \(n\times n\) matrix \(C\), for which \(\text{det}(C)\neq 0 \), possesses an inverse \(C^{-1}\) i.e.

\[CC^{-1}=C^{-1}C=I \quad \text{where } I \text{ denotes the } n\times n \text{ identity matrix.}\]

A singular matrix does not possess an inverse.

Exercise

Compute the determinants of \[ \mathrm{A}=\begin{bmatrix}[value]&[value]&[value]\\ [value]&[value]&[value]\\ [value]&[value]&[value]\\ \end{bmatrix} \ \text{ and } \ \mathrm{B}=\begin{bmatrix}[value]&0&0&[value]\\ [value]&0&0&[value]\\ [value]&0&[value]&0\\ 0&[value]&[value]&[value]\\ \end{bmatrix} . \] Laske matriisien \[ \mathrm{A}=\begin{bmatrix}[value]&[value]&[value]\\ [value]&[value]&[value]\\ [value]&[value]&[value]\\ \end{bmatrix} \ \text{ ja } \ \mathrm{B}=\begin{bmatrix}[value]&0&0&[value]\\ [value]&0&0&[value]\\ [value]&0&[value]&0\\ 0&[value]&[value]&[value]\\ \end{bmatrix} \] determinantit.

\(\operatorname{det}(\mathrm{A})=\)

\(\operatorname{det}(\mathrm{B})=\)
Feedback: Determinants can be calculated for example by reducing the matrix to an upper triangle matrix or developing the determinant with respect to some row or column. Let's compute \(\det(\mathrm{A})\) by developing it with respect to the first row: \begin{eqnarray} \det(\mathrm{A})&=&[value]\left| \begin{matrix}[value] & [value] \\ [value] & [value] \\ \end{matrix}\right| -[value] \left| \begin{matrix}[value] & [value] \\ [value] & [value] \\ \end{matrix}\right| +([value]) \left| \begin{matrix}[value] & [value] \\ [value] & [value] \\ \end{matrix}\right| \nonumber \\ &=& [value]([value]\cdot ([value])-[value]\cdot [value])-[value]([value]\cdot ([value])-[value]^2)+([value])([value]\cdot [value]-[value]\cdot [value])\nonumber \\ &=& [value]. \nonumber \end{eqnarray} The column of \(\mathrm{B}\) with the most zeros is the second column, so let's develop the determinant with respect to that: \[ \det(\mathrm{B}) = ([value])\left| \begin{matrix} [value]&0&[value]\\ [value]&0&[value]\\ [value]&[value]&0\\ \end{matrix}\right| . \] Here the \(3\times 3\)-determinant reduces to a \(2\times 2\)-determinant when developed with respect to the second column: \[ \det(\mathrm{B})=([value])\cdot (-([value]))\left| \begin{matrix} [value]&[value]\\ [value]&[value]\\ \end{matrix}\right| = -[value]^2([value]^2-([value])^2)=[value]. \] Mallivastaus: Determinantti voidaan laskea joko muuntamalla se yläkolmiomuotoon tai kehittämällä sopivan rivin tai sarakkeen mukaan. Lasketaan \(\det(\mathrm{A})\) kehittämällä 1. rivin mukaan: \begin{eqnarray} \det(\mathrm{A})&=&[value]\left| \begin{matrix}[value] & [value] \\ [value] & [value] \\ \end{matrix}\right| -[value] \left| \begin{matrix}[value] & [value] \\ [value] & [value] \\ \end{matrix}\right| +([value]) \left| \begin{matrix}[value] & [value] \\ [value] & [value] \\ \end{matrix}\right| \nonumber \\ &=& [value]([value]\cdot ([value])-[value]\cdot [value])-[value]([value]\cdot ([value])-[value]^2)+([value])([value]\cdot [value]-[value]\cdot [value])\nonumber \\ &=& [value]. \nonumber \end{eqnarray} Matriisissa \(\mathrm{B}\) on paljon nollia, eniten niitä on 2. sarakkeessa, joten kehitetään determinantti sen mukaan: \[ \det(\mathrm{B}) = ([value])\left| \begin{matrix} [value]&0&[value]\\ [value]&0&[value]\\ [value]&[value]&0\\ \end{matrix}\right| . \] Tässä \(3\times 3\)-determinantti kannattaa kehittää 2. sarakkeen mukaan, jolloin saadaan \[ \det(\mathrm{B})=([value])\cdot (-([value]))\left| \begin{matrix} [value]&[value]\\ [value]&[value]\\ \end{matrix}\right| = -[value]^2([value]^2-([value])^2)=[value]. \]

Systems of linear equations

We first recall some basic results in linear (matrix) algebra. Consider a system of \(n\) equations in \(n\) unknowns \(x_1,x_2,\dots , x_n\):

\(\qquad \begin{matrix}c_{11}x_1&+&c_{12}x_2&+&\dots &+&c_{1n}x_n&=&k_1\\ c_{21}x_1&+&c_{22}x_2&+&\dots &+&c_{2n}x_n&=&k_2\\ \vdots&+&\vdots&+&\dots &+&\vdots&=&\vdots\\ c_{n1}x_1&+&c_{n2}x_2&+&\dots &+&c_{nn}x_n&=&k_n\end{matrix}\)

We can write such a system in matrix form:

\(\qquad \left[\begin{matrix}c_{11}&c_{12}&\dots &c_{1n}\\ c_{21}&c_{22}&\dots &c_{2n}\\ \vdots&\vdots&\ddots &\vdots\\ c_{n1}&c_{n2}&\dots &c_{nn}\end{matrix}\right] \left[\begin{matrix}x_1\\ x_2\\ \vdots\\ x_n\end{matrix}\right]= \left[\begin{matrix}k_1\\ k_2\\ \vdots\\ k_n\end{matrix}\right],\)

or equivalently, \(C\underline{x}=\underline{k}\).

We see that \(C\) is an \(n\times n\) matrix (called the coefficient matrix), \(\underline{x}=[x_1, x_2,\dots ,x_n]^T\) is the \(n\times 1\) column vector of unknowns and \(\underline{k}=[k_1,k_2,\dots ,k_n]^T\) is an \(n\times 1\) column vector of given constants.

The zero vector will be denoted by \({\underline 0}\).

If \(\underline{k}\ne {\underline 0}\) the system is called inhomogeneous.

If \(\underline{k}={\underline 0}\) the system is called homogeneous.

Important results in linear algebra

Consider the system of equations \(C\underline{x}=\underline{k}\).

We are concerned with the nature of the solutions (if any) of this system. We shall see that this system only exhibits three solution types:

\(\bullet\)  The system is consistent and has a unique solution for \(\underline{x}\)

\(\bullet\)  The system is consistent and has an infinite number of solutions for \(\underline{x}\)

\(\bullet\)  The system is inconsistent and has no solution for \(\underline{x}\)

There are two basic cases to consider:

\(\det(C)\ne 0\)   or   \(\det(C)=0\)

Case 1: \(\det(C)\ne 0\)

In this case \(C^{-1}\) exists and the unique solution to \(C\underline{x}=\underline{k}\) is

\(\qquad \underline{x}=C^{-1}\underline{k}\)

Case 2: \(\det(C)=0\)

In this case \(C^{-1}\) does not exist.

1. If \(\underline{k}\ne{\underline 0}\) the system \(C\underline{x}=\underline{k}\) either has no solutions or has an infinite number of solutions. We can use Gaussian elimination to determine which case it is. If we have no solutions then the reduced augmented matrix will have a row which is zero in all but the rightmost position; otherwise we have infinitely many solutions.
2. If \(\underline{k}={\underline 0}\) the system \(C\underline{x}={\underline 0}\) has an infinite number of solutions.

We note that a homogeneous system

\[C\underline{x}={\underline 0}\]

has a unique solution \(\underline{x}={\underline 0}\) if \(\det(C)\ne 0\) (this is called the trivial solution) or an infinite number of solutions if \(\det(C)=0\).

Example

(Case 1) Solve the inhomogeneous system of equations

\[\left\{\begin{alignedat}{2} x_1 & {}+{} & x_2 & = 1 \\ 2x_1 & {}+{} & x_2 & = 2 \end{alignedat}\right.\]

which can be expressed as \(C\underline{x}=\underline{k}\) where

\[C=\begin{bmatrix}1&1\\ 2&1\end{bmatrix} \qquad \underline{x}=\begin{bmatrix} x_1\\ x_2\end{bmatrix} \qquad \underline{k}=\begin{bmatrix} 1\\ 2\end{bmatrix}\]

Solution

Here det\((C)=-1\neq 0\).

The system of equations has the unique solution: \(\underline{x}=\left[\begin{array}{c} x_1\\ x_2\end{array}\right]=\left[\begin{array}{c} 1\\ 0 \end{array}\right] \).

Example

(Case 2.a) Examine the following inhomogeneous system for solutions

\[\left\{\begin{alignedat}{2} x_1 & {}+{} & 2x_2 & = 1 \\ 3x_1 & {}+{} & 6x_2 & = 0 \end{alignedat}\right.\]

Solution

Here \(\det(C)=\begin{vmatrix}1&2\\3&6\end{vmatrix}=0\). In this case there are no solutions.

To see this we see the first equation of the system states \(x_1+2x_2=1\) whereas the second equation (after dividing through by 3) states \(x_1+2x_2=0\), a contradiction.

Example

(Case 2.b) Solve the homogeneous system

\[\left\{\begin{alignedat}{2} x_1 & {}+{} & x_2 & = 0 \\ 2x_1 & {}+{} & 2x_2 & = 0 \end{alignedat}\right.\]

Solution

Here \(\det(C)=\begin{vmatrix}1&1\\2&2\end{vmatrix}=0\). The solutions are any pair of numbers \(\{x_1,x_2\}\) such that \(x_1=-x_2\), i.e. \(\underline{x} = \begin{bmatrix}\alpha\\-\alpha\end{bmatrix}\), where \(\alpha\) is an arbitrary element of \(\mathbb{R}\). There are an infinite number of solutions.

A simple eigenvalue problem

We shall be interested in simultaneous equations of the form:

\(\qquad A\underline{x}=\lambda \underline{x},\)

where \(A\) is an \(n\times n\) matrix, \(\underline{x}\) is an \(n\times 1\) column vector and \(\lambda\) is a scalar (a constant) and, in the first instance, we examine some simple examples to gain experience of solving problems of this type.

We approach the following example by examining the equations. This should make the actions within the solution more transparent. In practice, we will work with the matrices directly which will give a much less cluttered solution.

Example

Consider the following system with \(n=2\):

\[\left\{\begin{alignedat}{2} 2x & {}+{} & 3y & = \lambda x \\ 3x & {}+{} & 2y & = \lambda y \end{alignedat}\right.\]

so that

\(A=\begin{bmatrix}2&3\\ 3&2\end{bmatrix}\)   and   \(\underline{x}=\begin{bmatrix}x\\ y\end{bmatrix}\).

It appears that there are three unknowns \(x,y,\lambda\). The obvious questions to ask are: can we find \(x,y\)? what is \(\lambda\)?

Solution

To solve this problem we firstly re-arrange the equations (take all unknowns onto one side);

\begin{align} (2-\lambda)x + 3y &= 0 \tag{1} \\ 3x + (2-\lambda)y &= 0 \tag{2} \end{align}

Therefore, from equation \((2)\):

\[x = -\frac{(2-\lambda)}{3}y \tag{3}\]

Then when we substitute this into \((1)\)

\(\displaystyle -\frac{(2-\lambda)^2}{3}y+3y = 0\)   which simplifies to \([-(2-\lambda)^2+9]y=0\).

We conclude that either \(y=0\) or \(9=(2-\lambda)^2\). There are thus two cases to consider:

Case 1

If \(y=0\) then \(x=0\) (from \((3)\)) and we get the trivial solution. (We could have guessed this solution at the outset.)

Case 2

\(\qquad 9=(2-\lambda)^2\)

which gives, on taking square roots:

\(\pm 3 = 2-\lambda\)   giving   \(\lambda=2\pm 3\),   so   \(\lambda = 5\)   or   \(\lambda =1\).

Now, from equation \((3)\), if \(\lambda=5\) then \(x=+y\) and if \(\lambda=-1\) then \(x=-y\).

We have now completed the analysis. We have found values for \(\lambda\) but we also see that we cannot obtain unique values for \(x\) and \(y\): all we can find is the ratio between these quantities. This behaviour is typical, as we shall now see, of an eigenvalue problem.

General eigenvalue problems

Consider a given square matrix \(A\). If \(\underline{x}\) is a column vector and \(\lambda\) is a scalar (a number) then the relation

\[A\underline{x}=\lambda \underline{x} \tag{1}\]

is called an eigenvalue problem. Our purpose is to carry out an analysis of this equation in a manner similar to the example above. However, we will attempt a more general approach which will apply to all problems of this kind.

Firstly, we can spot an obvious solution (for \(\underline{x}\)) to these equations. The solution \(\underline{x}=\underline{0}\) is a possibility (for then both sides are zero). We will not be interested in these trivial solutions of the eigenvalue problem. Our main interest will be in the occurrence of non-trivial solutions for \(\underline{x}\). These may exist for special values of \(\lambda\), called the eigenvalues of the matrix \(A\). We proceed as in the previous example:

Take all unknowns to one side:

\[(A-\lambda I)\underline{x}=\underline{0} \tag{2}\]

where \(I\) is a unit matrix with the same dimensions as \(A\). (Note that \(A\underline{x}-\lambda \underline{x}=\underline{0}\) does not simplify to \((A-\lambda)\underline{x}=\underline{0}\) as you cannot subtract a scalar \(\lambda\) from a matrix \(A\)). This equation \((2)\) is a homogeneous system of equations. In the notation of the earlier discussion \(C\equiv A-\lambda I\) and \(\underline{k}\equiv \underline{0}\). For such a system we know that non-trivial solutions will only exist if the determinant of the coefficient matrix is zero:

\[\det(A-\lambda I)=0 \tag{3}\]

Equation \((3)\) is called the characteristic equation of the eigenvalue problem. We see that the characteristic equation only involves one unknown \(\lambda\). The characteristic equation is generally a polynomial in \(\lambda\), with degree being the same as the order of \(A\) (so if \(A\) is \(2\times 2\) the characteristic equation is a quadratic, if \(A\) is a \(3\times 3\) it is a cubic equation, and so on). For each value of \(\lambda\) that is obtained the corresponding vector \(\underline{x}\) is obtained by solving the original equations \((1) \). These \(\underline{x}\) are called eigenvectors.

N.B. We shall see that eigenvectors are only unique up to a multiplicative factor: i.e. if \(\underline{x}\) satisfies \(A\underline{x}=\lambda \underline{x}\) then so does \(k\underline{x}\) when \(k\) is any constant.

Example

Find the eigenvalues and eigenvectors of the matrix \(A=\begin{bmatrix}1&0\\ 1&2\end{bmatrix}\).

Solution

The eigenvalues and eigenvectors are found by solving the eigenvalue problem

\[ A\underline{x}=\lambda \underline{x} \qquad \underline{x}=\begin{bmatrix}x\\ y\end{bmatrix} \quad \implies \quad (A - \lambda I) \underline{x} = \underline{0}\]

Non-trivial solutions will exist if \(\det(A-\lambda I)=0\), that is,\[\det\left\{\begin{bmatrix}1&0\\ 1&2\end{bmatrix} - \lambda\begin{bmatrix}1&0\\ 0&1 \end{bmatrix}\right\} = 0 \quad \therefore \quad \begin{vmatrix}1-\lambda&0 \\ 1&2-\lambda\end{vmatrix} = 0\]

Expanding this determinant gives: \((1-\lambda)(2-\lambda)=0\). Hence the solutions for \(\lambda\) are \(\lambda =1\) and \(\lambda=2\).

So we have found two values of \(\lambda\) for this \(2\times 2\) matrix \(A\). Since these are not equal they are said to be distinct eigenvalues.

To each value of \(\lambda\) there corresponds an eigenvector. We now proceed to find the eigenvectors.

Case 1

\(\lambda = 1\) (smaller eigenvalue). Our original eigenvalue problem then becomes: \(A\underline{x}=\underline{x}\). In full this is

\[\left\{\begin{alignedat}{2} & & x & = x \\ x & {}+{} & 2y & = y \end{alignedat}\right.\]

Simplifying,

\[\left\{\begin{alignedat}{2} & & x & = x \\ x & {}+{} & y & = 0 \end{alignedat}\right.\]

All we can deduce here is that \(x=-y\), so \(\underline{x}=\begin{bmatrix}x\\ -x\end{bmatrix}\) for any \(x\neq 0\).

(We specify \(x \ne 0\) as, otherwise, we would have the trivial solution.)

So the eigenvectors corresponding to eigenvalue \(\lambda = 1\) are all proportional to \(\begin{bmatrix} 1 \\ -1\end{bmatrix}\), e.g. \(\begin{bmatrix} 2\\ -2\end{bmatrix}\), \(\begin{bmatrix} -1 \\ 1\end{bmatrix}\) etc.

Sometimes we write the eigenvector in normalised form that is, with modulus or magnitude 1. Here, the normalised form of \(\underline{x}\) is

\(\displaystyle \frac{1}{\sqrt{2}} \begin{bmatrix} 1 \\ -1 \end{bmatrix}\)   which is unique.

Case 2

Now we consider the larger eigenvalue \(\lambda =2\). Our original eigenvalue problem \(A\underline{x}=\lambda \underline{x}\) becomes \(A\underline{x}=2\underline{x}\) which gives the following equations:

\[\begin{bmatrix} 1 & 0 \\ 1 & 2 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = 2 \begin{bmatrix} x \\ y \end{bmatrix}\]

i.e.,

\[\left\{\begin{alignedat}{2} & & x & = 2x \\ x & {}+{} & 2y & = 2y \end{alignedat}\right.\]

These equations imply that \(x=0\), whilst the variable \(y\) may take any value whatsoever (except zero as this gives the trivial solution).

Thus the eigenvector corresponding to eigenvalue \(\lambda =2\) has the form \(\begin{bmatrix}0\\ y\end{bmatrix}\), e.g. \(\begin{bmatrix} 0\\ 1 \end{bmatrix}\), \(\begin{bmatrix} 0\\ 2\end{bmatrix}\) etc.

The normalised eigenvector here is \(\begin{bmatrix} 0\\ 1\end{bmatrix}\).

In conclusion, the matrix \(A=\begin{bmatrix} 1&0\\ 1&2\end{bmatrix}\) has two eigenvalues and two associated normalised eigenvectors:

\[\begin{aligned} \lambda_1 &= 1 & \lambda_2 &= 2 \\ \underline{x}_1 &= \frac{1}{\sqrt{2}} \begin{bmatrix} 1 \\ -1 \end{bmatrix} & \underline{x}_2 &= \begin{bmatrix} 0 \\ 1 \end{bmatrix} \end{aligned}\]

Properties of eigenvalues and eigenvectors

There are a number of general properties of eigenvalues and eigenvectors which you should be familiar with. You will be able to use them as a check on some of your calculations.

Property 1: Sum of eigenvalues

For any square matrix \(A\):

\(\quad\)sum of eigenvalues \(=\) sum of diagonal terms of \(A\) (called the trace of \(A\))

Formally, for an \(n\times n\) matrix \(A\): \(\qquad \displaystyle{\sum_{i=1}^n \lambda_i=\ \text{ trace} (A)}\)

(Repeated eigenvalues must be counted according to their multiplicity.)
Thus if \(\lambda_1=4, \lambda_2=4, \lambda_3=1\) then \(\displaystyle{\sum_{i=1}^{3}\lambda_i=9}\)).

Property 2: Product of eigenvalues

For any square matrix \(A\):

\(\quad\)product of eigenvalues \(=\) determinant of \(A\)

Formally: \(\displaystyle{\qquad \lambda_1\lambda_2\lambda_3\cdots\lambda_n=\prod_{i=1}^n \lambda_i=\text{ det}(A)}\)

The symbol \(\prod\) simply denotes multiplication, as \(\sum\) denotes summation.

Example

Verify Properties 1 and 2 for the \(3\times 3\) matrix:

\(\qquad A = \left[\begin{array}{rrr}2&-1&0\\ -1&2&-1\\ 0&-1&2\end{array}\right]\)

whose eigenvalues were found earlier.

Solution

The three eigenvalues for this matrix are:

\(\qquad \lambda_1=2-\sqrt{2},\qquad \lambda_2=2,\qquad \lambda_3=2+\sqrt{2}.\)

Therefore

\(\qquad \lambda_1+\lambda_2+\lambda_3=(2-\sqrt{2})+2+(2+\sqrt{2})=6=\text{trace}(A),\)

whilst

\(\qquad \lambda_1\lambda_2\lambda_3=(2-\sqrt{2})(2)(2+\sqrt{2})=4=\text{det}(A).\)

Property 3: Linear independence of eigenvectors

Eigenvectors of a matrix \(A\) corresponding to distinct eigenvalues are linearly independent i.e. one eigenvector cannot be written as a linear sum of the other eigenvectors. The proof of this result is omitted but we illustrate this property with two examples.

We saw earlier that the matrix

\(\qquad A=\left[\begin{array}{cc}1&0\\ 1&2\end{array}\right]\)

has distinct eigenvalues \(\lambda_1=1\qquad \lambda_2=2\) with associated eigenvectors

\(\qquad \underline{x_1}=\dfrac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ -1\end{array}\right]\qquad \underline{x_2}=\left[\begin{array}{r}0\\ 1\end{array}\right]\)

respectively.

Clearly \(\underline{x_1}\) is not a constant multiple of \(\underline{x_2}\) and these eigenvectors are linearly independent.

We also saw that the \(3\times 3\) matrix

\(\qquad A=\left[\begin{array}{rrr}2&-1&0\\ -1&2&-1\\ 0&-1&2\end{array}\right]\)

had the following distinct eigenvalues  \(\lambda_1=2-\sqrt{2},\ \lambda_2=2,\ \lambda_3=2+\sqrt{2}\)  with corresponding eigenvectors of the form shown: \(\qquad \underline{x_1}=\left[\begin{array}{c}1\\ \sqrt{2}\\ 1\end{array}\right],\qquad \underline{x_2}=\left[\begin{array}{r}1\\0\\-1\end{array}\right],\qquad \underline{x_3}=\left[\begin{array}{c}1\\ -\sqrt{2}\\ 1\end{array}\right]\)

Clearly none of these eigenvectors is a constant multiple of any other. Nor is any one obtainable as a linear combination of the other two. The three eigenvectors are linearly independent.

Property 4: Eigenvalues of diagonal matrices

A \(2\times 2\) diagonal matrix \(D\) has the form

\(\qquad D=\left[\begin{array}{cc}a&0\\ 0&d\end{array}\right]\)

The characteristic equation

\(\quad |D-\lambda I|=0\) is \(\left|\begin{array}{cc}a-\lambda&0\\ 0&d-\lambda\end{array}\right|=0\)

i.e. \(\qquad (a-\lambda)(d-\lambda)=0\)

So the eigenvalues are simply the diagonal elements \(a\) and \(d\).

Similarly a \(3\times 3\) diagonal matrix has the form

\(\qquad D=\left[\begin{array}{ccc}a&0&0\\ 0&b&0\\ 0&0&c\end{array}\right]\)

having characteristic equation

\(\qquad |D-\lambda I|=(a-\lambda)(b-\lambda)(c-\lambda)=0\)

so again the diagonal elements are the eigenvalues.

We can see that a diagonal matrix is a particularly simple matrix to work with. In addition to the eigenvalues being obtainable immediately by inspection it is exceptionally easy to multiply diagonal matrices.

Exercise

If \[A = [value],\] \[B = [value]\] and \[C = [value]\] find:

1. \(AB =\)
2. \(BC =\)
Feedback: Here we have the following matrices: \[ \bf A = [value], B = [value], C = [value]. \] To multiply matrix \(\bf A\) by matrix \(\bf B\) we need to remember that the \((i,j)^{\textrm{th}}\) entry in the new matrix is the scalar product of the \(i^{\textrm{th}}\) row of \(\bf A\) with the \(j^{\textrm{th}}\) column of \(\bf B\). For example to obtain the element in row 1, column 2 of the new matrix \(\bf A \bullet B\) we take row 1 of \(\bf A\), [[value], [value]], and multiply it with column 2 of \(\bf B\), [[value], [value]], in the usual way to produce \[ [[value]\times[value] + [value]\times[value]] = [[value]]. \] So, \[ \bf A\bullet B = [value] \bullet [value] = [value] = [value].\] Similarly, \[ \bf B\bullet C = [value] \bullet [value] = [value] = [value].\]

Exercise

Let \(\textbf{A}\) be the following matrix: \(\textbf{A} = [value]\).

What are the eigenvalues of \(\textbf{A}\)? Input your answer, as a set (e.g. {7,3,5/4}), in the box below:

The eigenvalues of \(\textbf{A}\) are

Find any eigenvector of \(\textbf{A}\). Input your answer, using column vector notation, in the boxes below:

An eigenvector of \(\textbf{A}\) is
Feedback: Our matrix \(\textbf{A}\) is given to be \(\textbf{A} = [value]\). The eigenvalues \(\lambda_i\) (\(i = 1,\ldots, n\)) of \(\textbf{A}\) are the roots of the characteristic polynomial \(\chi\,(\lambda)\), which is calculated below: \[\begin{array}{rcl} \chi\,(\lambda) & = & \textrm{det}\,\left({[value] - \lambda\,[value]}\right)\\ & = & \textrm{det}\,\left([value]\right)\\ & = & \cdots\\ & = & [value].\end{array}\] Upon factorisation and solution of the characteristic equation \([value] = 0\), we arrive at the set of eigenvalues of \(\textbf{A}\), which is [value]. From the set of eigenvalues, we select any one eigenvalue \(\lambda\). Finding the kernel of [value] (i.e. the set of things which, when [value] is applied to them, gives the zero vector as its result) gives us a basis in the \(\lambda\)-eigenspace. Any eigenvector in the \(\lambda\)-eigenspace (which, therefore, is an example of an eigenvector of \(\textbf{A}\)) may be found by taking some linear combination of the relevant basis vectors. Whichever vector \(\textbf{v}\) you have chosen in this way will, by definition, satisfy the eigenvalue equation: \[\textbf{A}\,\textbf{v} = \lambda\,\textbf{v}.\]

Exercise

Let \(\textbf{A}\) be the following matrix: \(\textbf{A} = [value]\).

What are the eigenvalues of \(\textbf{A}\)? Input your answer, as a set (e.g. {7,3,5/4} including the curly braces), in the box below:

The eigenvalues of \(\textbf{A}\) are

Find any eigenvector of \(\textbf{A}\). Input your answer, using column vector notation, in the boxes below:

An eigenvector of \(\textbf{A}\) is
Feedback: Our matrix \(\textbf{A}\) is given to be \(\textbf{A} = [value]\). The eigenvalues \(\lambda_i\) (\(i = 1,\ldots, n\)) of \(\textbf{A}\) are the roots of the characteristic polynomial \(\chi\,(\lambda)\), which is calculated below: \[\begin{array}{rcl} \chi\,(\lambda) & = & \textrm{det}\,\left({[value] - \lambda\,[value]}\right)\\ & = & \textrm{det}\,\left([value]\right)\\ & = & \cdots\\ & = & [value].\end{array}\] Upon factorisation and solution of the characteristic equation \([value] = 0\), we arrive at the set of eigenvalues of \(\textbf{A}\), which is [value]. From the set of eigenvalues, we select any one eigenvalue \(\lambda\). Finding the kernel of [value] (i.e. the set of things which, when [value] is applied to them, gives the zero vector as its result) gives us a basis in the \(\lambda\)-eigenspace. Any eigenvector in the \(\lambda\)-eigenspace (which, therefore, is an example of an eigenvector of \(\textbf{A}\)) may be found by taking some linear combination of the relevant basis vectors. Whichever vector \(\textbf{v}\) you have chosen in this way will, by definition, satisfy the eigenvalue equation: \[\textbf{A}\,\textbf{v} = \lambda\,\textbf{v}.\]

Example

Find the eigenvalues and eigenvectors of the \(3\times 3\) matrix

\[A=\begin{bmatrix} 2&-1&0\\ -1&2&-1\\ 0&-1&2\end{bmatrix}\]

Solution

The eigenvalues and eigenvectors are found by solving the eigenvalue problem

\[A\underline{x}=\lambda \underline{x} \qquad \underline{x}=\begin{bmatrix} x\\ y\\ z\end{bmatrix}\]

Proceeding as in the previous example:

\((A-\lambda I)\underline{x}=\underline{0}\)   and non-trivial solutions for \(\underline{x}\) will exist if \(\det(A-\lambda I)=0.\)

That is,

\[\det\left\{\begin{bmatrix} 2&-1&0\\ -1&2&-1\\ 0&-1&2\end{bmatrix} -\lambda\begin{bmatrix} 1&0&0\\ 0&1&0\\ 0&0&1\end{bmatrix}\right\}=0\]

i.e.,

\[\begin{vmatrix} 2-\lambda&-1&0\\ -1&2-\lambda&-1\\ 0&-1&2-\lambda\end{vmatrix}=0\]

Expanding this determinant we find:

\[(2-\lambda)\begin{vmatrix} 2-\lambda&-1\\ -1&2-\lambda\end{vmatrix} + \begin{vmatrix} -1&-1\\ 0&2-\lambda\end{vmatrix} = 0\]

that is,

\[(2-\lambda)\left\{(2-\lambda)^2-1\right\}-(2-\lambda)=0\]

Taking out the common factor \((2-\lambda)\):

\[(2-\lambda)\left\{4-4\lambda+\lambda^2-1-1\right\}\]

which gives:

\[(2-\lambda)\left[\lambda^2-4\lambda+2\right]=0.\]

This is easily solved to give \(\lambda =2\) or \(\lambda = \dfrac{4\pm \sqrt{16-8}}{2} = 2\pm \sqrt{2}\).

So (typically) we have found three possible values of \(\lambda\) for this \(3\times 3\) matrix \(A\).

To each value of \(\lambda\) there corresponds an eigenvector.

Case 1: \(\lambda = 2 - \sqrt{2}\) (lowest eigenvalue)

Then \(A\underline{x}=(2-\sqrt{2})\underline{x}\) implies

\[\left\{\begin{alignedat}{3} 2x & {}-{} & y & & & = (2 - \sqrt{2})x \\ -x & {}+{} & 2y & {}-{} & z & = (2 - \sqrt{2})y \\ & & -y & {}+{} & 2z & = (2 - \sqrt{2})z \end{alignedat}\right.\]

Simplifying,

\begin{alignat}{3} \sqrt{2}x & {}-{} & y & & & = 0 \tag{a} \\ -x & {}+{} & \sqrt{2}y & {}-{} & z & = 0 \tag{b} \\ & & -y & {}+{} & \sqrt{2}z &= 0 \tag{c} \end{alignat}

We conclude the following:

\[\text{(c)} \implies y = \sqrt{2}z \qquad \text{(a)} \implies y = \sqrt{2}x\]

These two relations give \(x=z\), then \(\text{(b)} \implies -x+2x-x=0\).

The last equation gives us no information; it simply states that \(0=0\).

Therefore, \(\underline{x}=\begin{bmatrix} x\\ \sqrt{2}x\\ x\end{bmatrix}) for any \(x\neq 0\) (otherwise we would have the trivial solution). So the eigenvectors corresponding to eigenvalue \(\lambda =2-\sqrt{2}\) are all proportional to \(\begin{bmatrix} 1\\ \sqrt{2}\\ 1\end{bmatrix}\).

In normalised form we have an eigenvector \(\dfrac{1}{2}\left[\begin{array}{c}1\\\sqrt{2}\\ 1\end{array}\right].\)

Case 2: \(\lambda = 2\)

Here \(A\underline{x}=2\underline{x}\) implies

\[\left[\begin{array}{rrr}2&-1&0\\ -1&2&-1\\ 0&-1&2\end{array}\right]\left[\begin{array}{c}x\\ y\\ z\end{array}\right]=2\left[\begin{array}{c} x\\ y\\ z\end{array}\right]\]

i.e.,

\[\left\{\begin{alignedat}{3} 2x & {}-{} & y & & & = 2x \\ -x & {}+{} & 2y & {}-{} & z & = 2y \\ & & -y & {}+{} & 2z & = 2z \end{alignedat}\right.\]

After simplifying the equations become:

\begin{alignat}{3} & & -y & & &= 0 \tag{a} \\ -x & & & {}-{} & z &= 0 \tag{b} \\ & & -y & & & = 0 \tag{c} \end{alignat}

\(\text{(a)}\) and \(\text{(c)}\) imply \(y=0\), while \(\text{(b)}\) implies \(x=-z\).

Therefore, the eigenvectors have the form \(\left[\begin{array}{r}x\\0\\-x\end{array}\right]\) for any \(x\ne 0\).

That is, eigenvectors corresponding to \(\lambda=2\) are all proportional to \(\left[\begin{array}{r}1\\0\\-1\end{array}\right]\).

In normalised form we have an eigenvector \(\dfrac{1}{\sqrt{2}}\left[\begin{array}{r}1\\0\\-1\end{array}\right]\).

Case 3: \(\lambda = 2 + \sqrt{2}\) (largest eigenvalue)

Proceeding along similar lines to cases 1 and 2 above we find that the eigenvectors corresponding to \(\lambda=2+\sqrt{2}\) are each proportional to \(\left[\begin{array}{c}1\\-\sqrt{2}\\ 1\end{array}\right]\) with normalised eigenvector \(\dfrac{1}{2}\left[\begin{array}{c}1\\ -\sqrt{2}\\ 1\end{array}\right]\).

In conclusion, the matrix \(A %=\left[\begin{array}{rrr}2&-1&0\\ -1&2&-1\\ %0&-1&2\end{array}\right]\) has three distinct eigenvalues:

\[\lambda_1=2-\sqrt{2},\qquad \lambda_2=2\qquad \lambda_3=2+\sqrt{2}\]

and three corresponding normalised eigenvectors:

\[\underline{x}_1=\dfrac{1}{2}\left[\begin{array}{c}1\\ \sqrt{2}\\ 1\end{array}\right],\qquad \underline{x}_2=\dfrac{1}{\sqrt{2}}\left[\begin{array}{r}1\\ 0\\-1\end{array}\right],\qquad \underline{x}_3=\dfrac{1}{2}\left[\begin{array}{c}1\\-\sqrt{2}\\ 1\end{array}\right]\]

Note that even when normalised we have a choice of vector, namely \(\underline{v}\) and \(-\underline{v}\).

Paper Exercise

If \(\lambda_1,\lambda_2,\ldots \lambda_n\) are the eigenvalues of a matrix \(A,\), get a piece of paper and prove the following:

1. \(A^T\) has eigenvalues \(\lambda_1,\lambda_2,\ldots \lambda_n\).
2. If \(A\) is upper triangular, then its eigenvalues are exactly the main diagonal entries.
3. The inverse matrix \(A^{-1}\) has eigenvalues \({{1\over \lambda_1}},\ {{1\over \lambda_2}},\ldots {{1\over \lambda_n}}\).
4. The matrix \(A-kI\) has eigenvalues \(\lambda_1-k,\lambda_2-k,\ldots \lambda_n-k\).
5. (Harder) The matrix \(A^2\) has eigenvalues \(\lambda_1^2,\lambda_2^2,\ldots \lambda_n^2\).
6. (Harder) The matrix \(A^k\) (\(k\) a non-negative integer) has eigenvalues \(\lambda_1^k,\lambda_2^k,\ldots \lambda_n^k.\)

N.B. Some of these results are useful in the numerical calculation of eigenvalues.

We give the solution on the next page. Please try to complete the problem before referencing the solution.

Exercise

If \(\lambda_1,\lambda_2,\ldots \lambda_n\) are the eigenvalues of a matrix \(A,\), get a piece of paper and prove the following:

1. \(A^T\) has eigenvalues \(\lambda_1,\lambda_2,\ldots \lambda_n\).
2. If \(A\) is upper triangular, then its eigenvalues are exactly the main diagonal entries.
3. The inverse matrix \(A^{-1}\) has eigenvalues \({{1\over \lambda_1}},\ {{1\over \lambda_2}},\ldots {{1\over \lambda_n}}\).
4. The matrix \(A-kI\) has eigenvalues \(\lambda_1-k,\lambda_2-k,\ldots \lambda_n-k\).
5. (Harder) The matrix \(A^2\) has eigenvalues \(\lambda_1^2,\lambda_2^2,\ldots \lambda_n^2\).
6. (Harder) The matrix \(A^k\) (\(k\) a non-negative integer) has eigenvalues \(\lambda_1^k,\lambda_2^k,\ldots \lambda_n^k.\)

N.B. Some of these results are useful in the numerical calculation of eigenvalues.

We give the solution below. Please try to complete the problem before referencing the solution.

Solution

1. Using the property that for any square matrix \(A\),
\[\text{det}(A) = \text{det}(A^T),\]
we see that if
\[ \text{ det}(A-\lambda I)=0\]
then
\[ \text{ det}(A-\lambda I)^T=0.\]
This immediately tells us that \(\text{ det}(A^T-\lambda I)=0\) which shows that \(\lambda\) is also an eigenvalue of \(A^T\).
2. Here simply write down a typical upper triangular matrix \(U\) which has terms on the leading diagonal \(u_{11}, u_{22},\dots , u_{nn}\) and above it. Then construct \((U-\lambda I)\). Finally imagine how you would then obtain \(\text{ det}(U-\lambda I)=0\).

You should see that the determinant is obtained by multiplying together those terms on the leading diagonal. Here the characteristic equation is:
\[(u_{11}-\lambda)(u_{22}-\lambda)\dots (u_{nn}-\lambda)=0.\]
This polynomial has the obvious roots
\[\lambda_1=u_{11},\ \lambda_2=u_{22},\ \dots , \lambda_n=u_{nn}.\]
3. Here we begin with the usual eigenvalue problem \(A\underline{x}=\lambda \underline{x}\).
If \(A\) has an inverse \(A^{-1}\) we can multiply both sides by \(A^{-1}\) on the left to give
\[A^{-1}(A\underline{x})=A^{-1}\lambda \underline{x}\]
 which gives
\[\underline{x}=\lambda A^{-1}\underline{x}\]
or, dividing through by the scalar \(\lambda\) we get
\[ A^{-1}\underline{x}=\frac{1}{\lambda}\underline{x}\]
which shows that if \(\lambda\) and \(\underline{x}\) are respectively eigenvalue and eigenvector of \(A\) then \(\lambda^{-1}\) and \(\underline{x}\) are respectively eigenvalue and eigenvector of \(A^{-1}\).

As an example consider \(A=\left[\begin{array}{cc}2&3\\ 3&2\end{array}\right]\). This matrix has eigenvalues
\[\lambda_1=-1,\quad \lambda_2=5\]
with corresponding eigenvectors
\[\underline{x}_1=\left[ \begin{array}{r} 1\\-1 \end{array} \right]\quad \text{and} \quad \underline{x}_2=\left[ \begin{array}{c}1\\1\end{array} \right].\]

The reader should verify (by direct multiplication) that \(A^{-1}=-\dfrac{1}{5}\left[\begin{array}{cc}\hfill 2&\hfill -3\\ \hfill -3&\hfill 2\end{array}\right]\) has eigenvalues
\[\lambda_1=-1\quad \text{and} \quad \lambda_2=\frac{1}{5} \]
with respective eigenvectors
\[\underline{x}_1=\left[ \begin{array}{r} 1\\-1 \end{array} \right]\quad \text{and} \quad \underline{x}_2=\left[ \begin{array}{c}1\\1\end{array} \right].\]
4. Proved in similar way to the proof outlined in (c).
5. Proved in similar way to the proof outlined in (c).
6. Proved in similar way to the proof outlined in (c).