Title: Default for 22.3 Repeated Eigenvalues and Symmetric Matrices
Course: HELM Workbooks 2023-2024

Exercise

Let \(A = [value]\).

1. The eigenvalues of \(A\) (repeated according to multiplicities) are
.
2. Find the eigenvectors of \(A\).
Enter the eigenvectors as columns into the following matrix.
If there are fewer than three independent eigenvectors then fill the rest of the columns by zeroes:
3. Can \(A\) be diagonalized?
Feedback: 1. The characteristic equation of \(A\) is \(\det( A-\lambda I ) = [value] = 0\), i.e. \(0 = [value] = [value]\) which gives \(\lambda=[value]\), \(\lambda=[value]\), \(\lambda=[value]\). 2. For \(\lambda=[value]\) the associated eigenvectors satisfy \[[value][value] = [value].\] The general solution is \(x=[value]\), \(y=[value]\) and \(z=[value]\), where \(\alpha\) is arbitrary. Thus an eigenvector is \(X=[value]\), where \(\alpha\) is arbitrary, \(\alpha\neq0\). For the repeated eigenvalue \(\lambda=[value]\) we must solve \(AY = [value]Y\) for the eigenvector \(Y\): \[[value][value] = [value].\] The general solution of this system is \(x=[value]\), \(y=[value]\) and \(z=[value]\), where \(\beta\) is arbitrary, so the eigenvectors are of the form \(Y=\beta[value]\), where \(\beta\neq0\) is arbitrary. \(X\) and \(Y\) are certainly linearly independent (as we would expect since they correspond to distinct eigenvalues). However, there is only one independent eigenvector of the form \(Y\) corresponding to the repeated eigenvalue \([value]\). 3. The conclusion is that since \(A\) is \(3\times 3\) and we can only obtain two linearly independent eigenvectors, \(A\) cannot be diagonalized.

Matrices with repeated eigenvalues

So far we have considered the diagonalization of matrices with distinct (i.e. non-repeated) eigenvalues. We have accomplished this by the use of a non-singular modal matrix \(P\) (i.e. one where \(\det P\neq 0\) and hence the inverse \(P^{-1}\) exists). We now want to discuss briefly the case of a matrix \(A\) with at least one pair of repeated eigenvalues. We shall see that for some such matrices diagonalization is possible but for others it is not.

The crucial question is whether we can form a non-singular modal matrix \(P\) with the eigenvectors of \(A\) as its columns.

Example

Consider the matrix \[ A=\left[\begin{array}{rc}1&0\\ -4&1\end{array}\right]\] which has characteristic equation \[ \det (A-\lambda I)=(1-\lambda)(1-\lambda)=0.\] So the only eigenvalue is \(1\) which is repeated or, more formally, has multiplicity 2.

To obtain eigenvectors of \(A\) corresponding to \(\lambda=1\) we proceed as usual and solve \[ AX=1X\] or \[ \left[\begin{array}{rc}1&0\\ -4&1\end{array}\right]\left[\begin{array}{c} x\\ y\end{array}\right]= \left[\begin{array}{c}x\\ y\end{array}\right]\] implying \[ x=x\qquad \text{and} \qquad -4x+y=y\] from which \(x=0\) and \(y\) is arbitrary.

Thus possible eigenvectors are \(\qquad \left[\begin{array}{r}0\\-1\end{array}\right],\quad \left[\begin{array}{c}0\\ 1\end{array}\right],\quad \left[\begin{array}{c}0\\ 2\end{array}\right],\quad \left[\begin{array}{c}0\\ 3\end{array}\right]\quad\dots\)

However, if we attempt to form a modal matrix \(P\) from any two of these eigenvectors, e.g. \(\left[\begin{array}{r}0\\-1\end{array}\right]\) and \(\left[\begin{array}{c}0\\1\end{array}\right]\) then the resulting matrix \(P=\left[\begin{array}{rc}0&0\\ -1&1\end{array}\right]\) has zero determinant.

Thus \(P^{-1}\) does not exist and the similarity transformation \(P^{-1}AP\) that we have used previously to diagonalize a matrix is not possible here. The essential point, at a slightly deeper level, is that the columns of \(P\) in this case are not linearly independent since \[ \left[\begin{array}{r}0\\ -1\end{array}\right]=(-1)\left[\begin{array}{c}0\\ 1\end{array}\right]\] i.e. one is a multiple of the other.

This situation is to be contrasted with that of a matrix with non-repeated eigenvalues. Earlier, for example, we showed that the matrix \[ A=\left[\begin{array}{cc}2&3\\ 3&2\end{array}\right]\] has the non-repeated eigenvalues \(\lambda_1=-1,\ \lambda_2=5\) with associated eigenvectors \[ X_1=\left[\begin{array}{r}1\\-1\end{array}\right]\qquad\qquad X_2=\left[\begin{array}{c}1\\ 1\end{array}\right].\] These two eigenvectors are linearly independent, since \(\left[\begin{array}{r}1\\-1\end{array}\right]\neq k\ \left[\begin{array}{c}1\\1\end{array}\right]\) for any value of \(k\ne 0\).

Here the modal matrix \[ P=\left[\begin{array}{rc}1&1\\ -1&1\end{array}\right]\] has linearly independent columns: so that \(\det P\neq 0\) and \(P^{-1}\) exists.

The general result, illustrated by this example, is given in the following Key Point.

Key Point

Eigenvectors corresponding to distinct eigenvalues are always linearly independent.

It follows from this that we can always diagonalize an \(n\times n\) matrix with \(n\) distinct eigenvalues since it will possess \(n\) linearly independent eigenvectors. We can then use these as the columns of \(P\), secure in the knowledge that these columns will be linearly independent and hence \(P^{-1}\) will exist. It follows, in considering the case of repeated eigenvalues, that the key problem is whether or not there are still \(n\) linearly independent eigenvectors for an \(n\times n\) matrix.

We shall now consider two \(3\times 3\) cases as illustrations.

Key Point

An \(n\times n\) matrix with repeated eigenvalues can be diagonalized provided we can obtain \(n\) linearly independent eigenvectors for it. This will be the case if, for each repeated eigenvalue \(\lambda_i\), of multiplicity \(m_i>1\), we can obtain \(m_i\) linearly independent eigenvectors.

Symmetric matrices

Symmetric matrices have a number of useful properties which we will investigate in this Section.

Calculating the eigenvalues of an \(n\times n\) matrix with real elements involves, in principle at least, solving an \(n^{\text{th}}\) order polynomial equation, a quadratic equation if \(n=2\), a cubic equation if \(n=3\), and so on. As is well known, such equations sometimes have only real solutions, but complex solutions (occurring as complex conjugate pairs) can also arise. This situation can therefore arise with the eigenvalues of matrices.

In particular any \(2\times 2\) matrix of the form \[ A=\left[\begin{array}{rc}a&b\\ -b&a\end{array}\right]\] has complex conjugate eigenvalues \(a\pm \mathrm{i} b\).

A \(3\times 3\) example of a matrix with some complex eigenvalues is \[ B=\left[\begin{array}{crr}1&-1&-1\\ 1&-1&0\\ 1&0&-1\end{array}\right].\] A straightforward calculation shows that the eigenvalues of \(B\) are \[ \lambda=-1 \text{ (real)}, \lambda =\pm \mathrm{i} \text{ (complex conjugates)}.\]

With symmetric matrices on the other hand, complex eigenvalues are not possible.

Key Point

The eigenvalues of a symmetric matrix with real elements are always real.

The general proof of this result in the above Key Point is beyond our scope, but a simple proof for symmetric \(2\times 2\) matrices is straightforward.

Let \(A=\left[\begin{array}{cc}a&b\\ b&c\end{array}\right]\) be any \(2\times 2\) symmetric matrix, \(a,\ b,\ c\) being real numbers.

The characteristic equation for \(A\) is \[ (a-\lambda)(c-\lambda)-b^2=0\] or, expanding: \[ \lambda^2-(a+c)\lambda+ac-b^2=0 \] from which \[ \lambda=\dfrac{(a+c)\pm\sqrt{(a+c)^2-4ac+4b^2}}{2}.\] The quantity under the square root sign can be treated as follows: \[ (a+c)^2-4ac+4b^2=a^2+c^2+2ac-4ac+b^2=(a-c)^2+4b^2\] which is always positive and hence \(\lambda\) cannot be complex.

Key Point

Two \(n\times 1\) column vectors \(X\) and \(Y\) are orthogonal if \(X^TY=\mathbf{0}\) or (equivalently) \(Y^TX=\mathbf{0}\).

General theory

The following proof that eigenvectors corresponding to distinct eigenvalues of a symmetric matrix are orthogonal is straightforward and you are encouraged to follow it through.

Let \(A\) be a symmetric \(n\times n\) matrix and let \(\lambda_1,\lambda_2\) be two distinct eigenvalues of \(A\) i.e. \(\lambda_1\neq \lambda_2\) with associated eigenvectors \(X,\ Y\) respectively. We have seen that \(\lambda_1\) and \(\lambda_2\) must be real since \(A\) is symmetric. Then \[ AX=\lambda_1X\qquad AY=\lambda_2 Y \tag{1}\] Transposing the first of the above equations gives \[ X^TA^T=\lambda_1X^T \tag{2}\] (Remember that for any two matrices the transpose of a product is the product of the transposes in reverse order.)

We now multiply both sides of (2) on the right by \(Y\) (as well as putting \(A^T=A\), since \(A\) is symmetric) to give: \[ X^TAY=\lambda_1X^TY \tag{3}\] But, using the second eigenvalue equation of (1), equation (3) becomes \[ X^T\lambda_2Y=\lambda_1X^TY\] or, since \(\lambda_2\) is just a number, \[ \lambda_2X^TY=\lambda_1 X^TY.\] Taking all terms to the same side and factorising gives \[ (\lambda_2-\lambda_1)X^TY=\mathbf{0}\] from which, since by assumption \(\lambda_1\neq \lambda_2\), we obtain the result \[ X^TY=\mathbf{0}\] and the orthogonality has been proved.

Key Point

The eigenvectors associated with distinct eigenvalues of a symmetric matrix are mutually orthogonal.

The reader familiar with the algebra of vectors will recall that for two vectors whose Cartesian forms are \[\underline{a}=a_x\underline{i}+a_y\underline{j}+a_z\underline{k}\qquad \underline{b}=b_x\underline{i}+b_y\underline{j}+b_z\underline{k}\] the scalar (or dot) product is \[ \underline{a}\cdot\underline{b}=a_xb_x+a_yb_y+a_zb_z.\] Furthermore, if \(\underline{a}\) and \(\underline{b}\) are mutually perpendicular then \(\underline{a}\cdot \underline{b}=0\). (The word ‘orthogonal’ is sometimes used instead of perpendicular in the case.) Our result, that two column vectors are orthogonal if \(X^TY=\mathbf{0}\), may thus be considered as a generalisation of the 3-dimensional result \(\underline{a}\cdot\underline{b}=0\).

Diagonalization of symmetric matrices

Recall from our earlier work that

1. We can always diagonalize a matrix with distinct eigenvalues (whether these are real or complex).
2. We can sometimes diagonalize a matrix with repeated eigenvalues. (The condition for this to be possible is that any eigenvalue of multiplicity \(m\) had to have associated with it \(m\) linearly independent eigenvectors.)

The situation with symmetric matrices is simpler. Basically we can diagonalize any symmetric matrix. To take the discussions further we first need the concept of an orthogonal matrix.

A square matrix \(A\) is said to be orthogonal if its inverse (if it exists) is equal to its transpose: \[A^{-1}=A^T \qquad \text{or, equivalently,}\qquad AA^T=A^TA=I.\]

Example

An important example of an orthogonal matrix is \[A=\left[\begin{array}{rc}\cos\phi&\sin\phi\\ -\sin\phi&\cos\phi\end{array}\right]\] which arises when we use matrices to describe rotations in a plane. \[\begin{aligned} AA^T&=\left[\begin{array}{rc}\cos\phi&\sin\phi\\ -\sin\phi&\cos\phi\end{array}\right]\left[\begin{array}{cr}\cos\phi&-\sin\phi \\ \sin\phi&\cos\phi\end{array}\right]\\ \\ &=\left[\begin{array}{cc}\cos^2\phi+\sin^{2}\phi&0\\ 0&\sin^2\phi+\cos^2\phi\end{array}\right]=\left[\begin{array}{cc}1&0\\ 0&1\end{array}\right]=I\qquad\qquad\qquad\qquad\qquad\quad\end{aligned}\] It is clear that \(A^TA=I\) also, so \(A\) is indeed orthogonal.

It can be shown, but we omit the details, that any \(2\times 2\) matrix which is orthogonal can be written in one of the two forms: \[\left[\begin{array}{rc}\cos\phi&\sin\phi\\ -\sin\phi&\cos\phi\end{array}\right]\qquad\text{or}\qquad \left[\begin{array}{cc}\cos\phi&-\sin\phi\\\sin\phi&\cos\phi\end{array}\right].\] If we look closely at either of these matrices we can see that

1. The two columns are mutually orthogonal e.g. for the first matrix we have \[\left[\cos\phi\quad -\sin\phi\right]\left[\begin{array}{c}\sin\phi\\ \cos\phi\end{array}\right]=\cos\phi\sin\phi-\sin\phi\cos\phi=0.\]
2. Each column has magnitude \(1\) (because \(\sqrt{\cos^2\phi+\sin^2\phi}=1)\).

Although we shall not prove it, these results are necessary and sufficient for any order square matrix to be orthogonal.

Key Point

A square matrix \(A\) is said to be orthogonal, if its inverse (if it exists) is equal to its transpose: \[A^{-1} = A^T\] or, equivalently, \[AA^T=A^TA=I.\] A square matrix is orthogonal if and only if its columns are mutually orthogonal and each column has unit magnitude.

The following is the key result of this Section.

Key Point

Any symmetric matrix \(A\) can be diagonalized using an orthogonal modal matrix \(P\) via the transformation \[ P^TAP = D = \left[\begin{array}{cccc}\lambda_1 & 0 & \cdots & 0\\ 0 & \lambda_2 & \cdots & 0\\ \vdots & \vdots & \ddots & \\ 0 & 0 & & \lambda_n\end{array}\right]. \] It follows that any \(n\times n\) symmetric matrix must possess \(n\) mutually orthogonal eigenvectors even if some of the eigenvalues are repeated.

It should be clear to the reader that the above Key Point is a very powerful result for any applications that involve diagonalization of a symmetric matrix. Further, if we do need to find the inverse of \(P\), then this is a trivial process since \(P^{-1}=P^T\) (the previous Key Point).

Hermitian matrices

In some applications, of which quantum mechanics is one, matrices with complex elements arise.

If \(A\) is such a matrix then the matrix \(\overline{A}^T\) is the conjugate transpose of \(A\), i.e. the complex conjugate of each element of \(A\) is taken as well as \(A\) being transposed. Thus if \[ A=\left[\begin{array}{cc}2+\mathrm{i}&2\\ 3\mathrm{i}&5-2\mathrm{i}\end{array}\right]\] then \[ \overline{A}^T=\left[\begin{array}{cc}2-\mathrm{i}&-3\mathrm{i}\\ 2&5+2\mathrm{i}\end{array}\right].\] An Hermitian matrix is one satisfying \[ \overline{A}^T=A.\] The matrix \(A\) above is clearly non-Hermitian. Indeed the most obvious feature of an Hermitian matrix is that its diagonal elements must be real. (Can you see why?) Thus \[ A=\left[\begin{array}{cc}6&4+\mathrm{i}\\ 4-\mathrm{i}&-2\end{array}\right]\] is Hermitian.

A \(3\times 3\) example of an Hermitian matrix is \[ A=\left[\begin{array}{ccc}1&\mathrm{i}&5-2\mathrm{i}\\ -\mathrm{i}&3&0\\ 5+2\mathrm{i}&0&2\end{array}\right].\] An Hermitian matrix is in fact a generalization of a symmetric matrix. The key property of an Hermitian matrix is the same as that of a real symmetric matrix – i.e. the eigenvalues are always real.

Exercise

The matrix \(A = [value]\) has eigenvalues \([value],[value],[value]\). The eigenvector corresponding to the eigenvalue \([value]\) is \(X=[value]\) or any multiple.

1. Investigate the eigenvectors associated to the repeated eigenvalue \([value]\).
Enter linearly independent eigenvectors as columns into the following matrix.
If there are less than two linearly independent eigenvectors, fill the rest of the columns by zeroes:
.
2. Can \(A\) be diagonalized?
Feedback: 1. We must solve \(AY = [value]Y\) for the required eigenvector, i.e. \[[value][value] = [value].\] Each equation simplifies to \([value]\). We have just one equation in three unknowns, so we can choose two values arbitrarily. For example (choosing some variables to be zero) we find solutions \([value]=[value]\), \([value]=[value]\) and \([value]=[value]\); and \([value]=[value]\), \([value]=[value]\) and \([value]=[value]\). These give rise to linearly independent eigenvectors \[ Y_1 = [value], \quad Y_2 = [value].\] 2. We can thus form a non-singular modal matrix \(P\) from \(Y_1\) and \(Y_2\) together with \(X\) (given): \[P = [value].\] We can then indeed diagonalize \(A\) through the transformation \[P^{-1}AP = D = [value].\]

Exercise

A matrix is called symmetric if is it symmetric across the main diagonal. In other words, transposing these matrices, i.e. interchanging their rows and columns, does not change them: \[A^T=A.\]

Select exactly the symmetric matrices:
Feedback: Let \(A = [value]\). Then \(A^T = [value] = A\), therefore \(A\) is symmetric. Let \(A = [value]\). Then \(A^T = [value] \neq A\), therefore \(A\) is not symmetric. Let \(A = [value]\). Then \(A^T = [value] = A\), therefore \(A\) is symmetric. Let \(A = [value]\). Then \(A^T = [value] \neq A\), therefore \(A\) is not symmetric. Let \(A = [value]\). Then \(A^T = [value] = A\), therefore \(A\) is symmetric. Let \(A = [value]\). Then \(A^T = [value] \neq A\), therefore \(A\) is not symmetric.

Exercise

Consider the non-symmetric matrix \[A=[value].\] Obtain the eigenvalues of \(A\):
Feedback: The characteristic equation of \(A\) is \[\det( A-\lambda I ) = [value] = 0,\] i.e. \[[value] = 0\] leading to \[[value]=0,\] giving eigenvalues \([value]\) and \([value]\). These are clearly complex conjugate.

Exercise

Consider the \(2\times 2\) symmetric matrix \[A=[value].\]

1. Find the eigenvalues of \(A\):
2. Find (independent) eigenvectors of \(A\): \(X=\)  and \(Y=\)
3. For the two eigenvectors found above, evaluate \(X^TY=\)
Feedback: 1. The characteristic equation for \(A\) is \[[value]=0\qquad\text{or}\qquad[value]=0,\] giving \(\lambda=[value]\) and \(\lambda=[value]\), both of which are of course real and also unequal (i.e. distinct). 2. For the larger eigenvalue \(\lambda=[value]\) the eigenvector \(X=[value]\) satisfies \[[value][value] = [value].\] Both equations simplify to \([value]=0\), so an eigenvector for \(\lambda=[value]\) is \(X=[value]\) or any non-zero multiple of it. For \(\lambda=[value]\) the associated eigenvectors satisfy \([value]=0\) (from both equations), so an eigenvector is \(Y=[value]\) or any non-zero multiple. 3. We compute \[X^TY = [value][value] = [value] = [value] = \mathbf{0}.\] The relation \(X^TY = \mathbf{0}\) means that \(X\) and \(Y\) are orthogonal.

Exercise

Consider the matrix \[A=[value],\] which is, we emphasize, symmetric. Its eigenvalues are \([value], [value], [value]\), and the eigenvectors are, respectively, \[X=[value],\qquad Y=[value],\qquad Z=[value]\] (or, as usual, any non-zero multiple of them).

Compute the following quantities:

\(X^TY=\)

\(Y^TZ=\)

\(Z^TX=\)
Feedback: \[X^TY = [value][value] = [value] = [value] = \mathbf{0}\] \[Y^TZ = [value][value] = [value] = [value] = \mathbf{0}\] \[Z^TX = [value][value] = [value] = [value] = \mathbf{0}\] Thus the three eigenvectors \(X,Y,Z\) are mutually orthogonal.

Exercise

Determine \([value]\) and \([value]\) so that the matrices \[A=[value],\qquad B=[value]\] are orthogonal. Then verify that \(A^TA=AA^T=I\) and \(B^TB=BB^T=I\) i.e. that \(A^T=A^{-1}\) and \(B^T=B^{-1}\).

\([value]=\)     \([value]=\)

(Note: a square root can be entered with sqrt(); for example \([value]\) would be entered as sqrt(5).)
Feedback: Matrix \(A\) Approach 1: For \(A\) to be orthogonal, its columns must be orthogonal to each other. This means \[[value] = [value][value] = [value] = [value]\] from which \([value]=[value]\). Approach 2: From the above description of \(2\times2\) orthogonal matrices, we know they are all of the form \([value]\) for some real number \([value]\). In particular, their diagonal entries are the same, hence \([value]=[value]\). Matrix \(B\) The columns of the matrix \(B\) are orthogonal to each other for any choice of \([value]\). However each column of an orthogonal matrix has unit magnitude. For the second column of \(B\), this leads to \[1^2=[value]^2+[value]^2+[value]^2 = [value], \] thus \([value] =\pm 1\).

Exercise

The symmetric matrix \[A=[value]\] has eigenvalues \([value],[value],[value]\) (i.e. eigenvalue \([value]\) is repeated with multiplicity \(2\)).

Associated with the non-repeated eigenvalue \([value]\) is an eigenvector \[X=[value]\] (or any non-zero multiple).

1. Normalize the eigenvector \(X\):
2. Investigate the eigenvectors associated with the repeated eigenvalue \([value]\): find a general form of these eigenvectors:
Feedback: 1. Normalizing \(X\) which has magnitude \(\sqrt{([value])^2+([value])^2+([value])^2} = [value]\) gives \[\frac{1}{[value]}[value] = [value].\] 2. The eigenvectors associated with \(\lambda=[value]\) satisfy \(AY=[value]Y\) which gives \[[value][value] = [value].\] All three equations simplify to \([value]\). Having a single equation with three unknowns, we choose two of the variables to be parameters \([value],[value]\) and thus obtain that \[Y=[value]\] where \([value],[value]\) are arbitrary (but not both simultaneously zero). Further discussion A certain amount of care is now require in the choice of \([value]\) and \([value]\) if we are to find an orthogonal modal matrix to diagonalize \(A\). For any choice \[X^TY = [value][value] = [value] = [value],\] so \(X\) and \(Y\) are orthogonal. (The normalization of \(X\) does not affect this.) However, we also need two orthogonal eigenvectors of the form \([value]\). One possibility to calculate them is as follows. Obtain one vector by choosing both parameters, for example \([value]=0\), \([value]=1\) which yields \(Y^{(1)}=[value]\). To calculate the second vector, we consider the equation expressing orthogonality, that is, \(Y^TY^{(1)}=[value]\), which simplifies to \([value]=0\). Choosing \([value]=1\), we calculate \([value]\), which leads to the vector \(Y^{(2)}=[value]\). After normalization, these become \(Y^{(1)}=[value]\) and \(Y^{(2)}=[value]\). Hence the matrix \[P = \left[\begin{matrix}X&\vdots&Y^{(1)}&\vdots &Y^{(2)}\end{matrix}\right] = [value]\] is orthogonal and diagonalizes \(A\): \[P^TAP = [value].\]