Lec 2: Mathematical Economics

Introduction to Spectral Theory

Sugata Bag

Delhi School of Economics

24th August 2012

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 1 / 17
 Introduction Eigenvalues-Eigenvectors

Denition: Eigen values & vectors

Denition
Let T : V ! V be a LT. A scalar is an eigenvalue of T if there exists a
non-zero vector v 2 V s.t. T (v ) = v . Where v is then an eigenvector
of T , corresponding to the eigenvalue .

Note (1) If v is an eigenvector corresponding to , then so is cv , for all

0
scalars c, because T (cv ) = cT (v ) = cv = (cv ). Moreover, if v , v are
eigenvectors corresponding to the same eigenvalue , then for scalars
0 0
c1 , c2 we have T (c1 v + c2 v ) = c1 T (v ) + c2 T (v )
0 0 0
= c1 v + c2 v = (c1 v + c2 v ). So c1 v + c2 v is also an eigenvector
corresponding to . That is, the set of all eigenvectors corresponding to a
particular eigenvalue is a subspace. It (along with the zero vector) is called
the eigenspace corresponding to .

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 2 / 17
 Introduction Eigenvalues-Eigenvectors

Notions
Note (2) For a typical (n n ) matrix A and a typical (n 1) vector v ,
the vector Av is some point in <n (or C n , if A is a complex matrix). In
the eigenvalue problem, we are looking for vectors v such that Av lies on
the same line through the origin as v itself.
Example
(1). A matrix for which all non-zero vectors are eigenvectors: In n . For all
non-zero x 2 <n , Ix = 1x; all such x are eigenvectors corresponding to the
eigenvalue 1.
(2) Using the appropriate rotation matrix, rotate the earth through say
90o . Typically, vectors x 2 <3 will change direction on rotation. But
vectors lying on the axis of rotation will not change direction. For such
vectors v , Av = 1v .

The set of all eigenvalues of a LT T is called the spectrum of T , and

denoted (T ).
[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 3 / 17
 Introduction Eigenvalues-Eigenvectors

Characteristic Equation
T (v ) = v ) (T I )(v ) = 0. That is, eigenvectors v corresponding to
the eigenvalue comprise the Nullspace of the LT T I (along with the
zero vector). In fact the nullspace N (T I ) is called the eigenspace
w.r.t. .
For there to be a non-zero solution v to (T I )(v ) = 0, we need
Nullity (T I ) > 0, or Rank (T I ) < dim (V ). Let the matrix An n
represent T . So we have Rank (A I ) < n, which implies the
determinant det (A I ) = 0. det (A I ) is just
0 1
a11 a12 ... a1n
B a21 a22 . . . a2n C
B C
det B .. .. .. C
@ . ... . . A
an1 ... . . . ann
So det (A I ) = 0 is nth -degree
polynomial and can be written as -
f () = j(A I )j = ( ) + bn 1 ( )n 1 + ... + b1 ( )1 + b0 ... (1)
n

or in factor form, f () = (1 )(2 )...(n ) .... (2)

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 4 / 17
 Introduction Eigenvalues-Eigenvectors

Characteristic Equation contd...

So det (A I ) = 0 has n solutions (roots), possibly repeated, and

possibly complex. If is a complex root of the equation, its conjugate is
also a complex root.
Comparison of Eq1 and Eq2 gives us -
n
bn 1 = i = 1 + 2 + ... + n
i =1
n
bn 2 = i j = 1 2 + ... + 1 n + 2 3 ... + 2 n
j >i
+.... + n 1 n
n
bn r = i j k [each term is product of r of the i ]
k >j >i
..
.
b0 = 1 2 3 ....n

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 5 / 17
 Introduction Eigenvalues-Eigenvectors

Examples
The characteristic polynomial for a second-order matrix A = jjaij jj is
a11 a12
f () = = (a11 )(a22 ) a12 a21
a21 a22
= ( )2 + (a11 + a22 )( ) + a11 a22 a12 a21
= ( ) 2 + b1 ( ) + b0
hence, b1 = a11 + a22 , b0 = a11 a22 a12 + a21 = jAj
Two roots of f () = 0 are -
= 21 f(a11 + a22 ) [(a11 + a22 )2 4jAj]1/2 g
if 1 and 2 are the two roots then 1 2 = jAj = b0 and
1 + 2 = a11 + a22 = b1
Recall: A quadratic eqn in can be as: 2 trace (A) + det (A) = 0.
Note: The tsm of the diagonal elements of a square matrix A (called
trace (A)) equals the sum of eigen values ni=1 i . And det (A) = ni=1 i ,
the product of eigen values.
[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 6 / 17
 Introduction Eigenvalues-Eigenvectors

Examples

Example 1. Let A2 2 have rows (1, 2) and (8, 1).

det (A I ) = 2 2 15 = 0 has solutions 1 = 5, 2 = 3.
Eigenvectors for 1 :

1 5 2 x1 0
=
8 1 5 x2 0
The eigenspace is 1-dimensional; Rank (A 5I ) = 1. Its
f(x1 , x2 )j 4x1 + 2x2 = 0g. If we choose x1 = 1, we solve and get
x2 = 2, so x = (1, 2)T is an eigenvector corresponding to = 5.
Similarly, (1, 2)T is an eigenvector corresponding to 2 = 3.

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 7 / 17
 Introduction Eigenvalues-Eigenvectors

Examples

Example
A2 2 has rows (1, 2) and (p 2, 1). det (A I ) = 2 2 + 5 = 0
implies the roots equal 2 24 20 . So 1 = 1 + 2i and 2 = 1 .
Eigenvector corresponding to 1 :

2i 2 x1 0
(A (1 + 2i )I )x = =
2 2i x2 0
Rank (A 1 I ) = 1 (the 2nd row is i times the rst). Solving
2ix1 + 2x2 = 0 by setting x1 = 1, we get x2 = i. So (1, i )T is an
eigenvector corresponding to 1 . The conjugate of this vector, (1, i )T is
therefore an eigenvector corresponding to the conjugate root 2 (this is a
general result).

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 8 / 17
 Introduction Eigenvalues-Eigenvectors

Identical Spectrums
Can we replaced a linear transformation T with a matrix A representing
it? Indeed all matrices representing T in dierent bases are similar to each
other. If A and B are 2 such matrices, then A = SBS 1 for some
invertible (change of basis) matrix S.
Denition
Similarity: If there exists a nonsingular matrix S such that B = SAS 1,

the square matrices A and B are said to be similar.

Lemma
Similar matrices have an identical spectrum.

Proof: Let Av = v , so is an eigenvalue of A. If A = SBS 1 , then this

implies SBS 1 v = v . Premultiplying both sides by S 1 ,
BS 1 v = S 1 v = S 1 v . So is an eigenvalue of B (and S 1 v a
corresponding eigenvector).
[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 9 / 17
 Introduction Symmetric Matrix & Characteristic Value

Theorems: Symmetric Matrix

Theorem
The eigenvectors corresponding to dierent eigenvalues are orthogonal;
i.e., if xi , xj are eigen-vectors corresponding to eigenvalues i and j
respectively, then xi0 .xj = 0.

Proof.
By assumption, Axi = i xi and Axj = j xj . Thus xj 0Axi = i xj 0xi and
xj 0Axj = j xj 0xi . Subtracting and noting that xj 0Axi = xj 0Axj , we have
(i j )xj 0xi = 0, or xj 0xi = 0 , since i 6= j .If x 6= 0 is an eigenvector
of A, then any scalar multiple of x is also an eigenvector of A,
corresponding to the same eigenvalue.
Note: For convenience, we shall always assume that the eigenvectors of a
symmetric matrix are of unit length; denoted by ui ; and they are said to
have been normalized. A set of two or more normalized eigenvectors ui
corresponding to dierent eigenvalues of A satises the equation ui uj = ij
and are said to be orthonormal.
[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 10 / 17
 Introduction Symmetric Matrix & Characteristic Value

Example
p
2 2
Find the eigen values and eigen vectors of A = p .
2 1
p
2 2
The characteristic equationis jA I j = p = 2 3 = 0
2 1
thus the eigenvalues are 1 = 0 and 2 = 3.
To determine eigen vectors corresponding i , we must solve the set of
homogenous equation [A I ]x = 0. Let us take 1 = 0 rst, then the
set of equations becomes -
p p
2x1 + 2x2 = 0 and 2x1 + x2 = 0
Implies, x1 = p1 x2
2
If we wish to nd an eigen vector of unit length, we must require that
x12 + x22 = 1. Thus 32 x22 = 1; choosing the positive sqare root, we obtain -
p
x2 = 23 and x1 = p13 .
And the eigenvector of unit length corresponding to 1 is -
p
u1 = [ p13 , 23 ]
[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 11 / 17
 Introduction Symmetric Matrix & Characteristic Value

Example contd...

Note that u1 is not completely specied by the requirement of ju1 j = 1.

The vector u1 is also an eigen vector of length 1. However u1 and -u1
are not linearly independent. Only one linearly independent vector
corresponds to 1 .
For 2 = 3, the set of equations becomes -
p p
x1 + 2x2 = 0 and 2x1 2x2 = 0
p
implies, x1 = 2x2 .
If x12 + x22 = 1 then 3x22 = 1; taking poaitive square root we get -
p
x2 = p13 , x2 = 23
p
So u2 = [ 23 , p13 ].
Again there is only one linearly independent eigen vector that corresponds
to 2 ; it can easily be checked that u1 u2 = 0, in agrrement with the
theory.

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 12 / 17
 Introduction Diagonalization

Diagonal Form of Matrix

Theorem
Suppose An n has n LI eigenvectors. Then S 1 AS = D, where the
columns of S are the n eigenvectors and is the diagonal matrix whose
diagonals are the corresponding eigenvalues.

Proof.
Note S = (v1 , ..., vn ) is invertible (eigenvectors written as columns). Then
AS = A(v1 v2 ... vn ) = (Av1 Av2 ... Avn ) = (1 v1 2 v2 ... n vn )
0 10 0 ... 0
1
.. .. . . .. 1
B . . . . CB 0 ... 0 C
B 2 C
=B @ 1 2v v . . . vn CAB .. .. . . .. C = SD
.. .. . . . @ . . . . A
. . . .. 0 0 . . . n

Premultiplying both sides of AS = SD by S 1, S 1 AS = D.

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 13 / 17
 Introduction Diagonalization

When can we Diagonalize?

Lemma
Suppose An n has n distinct eigenvalues 1 , ..., n . Then a set of
eigenvectors v1 , ..., vn , vi corresponding to i is LI.

Proof.
(By induction). The statement is trivially true if n = 1. Suppose its true
for n = 1, 2, ..., k 1; i.e. if 1 , ..., k 1 are distinct and v1 , ..., vk 1 are
corresponding eigenvectors, then they are LI. Suppose k is an eigenvalue
distinct from 1 , ..., k 1 and vk is a corresponding eigenvector. For some
scalars c1 , ..., ck suppose that c1 v1 + ... + ck vk = 0 (1) .
So, A(c1 v1 + ... + ck vk ) = A0 = 0 . By linearity of A we get -
c1 Av1 + ... + ck Avk = 0 ) c1 1 v1 + ... + ck k vk = 0 (2).
Eq(2) minus k (Eq(1)) eliminates k , giving:
c1 (1 k )v1 + ... + ck 1 (k 1 k )vk 1 = 0. By LI of fv1 , ..., vk 1 g,
ci (i k ) = 0, 8i = 1, .., k 1. Since i 6= k , ci = 0, 8i = 1(1)k 1.
Plugging ci s in Eq(1), we get ck = 0 also, so that fv1 , .., vk g are LI.
[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 14 / 17
 Introduction Diagonalization

Why Diagonalize?

(1) Interpretation. If A is diagonalizable, then A = S S 1 . That is A and

are similar. Specically, suppose x = (x1 , ..., xn ) is a vector written in
terms of the standard basis. Suppose the LI eigenvectors fv1 , ..., vn g are
the new basis. If Sn n = (v1 v2 ... vn ), then w.r.t. the new basis, we
know that x is represented as y = S 1 x. Now, the matrix A carries x to
Ax. In terms of the new basis, Ax is represented as S 1 Ax. Since x = Sy ,
S 1 Ax = S 1 ASy = y . So wrt the new basis, the action of A on x
becomes the easier to visualize action of on y . just stretches the
coordinates of y by possibly dierent amounts.
(2) Uses - including more general uses of nding the spectrum (A): In
solving and interpreting solutions to dierential equations; also in some
specication testing in econometrics, interpreting quadratic forms,
frequency domain time series analysis etc.

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 15 / 17
 Introduction Diagonalization

Why Diagonalize

(3) If A is real, symmetric and full-ranked, then we can nd a set of

eigenvectors that are LI as well as orthogonal. Then the matrix S of
eigenvectors is called an orthogonal matrix. These are generally rotation
matrices; so A = S S 1 means the action of A is a composition of a
rotation followed by a stretch followed by another rotation. Examples of
such matrices: (i) the (X T X ) or (X X ) matrix discussed in the least
squares application; (ii) the Hessian D 2 f of a function f : <n ! <.

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 16 / 17
 Introduction Diagonalization

Examples

p
2 2
For the matrix p
2 1
And the eigenvector of unit length corresponding to 1 is -
p
u1 = [ p13 , 23 ]
p
u2 = [ 23 , p13 ]
p
1 1 2
S = ( u1 , u2 ) = p 3 [ p ]
2 1
p p p
1 2
S 0 AS = 31 p p2 2
p1 2
2 1 2 1 2 1
p p
2 2 0 3 2 0 0 1 0
= 13 p = =
2 1 0 3 0 3 0 2

[SB] (Delhi School of Economics) Introductory Math Econ 24th August 2012 17 / 17