Projections and orthonormal bases

Yehonatan Sella

Consider the following situation. Given a line L ⊂ R2 , and given any other vector
v ∈ R2 , we can “project” the vector v onto the line L by dropping a perpendicular
onto the line. The result is a new vector, which we can denote by P (v), that lies
along the line L, as in the picture below:

This is the basis of the idea of a projection. We want to generalize this idea and
consider projection onto an arbitrary subspace W of a vector space V .
In fact, in the above situation we had to drop a perpendicular onto the line L,
which implicitly relies on an inner product since the notion of being perpendicular,
a.k.a. orthogonal, is defined using the inner product. A general vector space does
not come equipped with an inner product so in fact “projection onto W ” is not

1
well-defined. In order to define a projection in the absence of an inner product,
we need to not only specify the subspace W that we are projecting onto, but we
also need to specify a complementary subspace W 0 along which we are projecting.
For the sake of clarity, we split up our discussion of projections into two cases:
vector spaces without inner products, and vector spaces equipped with inner prod-
ucts.

Projections in the absence of an inner product

Recall the following definition.

Definition: Let V be a vector space and let W, W 0 ⊂ V be subspaces. Then

we write V = W ⊕ W 0 if V = W + W 0 and W ∩ W 0 = {0}. We can describe this
situation by saying V is the direct sum of W and W 0 . We also say that W 0 is a
complement of W in V .

Note that a given subspace W ⊂ V can have many different complements W 0 .

Indeed, if V = R2 and W ⊂ V is a line, then any other line W 0 ⊂ V will be a
complement of W , satisfying W ⊕ W 0 = V .
In an earlier homework problem (2.2.9), we showed that, if V = W ⊕ W 0 , then
any vector v ∈ V can be decomposed uniquely as a sum of vectors in W and W 0 ,
that is, v = w + w0 for a unique choice of w ∈ W and w0 ∈ W 0 . This allows us to
define the projection of V onto W along W 0 .

Definition: Suppose V = W ⊕ W 0 . Then we define the projection of V onto

W along W 0 to be the linear operator P : V → V given by the formula

P (v) = w where v = w + w0 with w ∈ W, w0 ∈ W 0

That is, we decompose v as a sum of vectors in W and W 0 and pick only the compo-
nent that lies in W . This is uniquely defined by the uniqueess of the decomposition.

Claim: Suppose P : V → V is the projection of V onto W along W 0 . Then

a) If w ∈ W , then P (w) = w.
b) If w0 ∈ W 0 , then P (w0 ) = 0.
c) W = im(P ) and W 0 = ker(P ).

Proof: a) Indeed, given any w ∈ W its decomposition into W and W 0 is sim-

ply w = w + 0 so we have P (w) = w.
b) Similarly, if w0 ∈ W 0 then its decomposition into W and W 0 is simply
w0 = 0 + w0 so P (w0 ) = 0.

2
 c) By definition, im(P ) ⊂ W . On the other hand W ⊂ im(P ) by part a).
Similarly, W 0 ⊂ ker(P ) by part b). On the other hand, ker(P ) ⊂ W 0 since if
v ∈ ker(P ), we have P (v) = 0 so the decomposition of v into W and W 0 must be
of the form v = 0 + w0 , so v = w0 ∈ W 0 .

A consequence of the above claim is that, given a projection P : V → V , we

must have V = ker(P ) ⊕ im(P ).

There is a simple formula describing projection operators.

Claim: a linear operator T : V → V is a projection if and only if T 2 = T .

Proof: We first show that any projection P : V → V satisfies the equation P 2 = P .

Indeed, say P is the projection onto W = im(P ) along W 0 = ker(P ). Given any
v ∈ V , we have P (v) ∈ W . But by part a) of the above claim, P (w) = w for
w ∈ W so in particular P (P (v)) = P (v), so P 2 (v) = P (V ) as desired.

On the other hand, suppose T : V → V is any operator satisfying T 2 = T .

We want to show that T is the projection onto im(T ) along ker(T ). We leave this
as an exercise.
Hint: We first need to show that V = im(T ) ⊕ ker(T ). I in fact proved this in
the solution to 3.4.10 in HW 5. Look at that proof if you’re interested.

Example: Let L ⊂ R2 be the line given by y = x and let L0 ⊂ R2 be the

line given by y = 2x. We compute the projection P of R2 onto L along L0 .

The matrix of a projection is always very easy with respect to a certain spe-
cial basis. Indeed, let v1 be a nonzero vector
  in L and let v2 be a nonzero vector
1 1
in L0 . For example, we can choose v1 = and v2 = .
1 2
Since v1 ∈ L and we are projecting onto L, we have P (v1 ) = v1 . On the other
hand, since v2 ∈ L0 and we are projecting along 0
 L , we
 have P (v2 ) = 0. Thus the
1 0
matrix for P in the basis B = v1 , v2 is [P ]B = .
0 0
To figure out the matrix
 for P in the standard basis, we just need to do a
1 1
change of basis. Let S = . Then S is the change-of-basis matrix from B-
1 2  
2 −1
coordinates to standard coordinates. We can calculate S −1 = . Thus
−1 1

3
the matrix of P in standard coordinates is
     
−1 1 1 1 0 2 −1 2 −1
A = S[P ]B S = =
1 2 0 0 −1 1 2 −1

Projections in inner product spaces

In the previous section, we saw that if W ⊂ V , in order to define the projection of
V onto W we must specify a complement W 0 to W along which we project. We
also saw that there are many possible choices of a complement In this section, we
see that if our vector space V is equipped with an inner product, then in fact there
is a canonical choice of a complement to W , called its orthogonal complement. The
projection along the orthogonal complement of W will be called the orthogonal
projection onto W .

Definition: Let V be an inner product space and let W ⊂ V . Then we define

W ⊥ = {v ∈ V : hv, wi = 0 for all w ∈ W }. That is, W ⊥ , called the orthogonal
complement of W , consists of vectors which are orthogonal to every vector in W .

You can check that W ⊥ is indeed a subspace.

In order to compute orthogonal projection, we need the concept of an orthonormal

basis.

Definition: Let V be an inner product space. Then a basis B is orthonormal

if hv, vi = 1 for all v ∈ B and hv, v 0 i = 0 for any two distinct vectors v, v 0 ∈ B. In
other words, a basis is orthonormal if every basis vector has norm (length) 1 and
any two distinct basis vectors are orthogonal.
   
1 0
Example: a) The standard basis , of R2 is orthonormal.
   0  1
1 −1
b) Consider the basis , of R2 . The two vectors are orthogonal to
1 1
each other, but they do not have norm 1, so they do not form an "orthonormal # " basis.#
√1 − √1
2 2
To get an orthonormal basis, we divide by the norm, getting: √1
, √1
,
2 2
which is an orthonormal basis.

4
Just as a basis forms a tool for computing things in vector spaces, an orthonormal
basis forms a tool for computing things in an inner product space.
(Philosophical remark: just as a basis gives us an isomorphism between a vector
space V and F n , an orthonormal basis gives us an isomorphism between V and
F n which preserves inner products, where F n is equipped with the standard inner
product. So using orthonormal bases, we can translate any question about inner
products to the concrete setting of the standard inner product in F n ).
As an example, we show how an orthogonal basis can be used to compute
length.

Claim: Let V be an inner product space over R or C with orthonormal basis

B = v1 , . . . , vn . Let v ∈ V and let [v]B = (c1 , . . . , cn ). Then the length of v is given
by ||v||2 = |c1 |2 + · · · + |cn |2 .
Proof: By definition of coordinates, we can write v = c1 v1 + · · · + cn vn . Now
we have
X X
||v||2 = hv, vi = hc1 v1 +· · ·+cn vn , c1 v1 +· · ·+cn vn i = hci vi , cj vj i = ci c¯j hvi , vj i
i,j i,j

Now, if i 6= j, then hvi , vj i = 0 by definition of orthonormal basis so these terms

disappear. If i = j, then hvi , vj i = 1. So the above expression simplifies to
X X
||v||2 = ci c¯i = |ci |2
i i

as desired

Remark: The above computation reduces to finding the coordinates [v]B =

(c1 , . . . , cn ) of v in the orthonormal basis. These coordinates in turn can be com-
puted by the formula ci = hv, vi i. That is, the coordinate of vi in v is precisely the
inner product of v with vi .

Orthonormal bases similarly help us compute orthogonal projection.

Claim: Let V be an inner product space and let W ⊂ V be finite-dimensional.

Then
a) W ⊥ is indeed a complement of W , that is, V = W ⊕ W ⊥ .
b) If w1 , . . . , wn is an orthonormal basis of W , then the projection P : V → V
onto W along W ⊥ is given by the formula

P (v) = hv, w1 iw1 + · · · + hv, wn iwn

Proof: We prove a) and b) together.

5
 We first show that v − P (v) ∈ W ⊥ . Indeed, it suffices to show v − P (v) is
orthogonal to each basis element wi of W . Since wi form an orthonormal basis
of W , it follows from the above remark that hP (v), wi i is the coordinate of wi in
P (v), which by definition equals hv, wi i. Using the identity hP (v), wi i = hv, wi i,
we compute:

h(v − P (v)), wi i = hv, wi i − hP (v), wi i = hv, wi i − hv, wi i = 0

as desired.
Clearly P (v) ∈ W . It follows that

v = P (v) + (v − P (v))

is a decomposition of v as a sum of vectors in W and W ⊥ , so indeed V = W + W ⊥ .

On the other hand, we show that W ∩W ⊥ = {0}. Indeed, suppose v ∈ W ∩W ⊥ .
Since v ∈ W ⊥ , hv, wi = 0 for all w ∈ W . In particular, since v ∈ W , we have
hv, vi = 0. So v = 0, as desired.
Thus V = W ⊕ W ⊥ .

Since v = P (v) + (v − P (v)) is the decomposition of v into W and W ⊥ , it follows

that the projection of v onto W along W ⊥ is exactly P (v).

The transformation P above, which projects onto W along the orthogonal com-
plement W ⊥ , is called the orthogonal projection onto W .

The above formula for the orthogonal projection P can be used to explicitly com-
pute orthogonal projection. But first we have to be able to find an orthonormal
basis for the given subspace W . This can be done using the process of Gram-
Schmidt, which we don’t cover in these notes.

Finally, there is an important inequality concerning orthogonal projections.

Claim: Let W be a finite-dimensional subspace of an inner product space V .

Let P : V → V be the orthogonal projection of V onto W . Then, for any v ∈ V ,
||P (v)|| ≤ ||v||, and equality is attained only if P (v) ∈ W .

Proof: Write v = P (v) + w0 , where w0 = v − P (v) ∈ W ⊥ . In particular, w0 is

orthogonal to P (v), so we compute:

||v||2 = hv, vi = hP (v) + w0 , P (v) + w0 i

= hP (v), P (v)i + hP (v), w0 i + hw0 , P (v) + hw0 , w0 i = ||P (v)||2 + ||w0 ||2

6
But lengths are non-negative real numbers, so ||w0 ||2 ≥ 0 and

||v||2 = ||P (v)||2 + ||w0 ||2 ≥ ||P (v)||2

as desired. Furthermore, we see that equality occurs only when ||w0 || = 0, which
means v = P (v) ∈ W .

Note: The above inequality can be rephrased in terms of an orthonormal ba-

sis. If w1 , . . . , wn is an orthonormal basis of W , then P (v) follows the formula
P (v) = hv, w1 iw1 + · · · + hv, wn iwn . Combining this with the earlier formula for
length, we see:
||P (v)||2 = |hv, w1 i|2 + · · · + |hv, wn i|2
We obtain Bessel’s inequality:

|hv, w1 i|2 + · · · + |hv, wn i|2 ≤ ||v||2

whenever w1 , . . . , wn are orthonormal vectors (if they were simply orthogonal, we

would have to adjust by dividing by length).