6 Inner Product Spaces
6.1 Inner Product, Norm and Distance
Definition 6.1 An inner product on a real vector space V is a function that asso-
ciates a real number !u, v" with each pair of vectors u and v in V in such a way
that the following axioms are satisfied for all vectors u, v and z in V and all scalars
k.
(a) !u, v" = !u, v" (Symmetry axiom)
(b) !u + v, z" = !u, z" + !v, z" (Additive axiom)
(c) !ku, v" = k!u, v" (Homogeneity axiom)
(d) !v, v" ≥ 0 (Positivity axiom)
and if !v, v" = 0 if and only if v = 0.
A real vector space with an inner product is called a real inner product space.
Remarks.
1. Recall that for V = Rn , !u, v" = u · v = uT v (u, v ∈ V ) satisfies the
conditions above by Theorem 1.2. Hence it is an inner product defined in
Definition 6.1.
2. For all vectors u, v and z in V and all scalars k,
!z, u + v" = !z, u" + !z, v", !u, kv" = k!u, v".
3. !0, v" = 0 for all v ∈ V , as !0, v" = !00, v" = 0!0, v" = 0.
4. The definition above is only for real vector spaces, and the inequality in (d) is
the usual inequality among reals.
5. As for a complex vector space, a similar notion can be defined as in the next
definition. Then the following discussion is almost the same.
Definition 6.2 An inner product on a complex vector space V is a function that
associates a real number !u, v" with each pair of vectors u and v in V in such a
way that the following axioms are satisfied for all vectors u, v and z in V and all
scalars k (k ∈ C).
(a) !u, v" = !u, v" (Symmetry axiom)
(b) !u + v, z" = !u, z" + !v, z" (Additive axiom)
(c) !ku, v" = k!u, v" (Homogeneity axiom)
19
(d) !v, v" ≥ 0 (Positively axiom)
and if !v, v" = 0 if and only if v = 0.
A real vector space with an inner product is called a real inner product space.
Definition 6.3 If V is an inner product space, then the norm (or length) of a vector
u ∈ V is denoted by %u% and is defined by
%u% = !u, u"1/2 .
The distance between two points (vectors) u and v is denoted by d(u, v) and is
defined by
d(u, v) = %u − v%.
Example 6.1 [Excercise 6.1.30] Let A be an invertible n × n matrix. The following
defines an inner product on Rn .
!u, v" = Au · Av = (Au)T Av = uT AT Av.
Proof. The properties (a), (b), (c) are obvious. Clearly !u, u" = (Au) · (Au) ≥ 0
by the nonnegativity condition of the dot product in Rn . Moreover if !u, u" = 0
implies Au = 0. Since A is invertible, u = 0, and the condition (d) is proved.
Example 6.2 For A, B ∈ Matn (R) let
!A, B" = tr(AT B).
Then !A, B" defines an inner product on Matn (R).
Example 6.3 Let f = f (x) and g = g(x) be two functions on C[a, b], the set of all
continuous functions on [a, b]. Define
! b
!f , g" = f (x)g(x)dx.
a
Then !f , g" defines an inner product on C[a, b].
6.2 Properties of Inner Product Space
Theorem 6.1 ((6.2.1) Cauchy-Schwarz Inequality) If u and v are vectors in
a real inner product space, then
|!u, v"| ≤ %u%%v%.
Equality holds if and only if u and v are linearly dependent.
20
Proof. If u = 0, then there is nothing to prove. Assume that u )= 0. Let t be a
scalar. Then
%tu + v%2 = !tu + v, tu + v" = %u%2 t2 + 2!u, v"t + %v%2 .
Since the right hand side is a polynomial of degree exactly equal to 2 in t and the
left hand side is always nonnegative for all t ∈ R,
(!u, v")2 − %u%2 %v%2 ≤ 0.
Therefore we have the inequality.
Suppose the equality holds. Then there is a real number s such that %su+v% = 0.
Hence by the property (d) in Definition 6.1, su + v = 0 and u and v are linearly
independent. Conversely if u and v are linearly dependent, then either u = 0 or
there exists a real such that su + v = 0. Hence the discriminant above is 0 and we
have equality.
If u, v are nonzero vectors, then
!u, v"
−1 ≤ ≤ 1.
%u%%v%
Hence there is a unique angle θ such that
!u, v"
cos θ = and 0 ≤ θ ≤ π.
%u%%v%
Two vectors u, v ∈ V are said to be orthogonal when !u, v" = 0.
Theorem 6.2 (6.2.2) Let u and v be vectors in an inner product space V , and k
a scalar. Then:
(a) %u% ≥ 0.
(b) %u% = 0 if and only if u = 0.
(c) %ku% = |k|%u%.
(d) %u + v% ≤ %u% + %v%. (Triangle inequality)
Theorem 6.3 (6.2.3) Let u and v be vectors in an inner product space V , and k
a scalar. Then:
(a) d(u, v) ≥ 0.
(b) d(u, v) = 0 if and only if u = v.
(c) d(u, v) = d(v, u).
(d) d(u, v) ≤ d(u, w) + d(w, v). (Triangle inequality)
21
Theorem 6.4 ((6.2.4) Generalization Theorem of Pythagoras) If u and v
are vectors in an inner vector space, then
%u + v%2 = %u%2 + %v%2 ⇔ !u, v" = 0.
Exercise 6.1 [Quiz 6] Let A be an invertible m × n matrix. For u, v ∈ Rn let
!u, v" = Au · Av = (Au)T Av = uT AT Av.
1. Show that !u, v" satisfies the properties (a), (b) and (c) of an inner product
in Definition 6.1.
2. Show that if N (A) = {v ∈ Rn | Av = 0} = {0}, then !u, v" is an inner
product.
3. Show that if m > n, then !u, v" is not an inner product.
4. Suppose AT A is invertible. Show that m ≤ n.
22
