Math 114: Linear Algebra

Definition 1 Let V be a vector space over R. An inner product on V is a map

h, i : V V R

that takes two vectors u, v V as input and whose output is a scalar denoted by hu, vi satisfying the following
properties:

1. hu, vi = hu, vi for any u, v V

2. hu + v, wi = hu, wi + hv, wi for any u, v, w V
3. hcu, vi = chu, vi for any u, v V and c R
4. hu, ui 0 for any u V ; and hu, ui = 0 if and only if u = 0V .

A vector space that is equipped with an inner product is called an inner product space.

Examples 1: Standard inner products for some common vector spaces.

1. V = Rn hu, vi = uT v (dot product)

(a) Verify that the dot product defined above is an inner product on Rn .
   
1 3
(b) If u = and v = , compute hu, vi, hv, vi, hu + v, vi and hu, 2vi.
2 2

2. V = Rmn hA, Bi = trace(AT B) (where trace means sum of diagonal entries)

(a) Verify that the dot product defined above is an inner product on Rmn .

1 0 1 3
(b) If A = 0 2 and B = 2 0, compute hA, Bi, hA, Ai, hA + B, Bi and hA, 2Bi.
1 1 4 0
Z b
2
3. V = F ([a, b]) hf, gi = f (x)g(x) dx
ba a

(a) Verify that the dot product defined above is an inner product on F ([a, b]).
(b) Let a = and b = . If f1 (x) = sin(x) and f2 (x) = sin(2x), g1 (x) = cos(x) and g2 (x) = cos(2x),
compute hf1 , f2 i, hf1 , g1 i, hg2 , g2 i and hf1 , f1 + g1 i.

Definition 2 Let V be a vector space with inner product h, i. We define the norm/length of the vector u V by
p
||u|| = hu, ui.

Properties of || ||
1. ||cu|| = |c| ||u||

2. ||u + v|| ||u|| + ||v|| (Triangle inequality)

3. ||u|| 0 and ||u|| = 0 if and only if ||u|| = 0V
4. |u v| ||u|| + ||v|| (Cauchy-Schwarz inequality)

More terminologies:
If ||u|| = 1, then we say that u is a unit vector.
If u 6= 0, we define and denote the unit vector in the direction of u by
1
u = u.
||u||

Examples 2: From Examples 1, find the norm of the given vectors in V and the unit vector in the direction of each
vector.
Definition 3 Given a norm || || on V , we define the distance between two vectors u and v in V by

d(u, v) = ||u v||

Properties of d(, )

1. d(u, v) = d(v, u)
2. d(u, v) d(u, z) + d(z, v) (Triangle inequality)
3. d(u, v) 0 and d(u, v) = 0 if and only if u = v
Examples 3: In Example 1, find the distance bet. two given vectors in V , relative to the std inner product of V .
Examples 4: For the following examples, let h, i be
the standard
inner product in Rn .
1
1 1 2 3 1
Let w = 0 , x = 1, y = 5, z = 2 , a =
1
1 1 1 3
1
1. ||a|| = 5. d(x, y) =
2. || 2a||
6. d(w, y) =
3. a
4. d(w, x) = 7. d(z, z) =

Definition 4 Two vectors u and v are said to be orthogonal if and only if hu, vi = 0. In this case we write u v.
The angle between u and v is the angle [0, ] satisfying

hu, vi
cos =
||u||||v||

Examples 5:
   
1 3
1. Relative to the standard inner product on R2 , is u = orthogonal to v = ?
2 2

1 1
2. Relative to the standard inner product on R3 , is u = 0 orthogonal to v = 1?
1 1

Remark: If u v, then ||u + v||2 = ||u||2 + ||v||2 (Pythagorean Theorem)

Definition 5 A set S V is an orthogonal set if its elements are pairwise orthogonal. If, in addition, all the
vectors in the set are unit vectors, we say that the set is orthonormal.
Examples 6:
1. In the space V = R22 , show that the set S below is orthogonal relative to the standard inner product
       
0 1 1 0 0 1 1 0
S= , , ,
1 0 0 1 1 0 0 1

2. Let fn be the function fn (x) = sin(nx). Show that the set S = {f1 , f2 , f3 , . . .} is an orthonormal set in
F ([, ]).
3. Determine if the following sets are orthogonal/orthonormal
1 1 1 1
3 1 1
21 21 12 21


(a) S1 = 1 , 2 , 2

(c) S3 = 2 , 2 , 2 , 2

1 1 7
1 1 1 1
21 2 2 2

1 1 1


0 1 1 2 2 2 2
(b) S2 = 0 , 2 , 2

0 1 7
(d) S4 = {e1 , . . . , en }.

Remark: An orthogonal set that does not contain the zero vector is linearly independent.
Definition 6 An orthogonal/orthonormal basis for V is a basis that is also an orthogonal/orthonormal set.

3 0 2
Example 7: Let B = 1 , 1 , 3 .
1 1 3

1. Verify that B is an orthogonal basis for R3 .

2. Find an orthonormal basis for Span(B).

3. Write e1 as a linear combination of the vectors in B.

Remarks: Let V = Rn
hu,vi
is the length of the shadow that u makes onto v.
||v||
 
Let U = v1 v2 vk . Then the (i, j) entry of U T U is vi vj . Thus, {v1 , v2 , . . . , vk } is orthogonal if
and only if U T U is a diagonal matrix (orthonormal if U T U = Ik ).
 
Suppose that the columns of U = v1 vn form an orthonormal basis for Rn .

b = Ux x = UT b

Suppose that the columns of {v1 , . . . , vn } form an orthogonal basis for Rn .

vi b
b = x1 v1 + a2 v2 + + xn vn xi = vi vi

Example 8: Consider the orthogonal basis B of R3 given in Example 7. Express e1 as a linear combination of
elements of B.
Remark: Let V be a general vector space. If {v1 , . . . , vn } form an orthogonal basis for V relative to the inner
product h, i, then
hvi ,bi
b = x1 v1 + a2 v2 + + xn vn xi = hvi ,vi i

Example 9: In Fourier analysis, it is known that the set

1
{ , sin(x), cos(x), sin(2x), cos(2x), . . .}
2
form an orthonormal basis for the space of all functions that are piecewise continuous on [, ] and are periodic of
periodicity 2. For example, if f (x) = x2 on [(2k 1), (2k + 1)] for all integer k, then

a0 X X
f (x) = + an cos(nx) + bm sin(mx)
2 n=1 m=1

for some scalars a0 , a1 , b1 , a2 , b2 . . . ,. How do we compute for these scalars?

 Math 114: Linear Algebra
Orthogonal Complements
Definition 7 Let W be a subset of an inner product space V and z V .
1. We say that z W if z w for all w W .

2. The orthogonal complement of W is the set of all vectors orthogonal to W .

W = {z Rn | hz, wi = 0 for any w W }

Remark: W is a subspace of Rn (use subspace test and properties of the dot product).
Example 10: Find the orthogonal complement of the following sets in Rn .

1 0 1 6
1. W1 = 0 , 1 3. W3 = 0 , 1
0 0 1 0


a  c 6d 
2. W2 = b  a, b R 4. W4 = d  c, d R
 
0 c


Remark: If S is a set in V , then S = Span(S) .

Example 11: Determine the orthogonal complement of S = {A R33 | AT = A}.
Remarks: Suppose W is a subspace of V .
1. (W ) = W

2. W W = {0V }
3. Let B be a basis for W . Then z W if and only if z B for all i = 1, . . . , k.
T
v1 0
n .. ..  T
4. Let S = {v1 , . . . , vk } R . If z S . z = . z N ul( v1 vk )
vkT 0

Theorem: Col(A) = N ul(AT ) and N ul(A) = Col(AT ).

Examples: Find Col(A) and N ul(A) .

1 0 0
2 4 2 1 1 0 9 0 0 1 0
1. A = 2 5 7 3 Note that A 0 1 5 0 and AT
0
.
0 1
3 7 8 6 0 0 0 1
0 0 0
1

1 0 5
13

3 6 1 1 7 1 2 0 1 3 0
1 5
2. A = 1 2 2 3 1 Note that 0 0 1 2 2 and AT
0 0 0.
2 4 5 8 4 0 0 0 0 0 0 0 0
0 0 0

Theorem: If W is a subspace of V and dim(V ) = n. Then dim W + dim W = n.

Remarks:

V = {0V }
If {v1 , . . . , vk } forms a basis for W and {u1 , . . . , up } is a basis for W , then {v1 , . . . , vk , u1 , . . . , up } is a basis
for V .