ABDULRAHMAN AL SUMAIT UNIVERSITY

MT121: LINEAR ALGEBRA

4: Inner Product Spaces

Instructor: NAMANOLO, Hassani Saidi

Faculty of Science
Department of Mathematics and Computer Science

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025

 Outline Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Outline

1 Introduction to Inner Product Spaces

2 Inequalities and Orthogonality

3 Orthonormal Bases

4 Orthogonal Matrices

5 QR Factorization

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 1/28

Introduction to Inner Product
Spaces Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

1 Introduction to Inner Product Spaces

2 Inequalities and Orthogonality

3 Orthonormal Bases

4 Orthogonal Matrices

5 QR Factorization

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 2/28

Introduction to Inner Product
Spaces Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Definition of inner product

How did we define the inner or dot product in chapter 2?

   
u1 v1
 ..   .. 
Let u =  .  and v =  .  be vectors in Rn . Then the inner
un vn
product of u and v, denoted by u · v, is given by

u · v = uT v = u1 v1 + u2 v2 + u3 v3 + · · · + un vn

Remember, the answer was a scalar not a vector. This inner prod-
uct was named the dot product (also called the scalar product) in
Rn .
This is the usual (or standard) inner product in Rn but there are
many other types of inner products in Rn .

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 2/28

Introduction to Inner Product
Spaces Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Cont...
For the general vector space, the inner product is denoted by
hu, vi rather than u · v.

Definition
An inner product on a real vector space V is an operation which as-
signs to each pair of vectors, u and v, a unique real number hu, vi
which satisfies the following axioms for all vectors u, v, w ∈ V and all
scalars k:
(i) hu, vi = hv, ui [commutative law]
(ii) hu + v, wi = hu, wi + hv, wi [distributive law]
(iii) hku, vi = khu, vi [taking out the scalar k]
(iv) hu, ui ≥ 0 and we have hu, ui = 0 ⇐⇒ u = 0
[Means the inner product btn the same vectors is zero or +ve.]

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 3/28

Introduction to Inner Product
Spaces Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Cont...
A real vector space which satisfies these axioms is called a real
inner product space.
Note that evaluating h, i gives a real number (scalar) not a vector.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 4/28

Introduction to Inner Product
Spaces Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Properties of inner products

Proposition
Let u, v, w be vectors in a real inner product space V, and let k be any
real scalar. We have the following properties of inner products:
(i) hu, Oi = hO, vi = 0
(ii) hu, kvi = khu, vi
(iii) hu, v + wi = hu, vi + hu, wi

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 5/28

Introduction to Inner Product
Spaces Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

The norm or length of a vector

Do you remember how the (Euclidean) norm was defined?

The norm or length of a vector u in Rn is defined by Pythagoras’
Theorem: √
kuk = u · u.
The norm is defined in the same manner for a general vector space
V. Let u be a vector in V, then the norm denoted by kuk is defined
as: q
kuk = hu, ui
Note that for the general vector space we cannot use the definition
for the dot product because that is only defined for Euclidean
space, Rn , and in this chapter we are examining inner products
on general vector spaces.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 6/28

Introduction to Inner Product
Spaces Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

How do we find the norm of a function or a matrix?

The norm of a matrix A in the vector space Mm×n is given by:
q
kAk = hA, Ai

where hA, Ai is the inner product of A with itself.

Norm measures the magnitude of things. For example, if we take
our earlier inner product defined for matrices:

hA, Bi = tr(BT A),

then we can evaluate the magnitude of kAk and kBk by calculating

their respective norms.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 7/28

Introduction to Inner Product
Spaces Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Example
  
1 2 10 20
Let A = and B = , find kAk and kBk.
3 4 30 40

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 8/28

Introduction to Inner Product
Spaces Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Properties of the norm of a vector

Proposition
Let V be an inner product space and u and v be vectors in V. If k is
any real scalar, then we have the following properties of norms:
(i) kuk ≥ 0 [non-negative]
(ii) kuk = 0 ⇐⇒ u = 0
(iii) kkuk = |k| kuk
Note that for a real scalar k, we have
√
k2 = | k | ,

where |k| is the modulus of k.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 9/28

Inequalities and Orthogonality Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

1 Introduction to Inner Product Spaces

2 Inequalities and Orthogonality

3 Orthonormal Bases

4 Orthogonal Matrices

5 QR Factorization

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 10/28

Inequalities and Orthogonality Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Refresh
   
1 3
Let u = and v = be in R2 . Determine the following with
2 4
respect to the dot product:
(i) |hu, vi|
(ii) kukkvk
(iii) kuk + kvk
(iv) ku + vk

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 10/28

Inequalities and Orthogonality Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Inequalities

Introduction
In this subsection, we prove two important inequalities:
- |hu, vi| ≤ kuk · kvk (Cauchy–Schwarz inequality)
- ku + vk ≤ kuk + kvk (Minkowski inequality)
First, we state and prove the Cauchy–Schwarz inequality. It is
an important inequality used in many applications and fields of
mathematics.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 11/28

Inequalities and Orthogonality Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

The Cauchy–Schwarz inequality

Let u and v be vectors in an inner product space, then

|hu, vi| ≤ kuk · kvk.

Proof: Exercise

The Minkowski or triangular inequality

Let V be an inner product space. For all vectors u and v, we have

ku + vk ≤ kuk + kvk.

Proof: Exercise

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 12/28

Inequalities and Orthogonality Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Orthogonal vectors

Can you remember what orthogonal vectors meant in the Eu-

clidean space Rn ?
Two vectors u and v in Rn are said to be orthogonal or perpen-
dicular if and only if
u · v = 0.

Definition
Two vectors u and v in the vector space V are said to be orthogo-
nal if and only if
hu, vi = 0.
This is a fundamental and very useful result in linear algebra. If
vectors u and v are orthogonal, we say that u is orthogonal to v,
or vice versa, that is, v is orthogonal to u because

hu, vi = hv, ui = 0.
HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 13/28
Inequalities and Orthogonality Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Normalizing vectors

What is a unit vector?

A unit vector u is a vector of length 1, or a norm of 1, that is,
kuk = 1.
The process of converting a given vector into a unit vector is called
normalizing.

Proposition
Every non-zero vector w in an inner product space V can be normal-
ized by setting
w
u= .
kwk

We write the normalized vector w as ŵ, which is pronounced as “w

hat.” We have u = ŵ.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 14/28

Inequalities and Orthogonality Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Orthonormal set

Definition
A set of vectors which are orthogonal (perpendicular) to each
other is called an orthogonal set.
A set of vectors in which all the vectors have a norm or length of
1 is called a normalized set.
A set of perpendicular unit vectors is called an orthonormal set .
This is a set of vectors which are both orthogonal and normalized.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 15/28

Inequalities and Orthogonality Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Definition
Let V be a finite-dimensional vector space with an inner product.
A set of vectors
B = {u1 , u2 , u3 , . . . , un }
for V is called an orthonormal set if:
(i) They are orthogonal, that is,

hui , uj i = 0 for i 6= j

(ii) They are normalized, that is,

kuj k = 1 for j = 1, 2, 3, . . . , n

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 16/28

 Orthonormal Bases Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

1 Introduction to Inner Product Spaces

2 Inequalities and Orthogonality

3 Orthonormal Bases

4 Orthogonal Matrices

5 QR Factorization

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 17/28

 Orthonormal Bases Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Introduction to an orthonormal bases

Definition
Let V be a finite-dimensional vector space with an inner product.
A set of basis vectors B = {u1 , u2 , u3 , . . . , un } for V is called an
orthonormal basis if they are:
(i) Orthogonal, that is,

hui , uj i = 0 for i 6= j

(ii) Normalized, that is,

kuj k = 1 for j = 1, 2, 3, . . . , n

What do you notice about this definition?

It’s the same as the definition of an orthonormal set given in the last
section, but this time the set of vectors are the basis vectors of the
vector space.
HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 17/28
 Orthonormal Bases Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Properties of an orthonormal basis

Proposition
If {v1 , v2 , v3 , . . . , vn } is an orthogonal set of non-zero vectors in an inner
product space, then the elements of this set are linearly independent.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 18/28

 Orthonormal Bases Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

The Gram–Schmidt process

Introduction
Given any arbitrary basis {v1 , v2 , v3 , . . . , vn } for a finite-
dimensional inner product space, we can find an orthogonal ba-
sis {p1 , p2 , p3 , . . . , pn } by the Gram–Schmidt process, which is de-
scribed next:

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 19/28

 Orthonormal Bases Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Gram–Schmidt process

Let p1 = v1
hv2 , p1 i
p2 = v2 − p
k p1 k 2 1
hv3 , p1 i hv3 , p2 i
p3 = v3 − p − p2
k p1 k 2 1 kp2 k2
hv4 , p1 i hv4 , p2 i h v4 , p3 i
p4 = v4 − p − p2 − p3
k p1 k 2 1 kp2 k 2 kp3 k2
..
.
n−1
hvn , pk i
pn = vn − ∑ k pk k 2 k
p
k =1

These vectors p1 , p2 , . . . , pn are orthogonal, and therefore form a basis.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 20/28

 Orthonormal Bases Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Examples
   
2 1
(i) Transform the basis v1 = and v2 = in R2 to an orthonor-
1 1
mal basis (axes) for R2 with respect to the dot product.
(ii) Transform the following basis vectors in R3 to an orthonormal
basis for R3 with respect to the dot product.
     
1 3 −1
(a) v1 = 0 , v2 = 1 , v3 = −1
1 1  −1 
2 −1 −1
(b) v1 = 2 , v2 =  0  , v3 =  2 
2
−1
   
3
1 2 1
(c) v1 = 2 , v2 = 0 , v3 = 0
0 2 3

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 21/28

 Orthonormal Bases Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Class work
     
1 3 1
Let u1 = 1 ,
  u2 = 1 ,  u1 = −3 . Apply the Gram-

0 1 −1
Schmidt Orthogonalization process to u1 , u2 , u3 to obtain an orthog-
onal basis v1 , v2 , v3 for R3 .

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 22/28

 Orthogonal Matrices Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

1 Introduction to Inner Product Spaces

2 Inequalities and Orthogonality

3 Orthonormal Bases

4 Orthogonal Matrices

5 QR Factorization

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 23/28

 Orthogonal Matrices Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Orthogonal Matrices

Definition
 
A square matrix Q = v1 v2 v3 · · · vn , whose columns
v1 , v2 , v3 , . . . , vn are orthonormal (perpendicular unit) vectors, is
called an orthogonal matrix.
An example of an orthogonal matrix is the identity matrix.

Proposition
 
Let Q = v1 v2 v3 · · · vn be a square matrix. Then Q is an or-
thogonal matrix if and only if

QT Q = I.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 23/28

 Orthogonal Matrices Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Example
 
Let Q = u1 u2 u3 where
r 1 r  1  r  1 
1  1  1 
u1 = 1 , u2 = 1 , u3 = −1
3 6 2
1 −2 0

which is an orthonormal basis for R3 . Determine QT Q.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 24/28

 Orthogonal Matrices Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Properties of orthogonal matrices

Refresh
Let r      
1 1 1 1 3
Q= , u= , w= .
2 1 −1 2 1
Determine the dot products:
(i) Qu · Qw
(ii) u · w
(iii) What do you notice about your results?

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 25/28

 Orthogonal Matrices Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

Proposition
Let Q be an n × n matrix and u and w be vectors in Rn . Then

Q is an orthogonal matrix ⇐⇒ Qu · Qw = u · w.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 26/28

 QR Factorization Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

1 Introduction to Inner Product Spaces

2 Inequalities and Orthogonality

3 Orthonormal Bases

4 Orthogonal Matrices

5 QR Factorization

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 27/28

 QR Factorization Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

QR Factorization

Class Work
Read the concept of QR factorization and perform at least 3 questions.

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 27/28

 QR Factorization Outline Introduction to Inner Product Spaces Inequalities and Orthogonality Orthonormal B

THANK YOU

HASSANI SAIDI NAMANOLO LECTURE NOTES June 12, 2025 28/28