Chapter.
5 Orthogonality
5.5. Orthonormal Sets
1. Orthogonal Sets
2. Orthonormal Sets
3. Orthonormal Basis
4. Orthogonal Matrices
5. Properties of Orthogonal Matrices
6. Permutation Matrices
7. Orthonormal Sets and Least Squares
1
1. Orthogonal Sets
Definition
Let 𝐯1 , 𝐯2 , … , 𝐯𝑛 be nonzero vectors in an inner product space 𝑉. If 𝐯𝑖 , 𝐯𝑗 = 0, where 𝑖 ≠ 𝑗, then
𝐯1 , 𝐯2 , … , 𝐯𝑛 is said to be an orthogonal set of vectors.
2
2. Orthonormal Sets
Definition
An orthonormal set of vectors is an orthogonal set of unit vectors.
Or, each pair of vectors in the set is orthogonal (or perpendicular) to each other, and every vector in the set has a unit
norm.
Or, a set of vectors will be considered an orthonormal set if
i. The set of vectors is orthogonal, i.e. 𝐯𝑖 , 𝐯𝑗 = 0
𝑽𝑖 𝑽𝑖
ii. Length of the vectors is one. i.e. 𝐮𝒊 = = 𝑣12 + 𝑣22 + ⋯ + 𝑣𝑛2 = 1 , ∵ 𝐮𝒊 =
𝑽𝑖 𝑽𝑖
Remarks:
The se 𝐮1 , 𝐮2 , … , 𝐮𝑛 will be orthonormal if and only if
1 𝑖𝑓 𝑖 = 𝑗
𝐮𝑖 , 𝐮𝑗 = 𝛿𝑖𝑗 , where 𝛿𝑖𝑗 = ቊ
0 𝑖𝑓 𝑖 ≠ 𝑗
Given any orthogonal set of nonzero vectors 𝐯1 , 𝐯2 , … , 𝐯𝑛 it is possible to form an orthonormal set by defining
1
𝐮𝑖 = 𝐯𝑖 , for 𝑖 = 1,2, … , 𝑛 3
𝐯𝑖
2. Orthonormal Sets
Here 𝐮1 , 𝐮2 , 𝐮2 are unit vectors, because the length of these vectors is one, i.e.
2 2 2
1 1 1
𝐮1 = + + =1
3 3 3
For 𝐮2 and 𝐮3 you can find by yourself , also practice example.3
4
3. Orthonormal Basis
5
3. Orthonormal Basis
Example:
1 2 2 𝑇 2 1 −2 𝑇
Let 𝐯1 , 𝐯2 ∈ 𝑉, and 𝐯1 = , , , 𝐯2 = , , , let 𝐵 = 𝐯1 , 𝐯2
3 3 3 3 3 3
1st to check orthogonality
1 2 2 1 2 −2
𝐯1 . 𝐯2 = + + = 0, therefore 𝐯1 and 𝐯2 are orthogonal vectors.
3 3 3 3 3 3
2nd to check orthonormality
1 2 2 2 2 2 2 2 1 2 −2 2
𝐯1 = + + = 1, 𝐯2 = + + = 1, ∴ 𝐵 is an orthonormal set
3 3 3 3 3 3
3rd to check basis
Please see the definition of basis in ppt.3.4, page.2, also see the definition of span and linear independence vectors.
𝑉 = 𝑠𝑝𝑎𝑛(𝐯1 , 𝐯2 )
∴ 𝐵 is an orthonormal basis.
See examples. 4,5
6
4. Orthogonal Matrices
7
4. Orthogonal Matrices
Example.6
Since, 𝑆𝑖𝑛𝜗2 + 𝐶𝑜𝑠𝜗2 = 1
Therefore, the matrix Q is an orthogonal matrix 8
5. Properties of Orthogonal Matrices
9
6. Permutation Matrices
Definition:
A permutation matrix is a matrix formed from the identity matrix by reordering its columns.
Clearly, then, permutation matrices are orthogonal matrices.
If P is the permutation matrix formed by reordering the columns of I in the order (k1 , ... , k𝑛 ), then P = (e𝑘1 , ... , e𝑘𝑛 ).
If A is an m × n matrix, then
Post multiplication of A by P reorders the columns of A in the order (k1 , ... , k𝑛 ). For example, if
Then Since P = (e𝑘1 , ... , e𝑘𝑛 ) is orthogonal, it follows that
10
7. Orthonormal Sets and Least Squares
Theorem 5.5.6
If the column vectors of 𝐴 form an orthonormal set of vectors in ℝ𝑚 , then 𝐴𝑇 𝐴 = 𝐼
and the solution to the least squares problem is
11
7. Orthonormal Sets and Least Squares
d
12
7. Orthonormal Sets and Least Squares
Theorem 5.5.8:
Under the hypothesis of theorem 5.5.7, 𝐩 is the element of 𝑆 that is closest to 𝐱; that is,
For any 𝐲 ≠ 𝐩 in 𝑆.
13
7. Orthonormal Sets and Least Squares
14
7. Orthonormal Sets and Least Squares
15
7. Orthonormal Sets and Least Squares
16
