1 VECTOR SPACES
Objectives
By the end of this topic, you will be able to:
define the terms Vector Spaces and Subspaces.
compute the basis and dimension of vector spaces.
compute the column and row spaces of vector spaces.
1.1 General Vector Spaces and Subspaces
Definition:
Let V be a set of objects on which two operations (addition and scalar
multiplication) are defined. Then for x, y, z ∈ V and a, b ∈ R, V is called a
Vector Space if the following axioms (rules) are satisfied:
1) x + y ∈ V 2) x+y = y+x 3) (x+y) + z = x + (y+z) 4)
∃ 0 ∈ V such that x+0 = x
5) x + (-x) = 0 6) ax ∈ V 7) a(x + y) = ax + by 8) (a + b)x
= ax + by
9) (ab)x = a(bx) 10) 1.x = x ∀x ∈ V .
Examples:
#1.) V = Rn (with + and . defined).
#2.) V = set of all ordered n tupples of real numbers with + and . defined
by:
(x1 , x2 , · · · , xn ) + (y1 , y2 , · · · , yn ) = (x1 + y1 , x2 + y2 , · · · , xn + yn ).
c.(x1 , x2 , · · · , xn ) = (cx1 , cx2 , · · · , cxn )
#3.) The set of m × n matrices with the usual matrix addition and scalar
multiplication.
#4.) The set of all polynomials of degree ≤ n with the usual + and . defined
#5.) the following does not form a vector space: c.(x, y, z) = (cx, y, z).
1
Theorem:
Given (V, +, .) or simply V, the following are true:
a) 0.x = 0 ∀ x ∈ V b) c.0 = 0 ∀ c ∈ R c) cx = 0 ⇒ either c = 0 or
x=0
d) (−1)x = −x∀ x ∈ V .
Subspaces:
A subset W of a vector space V is called a subspace of V if W is itself a
vector space under + and . defined on V.
Examples:
(i) V is a subspace of itself. (ii) {O} is a subspace of V. (iii) W = (a,
b, 0); a, b ∈ R is a subspace of R3 i.e W ⊆ R3 .
Theorem:
Given (V; +, .) and W ⊆ V , (W non-empty); then W is a subspace of V ,
iff :
i) x, y ∈ W ⇒ x + y ∈ W .
ii) x ∈ W & c ∈ R ⇒ cx ∈ W .
All the other axioms are implied.
Examples: !
a b 0
#1. W the set of all 2×3 matrices of the form where a, b, c, d ∈
0 c d
R
Solution
Chose any two such matrices and verify with the axioms.
#2. W = set of all vectors of the form(a, b, 1), a, b ∈ R is not a subspace of
R3 .
Remarks: (a1 , b1 , 1) + (a1 , b2 , 1) = (a1 + a1 , b1 + b2 , 2) ⊈ W since the last
figure in W must be a 1.
Definition: Linear Combination
A vector x ∈ V (+, .) is defined to be a linear combination of the vectors
2
x1 , x2 , · · · , xk ∈ V iff,
x = c1 x 1 + c2 x 2 + · · · + ck x k ; ci ∈ R
Examples:
#1. Take V = R4 e.g V = (2, 1, 5, −5) as a linear combination of vectors:
v1 = (1, 2, 1, −1),
v2 = (1, 0, 2, −3) and v3 = (1, 1, 0, −2). Find c1 , c2 , c3 ∈ R to make V a linear
combination of the given vectors.
Solution: c1 = 1, c2 = 2 & c3 = −1.
#2. In R3 , x = (1, 0, 2) is not a linear combination of vectors v1 =
(1, 2, −1) & v2 = (1, 0, −1). Verify.
Definition:
Consider V (+, .) and S = {x1 , x2 , · · · , xk }; xi ∈ V . Then S spans V or V
is spanned by S if every u ∈ V is a linear combination of vectors of S, i.e
∃ c1 , c2 , · · · , ck ∈ R s.t
u = c1 x 1 + c2 x 2 + · · · + ck x k .
Examples:
#1. The vectors i, j, k span R3 because every vector (a, b, c) ∈ R3 can be
expressed as a linear combination of i, j, k, i.e (a, b, c) = ai + bj + ck.
#2. Let V = R3 , S = (x1 , x2 , x3 ) where x1 = (1, 2, 1); x2 = (1, 0, 2) & x3 =
(1, 1, 0). Does S span V?
Hint: Solve for the constants ci
#3. Let V = P2 (t) be a set of all polynomials of degree ≤ 2 and S =
{p1 (t), p2 (t)} with p1 (t) = t2 + 2t + 1; p2 (t) = t2 + 2. Does S span P2 (t)?
Solution:
Find scalars α1 , α2 ∈ R such that:
P2 (t) = at2 + bt + c = α1 p1 + α2 p2 ; a, b, c ∈ R
= α1 (t2 + 2t + 1) + α2 (t2 + 2)
= (α1 + α2 )t2 + 2α1 t + (α1 + 2α2 )
3
∴ α1 + α2 = a
2α1 = b Solving using ERO;
α + 2α = c
1 2
1 1 a 1 1 a
⇒ =⇒ 0 1 2a−b
2 0 b 2
2c−b−2a
1 2 c 0 0 2
The final system is not consistent, hence S does not span P2 (t).
EER:
#1. Express the following as linear combinations of u = (2, 1, 4), v = (1, −1, 3) & w = (3, 2, 5)
(i) (5, 9, 5) (ii) (2, 0, 6) (iii) (2, 2, 3)
#2. Determine whether the following polynomials span P2 (x)
(i) p1 = 1 + 2x − x2 ; (ii) p2 = 3 + x2
(iii) p3 (x) = 5 + 4x − x2 ; (iv) p4 (x) = −2 + 2x − 2x2 .
#3. Express the following as linear combinations of p1 = 2 + x + 4x2 ;
p2 = 1 − x + 3x2 & p3 = 3 + 2x + 5x2 :
a) 5 + 9x + 5x2 b)2 + 6x2 c) 0; 2 + 2x + 3x2 .
1.2 Linear Independence and Dependence
Definition:
Given S = {x1 , x2 , · · · , xk }; xi ∈ V , V a vector space. Then S is said to be
Linearly Dependent (LD) if ∃ c1 , c2 , · · · , ck ∈ R not all zeros such that,
c1 x1 + c2 x2 + · · · + ck xk = 0.
If c1 = c2 = · · · = ck = 0, then S is Linearly Independent (LI).
Examples:
#1. The set of vectors i = (1, 0, 0); j = (0, 1, 0), & k = (0, 0, 1) ∈ R3 are
LI.
#2. Let S = {x1 , x2 , x3 } where x1 = (1, 0, 1, 2), x2 = (0, 1, 1, 2) &x3 =
(1, 1, 1, 1, 3). Show that S is LI.
4
Hint: Solve for c1 , c2 , c3 st c1 = c2 = c3 = 0.
#3. Show that S = {x1 , x2 , x3 , x4 } where x1 = (1, 2, −1), x2 = (1, −2, 1), x3 =
(−3, 2, −1), &x4 = (2, 0, 0) is LD.
#4. Verify that S = {(t2 + t + 2), (2t2 + t), (3t2 + 2t + 2)) is LD.
Theorem:
A set S with two or more vectors is:
i) LD iff at least one of the vectors in S is expressible as a linear combi-
nation of the other vectors in S.
ii) LI iff no vector in S is expressible as linear combination of the other
vectors in S.
Theorem:
If a set contains the zero vector, then it is LD.
A set with exactly two vectors is LD iff at least one of the vectors is a
scalar multiple of the other vectors.
1.3 Basis and Dimensions
Definition:
Given V(+,.) and S = {v1 , v2 , · · · , vr } a finite set of vectors in V, then S is
called a basis for V if:
i) S is Linearly Independent.
ii) S spans V.
Examples:
Q1. V = {i, j, k} is a basis for R3 . The set of vectors {(1, 0, · · · , 0), (0, 1, 0, · · · , 0), · · · , (0, 0, · · · , 1)
Rn is a standard basis for Rn .
Q2. Let v1 = (1, 2, 1), v2 = (2, 0, 0) &v3 = (3, 3, 4). Show that S =
{v1 , v2 , v3 } is a basis for R3 .
Solution:
To show S spans R3 , an arbitrary vector b = (b1 , b2 , b3 ) can be expressed
5
as a linear combination of S vectors i.e.;
(b1 , b2 , b3 ) = k1 v1 + k2 v2 + k3 v3 = k1 (1, 2, 1) + k2 (2, 0, 0) + k3 (3, 3, 4).
Formulate the linear system of equations:
⇒ k1 + 2k2 + 3k3 = b1
2k1 + 9k2 + 3k3 = b2 (1)
k + 4k = b
1 3 3
The system (1) must have a solution for all choices of the vector b. To prove
that S is LI, it must be shown that the only solution of k1 v1 + k2 v2 + k3 v3 = 0
is k1 = k2 = k3 = 0, i.e.
⇒ k1 + 2k2 + 3k3 = 0
2k1 + 9k2 + 3k3 = 0 (has only trivial solution) (2)
k
1 + 4k = 0 3
Note
thatthe equations (1) and (2) have the same coefficient matrix A =
1 2 3
2 9 3. Thus if A is invertible, then S is both LI and spans R3 .
1 0 4
1 2 3
Now det(A) = 2 9 3 = −1. Work this out.
1 0 4
Hence A is invertible. Thus A is a basis for R3 .
Q3. The set S = {1, x, x2 , x3 , · · · , xn } is a standard basis for Pn – a nonzero
polynomial of degree n.
Definition:
A nonzero vector space V is finite dimensional if it contains a finite set of
vectors {v1 , v2 , · · · , vn } that forms a basis, otherwise it is infinite dimensional.
The space {O} of V has dimension 0.
Theorems:
6
1) If V is a vector space of dimension n > 0; then;
a) any set of n LI vectors span V.
b) any n vectors that span V are LI
2) If V is a vector space of dimension n > 0; then;
i) no set less than n vectors can span V.
ii) any subset of less than n LI vectors can be extended to form a
basis for V.
iii) any spanning set more than n vectors can be pared down (reduced)
to form a basis for V
Definition:
The dimension of a finite dimensional vector space V is defined as the number
of vectors in a basis for V. The zero vector space has dimension zero.
Examples:
#1. Determine the basis and the dimension for the solution space of the
following homogeneous system:
2x1 + 2x2 − x3 + x5 = 0
−x1 − x2 + 2x3 − 3x4 + x5 = 0
x1 + x2 − 2x3 − x5 = 0
x3 + x4 + x5 = 0
Solution:
Verify that the solution through ERO is: x1 = −s−t, x2 = s, x3 = −t, x4 =
0; x5 = t, i.e
x1 −s − t −s −t −1 −1
x2 s s 0 1 0
x3 = −t = 0 + −t = s 0 + t −1
x 0 0 0 0 0
4
x5 t 0 t 0 1
7
−1 −1
1 0
Thus the vectors v1 = 0 and v2 = −1 span the solution space, so
0 0
0 1
the are LI and {v1 , v2 } is a basis and the solution space thus has dimension
2.
EER:
#1. Find the dimension of the solution space W of the following homogeneous
equations:
x1 + 2x2 + 3x4 + x5 = 0
2x1 + 3x2 + 3x4 + x5 = 0
x1 + x2 + 2x3 + 2x4 + x5 = 0
3x1 + 5x2 + 6x4 + 2x5 = 0
2x1 + 3x2 + 2x3 + 5x4 + 2x5 = 0
Solution: Dim(W) = 2
Hint: Use ERO to solve as in the example above.
#2. Given the vectors x1 = (3, −2, 4)T ; x2 − (−3, 2, −4)T ; x3 = (−6, 4, −8)T .
Find the dimension of the span (x1 , x2 , x3 ).
#3. The vectors x1 = (1, 2, 2)T ; x2 = (2, 5, 4)T ; x3 = (1, 3, 2)T ; x4 = (2, 7, 4)T ; x5 = (1, 1, 0)T
span R3 . Pare down the given vectors to form a basis for R3 .
#4. Determine the dimension of and the basis for the solution space of the fol-
(b)
(a)
x+y+z =0
x1 − 3x2 + x3 = 0 3x + 2y − 2z = 0
2x1 − 6x2 + 2x3 = 0 4x + 3y − z = 0
lowing systems: 3x1 − 9x2 + 3x3 = 0 6x + 5y + z = 0
8
1.4 Row and Column Space; Rank
0≤i≤m
Given any m × n matrix A = [ai,j ]0≤j≤n ;
a) the row vectors of A are formed by the rows of A.
b) the column vectors of A are formed by the columns of A.
The subspace of Rn spanned by the row vectors of A is called the row space
of A and similarly for the column
! space of A.
1 2 3
For example; let A = .
4 5 6
(2×3)
The row vectors are: r1 = (1, 2, 3) & r2 = (4, 5, 6)
The columns vectors are: c1 = (1, 4)T , c2 = (2, 5)T & c3 = (3, 6)T .
* Row space Rs = {αr1 + βr2 }; Rs ⊆ R3 .
* Column space, Cs = {ac1 + bc2 + kc3 }, Cs ⊆ R2 .
Theorem:
ERO do not change the row space of a matrix. Thus the row space of A and
its row echelon form equivalent are the same. The nonzero rows of A of the
row echelon form are always LI and so form a basis for the row space of the
matrix.
Examples:
Q1. Find the basis for the space spanned by the vectors:
v1 = (1, −2, 0, 0, 3); v2 = (2, −5, −3, −2, 6); v3 = (0, 5, 15, 10, 0) & v4 =
(2, 6, 18, 8, 6).
Solution:
The space
spanned by thevectors is the row space of the matrix A where,
1 −2 0 0 3
2 −5 −3 −2 6
A= 0 5 15
.
10 0
2 6 18 8 6
9
1 −2 0 0 3
0 1 3 2 0
The row echelon form of this matrix is
. Verify.
0 0 1 1 0
0 0 0 0 0
The nonzero rows (the first three rows) form a basis for the row space and
consequently a basis for the space spanned by the given vectors.
Q2. Let S = {(1, −2, 0, 3, −4), (3, 2, 8, 1, 4), (2, 3, 7, 2, 3), (−1, 2, 0, 4, −3)}.
If V = span S; S ⊆ R5 ; find the dimension of V.
Solution:
Hint: Build up the matrix A whose rows are formed by the given vectors,
reduce it its row equivalent form. The nonzero rows form the basis and the
dimension is the number of nonzero rows.
Definitions:
i) Row Rank of A = dimension of row space of A.
ii) Column Rank of A = dimension of column space of A.
iii) For any matrix A; row rank = column rank = rank of A, rank(A).
Note: To get the column rank of A, obtain the row rank of AT .
Example:
1 0 1 1
Find a basis for the column space of A =
3 2 5 .
1
0 4 4 −4
Solution:
Hint: Obtain AT and reduce it to its row echelon form to find the basis.
Theorem:
If A is an n × n matrix, the following statements are equivalent:
a) A is invertible.
b) Det(A) ̸= 0.
c) Ax = 0 has only the trivial solution
d) A is row equivalent to In .
10
e) Ax = b is consistent for any matrix bn×1 .
f) A has rank n.
g) The row and column vectors of A are LI.
Theorems:
1) A system of linear equations Ax = b is consistent iff b is the column space
of A.
2) A system of linear equations Ax = b is consistent iff rank(A) = rank[A |
b]– the augmented matrix.
EER:
Calculate
a) a basis for the row space
b) a basis for the column space
c) the rank (A)
1 −3 2 2 1
0 3
6 0 −2
of the following matrix: A = 2 −3 −2 4 4 .
3 −3 6 6 3
5 −3 10 10 5
1.5 Orthonormal Basis
Definition:
An orthonormal set of vectors is an orthogonal set in which each vector has
norm one.
The process of multiplying a nonzero vecor V by the reciprocal of its norm
to obtain a unit vector is called normalizing the vector.
11
