Matrix Algebra

Math 203
Lecture 15

Lecturers
Dr. Hasan Ibrahim
Dr. Mohamed Salman
Dr. Rami Younes
 Matrices

Lecture Outline

1 Matrices
Matrix Algebra

Matrix Algebra
 Matrices

Recommended Reading

Elementary Linear Algebra, Howard Anton and Chris Rorres,

10th ed.: sections 1.3 and 1.4
Introduction to Applied Linear Algebra, Stephen Boyd, Lieven
Vandenberghe: skim through the titles covered in chapters 6,
and sections 10.1, 10.3, 11.1, 11.2.

Matrix Algebra
 Matrices Matrix Algebra

Basics of Matrices
If the number of rows n of a matrix equals the number of
A matrix is a rectangular array of numbers. The numbers
columns, the matrix is called an order n square matrix or
in the array are called the entries of the matrix.
an n ⇥ n square matrix. The entries a11 , a22 , . . . , ann of
If the dimensions of a matrix A are n rows and m columns an order n square matrix are called the (main) diagonal
then A is referred to as n ⇥ m matrix. entries.
The entry that occurs in row i and column j of a matrix A
will be denoted by aij or (A)ij .

The sum of its diagonal entries of a square matrix is

called the trace of the matrix. If A = [aij] is n ⇥ n, then
tr(A) = a11 + a22 + · · · + ann .
Examples

(i) 0.3 1 5
A= is a 2 ⇥ 3 matrix.
0 0.2 16
a21 = 0, a23 = 16,
An n ⇥ 1 matrix (n rows and one column) is called a
2 3 a32 is undefined since the A has only 2 rows.
c1
6 c2 7 2 3
6 7 1 3 0
column vector c = 6 7
6 .. 7. (ii) B =4 2 5 45 is a square matrix of order 3.
4.5 7 1 0
cn The diagonal entries are
An 1 ⇥ m matrix (1 row and⇤ m columns) is called a row b11 = 1, b22 = 5, b33 = 0, and tr(B) = 6.
⇥ 2 3
vector r = r1 r2 . . . rm . 4
For an n ⇥ m matrix if n > m the matrix is said to be (iii) c = 40.55 is a column vector.
tall and if n < m the matrix is said to be wide. 1
⇥ ⇤
(iv) r = 2 5 0.8 0 1 is a row vector.
Matrix Algebra
 Matrices Matrix Algebra

Matrices in Practice
Examples
1 Matrices may represent data tables. Assume that a 3 Matrices can be used to characterize connections in
company produces two computer models PC1086 and electrical networks, in nets of roads, in production
PC1186. The cost per computer (in thousands of dollars) processes, etc. For example, consider the following nodal
is shown in matrix A and the production figures for the incidence matrix. The network can be described by a
year 2019 (in multiples of 10,000 units) is shown in matrix matrix A = [ajk ] such that
B

Thus, the following network is captured by the following

incidence matrix
2 A gray scale image with M ⇥ N pixels is naturally
represented as an M ⇥ N matrix whose entries are
numbers in the interval [0, 1]. The row index i gives the
vertical position of the pixel, the column index j gives the
horizontal position of the pixel, and the i, j entry gives
the pixel value.

Matrix Algebra
 Matrices Matrix Algebra

Basics of Matrices
Algebra of Matrices Example  
2 3 4 0 2 7
1 Two matrices A and B are equal if and only if A and B Let A = ,B = , and
1 3 1 1 3 5
have the same dimensions and 
1 5
C = .
(A)ij = (B)ij for all i and j. 3 1
2 The sum A + B of two matrices A and B is defined if the By properties 2-5, 2A 3B can be computed as
matrices have the same dimensions and A + B is the
 
matrix such that 4 6 8 0 6 21
2A 3B = +
(A + B)ij = (A)ij + (B)ij for all i and j. 2 6 2 3 9 15

3 If c is a scalar then the scalar multiple cA of a matrix A 4 0 13
= .
is defined as the matrix with the same dimesnions as A 5 3 17
and whose entries are
(cA)ij = c(A)ij for all i and j. Remark
4 The opposite of a matrix A is defined as the scalar The zero matrix of dimensions n ⇥ m is the matrix 0 (or
multiple of A, A = 1A, that is 0n⇥m ) such that (0)ij = 0 for all i and j. For example,
( A)ij = (A)ij for all i and j.
5 The di↵erence between two matrices A and B is defined
as A B = A + ( B), that is
(A B)ij = (A)ij (B)ij for all i and j.
Example
 
2 1 2 1
A= ,B = . Properties of Matrix Operations
3 x y 5
By property 1,

A = B if and only if x = 5 and y = 3.

Matrix Algebra
 Matrices Matrix Algebra

Basics of Matrices
Transpose of a matrix Types of Matrices
Given an n ⇥ m matrix A, the transpose of A is the 1 A square matrix whose o↵-diagonal entries are all zero is called
m ⇥ n matrix, denoted by AT , whose columns are a diagonal matrix.
formed from the corresponding rows of A. Thus, A is diagonal () (A)ij = 0 for any i 6= j
2 A square matrix is upper triangular if all the terms below the
(AT )ij = (A)ji for all i and j. diagonal are zero.
A is upper triangular () (A)ij = 0 for any i > j
Examples
3 A square matrix is lower triangular if all the terms above the
diagonal are zero.
A is lower triangular () (A)ij = 0 for any i < j
4 A square matrix A is called symmetric if it is equal to its
transpose, that is AT = A.
A is symmetric, (AT )ij = (A)ij , (A)ji = (A)ij for all i, j.

Further Properties of the Matrix Operation Examples

Let A and B denote matrices whose sizes are (i) The following matrices are respectively, diagonal, upper
approperiate triangular and lower triangular
a. (AT )T = A 2 3
2 3 2 3 4 0 0 0
2 0 0 2 3 5 6
b. (A + B)T = AT + B T 1 0.5 0 07
40 1 05, 40 1 25, 6
4 2
7.
0.7 3 05
0 0 9 0 0 9
c. (cA)T = cAT 0.1 5 2 1
2 3
d. tr(AT ) = tr(A) 1 3 1 8
e. 6 3 2 5 07
tr(A + B) = tr(A) + tr(B) (ii) A=6 4 1
7 is symmetric.
5 0 35
f. tr(cA) = ctr(A) 8 0 3 4

Matrix Algebra
 Matrices Matrix Algebra

Matrix Multiplication
Product of a Matrix and a Column Vector
An n ⇥ m matrix A may be multiplied by a m ⇥ 1 column vector Multiplying Two Matrices
x to produce a n ⇥ 1 column vector as follows: An n ⇥ m matrix A may be multiplied by a m ⇥ p matrix B to
produce an n ⇥ p matrix, denoted AB, as follows:
2 32 3
a11 a12 ... a1m x1
6a21 a22 ... a2m 7 6 x2 7 (AB)ij = ai1 b1j + ai2 b2j + · · · + aim bmj
6 76 7
Ax = 6 . 76 . 7 m
X
6 . 76 . 7
4 . 54 . 5 = aik bkj .
an1 an2 ... anm xm k=1
2 3
a11 x1 + a12 x2 + · · · + a1m xm
6 a21 x1 + a22 x2 + · · · + a2m xm 7 That is, the entry in the i th row and j th column of AB is the
6 7
=6 . 7 dot product of the i th row of A and the j th column of B.
6 . 7
4 . 5
an1 x1 + an2 x2 + · · · + anm xm Illustration

That is, the entry in the i th row of Ax is obtained from the dot
product of the i th row of A and x:

(Ax)i = ai1 x1 + ai2 x2 + · · · + aim xm

Example Example
The product can be found entry by entry by taking the dot
2 3
 1   product of the i th row of A and the j th column of B to get (AB)ij
2 3 4 4 5 2⇥1+3⇥0+4⇥ 1 2
0 = =
1 3 1 1⇥1+3⇥0+1⇥ 1 0
| {z } 1 | {z }
A is 2⇥3 | {z } 2⇥1
x is 3⇥1

Matrix Algebra
 Matrices Matrix Algebra

Properties of Matrix Multiplication

Remark
An n ⇥ m matrix A may be thought of as an array of column Remarks
A square matrix with 1’s on the main diagonal and zeros
vectors A = [a1 a2 . . . am ] where aj is the j th n ⇥ 1
elsewhere is called an identity matrix. The n ⇥ n identity
column of A.
matrix is denoted In . For example,
Example
 
3 0 7 3
B = = [b1 b2 b3 ], where b1 = ,
1 4 6 1
 
0 7
b2 = , b3 = .
4 6
Remark
The product AB can be seen as the array Properties of Matrix Multiplication

⇥ ⇤
AB = Ab1 Ab2 . . . Abp ,

Example
 
4 1 3 0 7
A= ,B = .
5 2 1 4 6
| {z } | {z }
2⇥2 2⇥3
  
3 0 7
The columns of B are b1 = , b2 = , b3 = .
1 4 6
Remark
   For matrices A and B with appropriate sizes,
11 4 34
Ab1 = , Ab2 = and Ab3 = .
17 8 23
(AB)T = B T AT .

11 4 34
Therefore, AB = .
17 8 23

Matrix Algebra
 Matrices Matrix Algebra

Properties of Matrix Multiplication

Powers of Matrices
Remark If A is an n ⇥ n matrix and if k is a positive integer, then Ak
If a and b are two n ⇥ 1 column vectors then their dot denotes the product of k copies of A then
product can be written in terms of matrix multiplication as
follows:
Ak = A . . . A.
a.b = aT b = a1 b1 + a2 b2 + · · · + an bn
We define A0 = In .
Example
Example 
If a = (3, 6, 1) and b = (1, 2, 4) then their dot product is
1 1 2 2 2
If A = then A = AA = = 2A.
2 3 1 1 2 2
⇥ ⇤ 1 ⇥ ⇤ A3 = AAA = AA2 = A(2A) = 2A2 = 22 A.
a.b = 3 6 1 425 = 3 ⇥ 1 + 6 ⇥ 2 + 1 ⇥ 4 = 19.
4 Similarly, A4 = 23 A.
More generally,
Remarks
 n 1
(i) For all n ⇥ m matrix A: n2 2n 1
A = .
0p⇥n A = 0p⇥m and A0m⇥p = 0n⇥p . 2n 1 2n 1
(ii) Matrix Multiplication is not commutative, that is,
in general AB 6= BA. Remark
Powers of diagonal matrices are easily computed by taking
powers of diagonal terms:

2 3 2 k 3
d1 0 ... 0 k d1 0 ... 0
60 d2 ... 07 6 d2k 07
6 7 60 ... 7
6 7
6.
6. . . 7
7 =6 . . . 7
4. . . 5 6 . . . 7
. . 4 . . . 5
0 0 ... dn 0 0 ... dnk

Matrix Algebra
 Matrices Matrix Algebra

Inverse of a Matrix
Remark
If A and B are nonsingular n ⇥ n matrices, then AB is also
A square n ⇥ n matrix A is said to be nonsingular
nonsingular and
or invertible if there exists a matrix B such that
AB = BA = In
(AB) 1 = B 1 A 1 .
The matrix B is said to be the inverse of A and is
denoted by A 1 . Proof
A matrix is said to be singular if it admits no inverse. It is enough to show that the definition applies to the
matrices
If a matrix is nonsingular its inverse is unique.
M = AB and N = B 1 A 1 .
Examples
  Using the associative property of matrix product, it follows
(i) 2 3 1/10 3/10 that:
If A = and B = verify
4 1 2/5 1/5
that B = A 1 . 1 1 1 1 1
MN = (AB)(B A ) = A(BB )A = AIn A = In
Solution 1 1 1 1 1
One only needs to show that AB = BA = I2 . NM = (B A )(AB) = B (A A)B = B In B = I n .

(ii) 1 0
Show that the matrix A = is singular. Thus, the claim is proved.
0 0
Solution  Remarks
a b
In fact, for any 2 ⇥ 2 matrix B = , 1 If A is nonsingular, negative exponents of A may be
c d
   defined as follows:
a b 1 0 a 0
BA = = . A n = (A 1 )n , n 1.
c d 0 0 c 0
2 If A is nonsingular then for any n, An is also
Thus, for any 2 ⇥ 2 matrix B, BA 6= I2 and nonsingular and
consequently, A is singular.
(An ) 1 = A n .

Matrix Algebra