% Options for packages loaded elsewhere

\PassOptionsToPackage{unicode}{hyperref}

\PassOptionsToPackage{hyphens}{url}

\PassOptionsToPackage{dvipsnames,svgnames,x11names}{xcolor}

\documentclass[

12pt,

a4paper,

]{article}

\usepackage{xcolor}

\usepackage[a4paper,margin=2.5cm,heightrounded]{geometry}

\usepackage{amsmath,amssymb}

\setcounter{secnumdepth}{5}

\usepackage{iftex}

\ifPDFTeX

\usepackage[T1]{fontenc}

\usepackage[utf8]{inputenc}

\usepackage{textcomp} % provide euro and other symbols

\else % if luatex or xetex

\usepackage{unicode-math} % this also loads fontspec

\defaultfontfeatures{Scale=MatchLowercase}

\defaultfontfeatures[\rmfamily]{Ligatures=TeX,Scale=1}

\fi

\usepackage{lmodern}

\ifPDFTeX\else

% xetex/luatex font selection

\setmainfont[]{Noto Serif}

\setmonofont[]{Noto Sans Mono}

\fi

% Use upquote if available, for straight quotes in verbatim environments

\IfFileExists{upquote.sty}{\usepackage{upquote}}{}

\IfFileExists{microtype.sty}{% use microtype if available

\usepackage[]{microtype}

\UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts

}{}

\makeatletter

\@ifundefined{KOMAClassName}{% if non-KOMA class

\IfFileExists{parskip.sty}{%

\usepackage{parskip}

}{% else

\setlength{\parindent}{0pt}

\setlength{\parskip}{6pt plus 2pt minus 1pt}}

}{% if KOMA class

\KOMAoptions{parskip=half}}

\makeatother

\setlength{\emergencystretch}{3em} % prevent overfull lines

\providecommand{\tightlist}{%

\setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}

\usepackage{amsmath} % already pulled in by Pandoc, but safe

\renewenvironment{equation}{\begin{equation*}}{\end{equation*}}

\usepackage{microtype}

\newcommand{\docversion}{0.1.1}

\setcounter{secnumdepth}{0}

\setcounter{tocdepth}{2} % keep TOC depth as desired

\makeatletter

\renewcommand{\sectionmark}[1]{\markboth{#1}{}}

\renewcommand{\subsectionmark}[1]{\markright{#1}}

\makeatother

\usepackage{fancyhdr}

\fancyhf{}%

\fancyhead[LE,RO]{\thepage}%

\fancyhead[LO,RE]{\nouppercase{\leftmark}}%

\fancyfoot[C]{Version \docversion}%

\renewcommand{\headrulewidth}{0.4pt}%

\setlength{\headheight}{13.6pt}%

}

\usepackage{etoolbox}

\AtBeginDocument{\thispagestyle{plain}\pagestyle{content}}

\let\oldsection\section

\renewcommand{\section}{\clearpage\oldsection}

\usepackage{bookmark}

\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available

\urlstyle{same}

\hypersetup{

pdftitle={The Little Book of Linear Algebra},

pdfauthor={Duc-Tam Nguyen},

colorlinks=true,

linkcolor={blue},

filecolor={Maroon},

citecolor={Blue},

urlcolor={blue},

pdfcreator={LaTeX via pandoc}}

\title{The Little Book of Linear Algebra}

\usepackage{etoolbox}

\makeatletter

\providecommand{\subtitle}[1]{% add subtitle to \maketitle

\apptocmd{\@title}{\par {\large #1 \par}}{}{}

}

\makeatother

\subtitle{Version 0.1.0}

\author{Duc-Tam Nguyen}

\date{\today}

\begin{document}

\maketitle

{

\hypersetup{linkcolor=}

\setcounter{tocdepth}{2}

\tableofcontents

}

\section{Chapter 1. Vectors}\label{chapter-1-vectors}

\subsection{1.1 Scalars and Vectors}\label{11-scalars-and-vectors}

A scalar is a single numerical quantity, most often taken from the real

numbers, denoted by \(\mathbb{R}\). Scalars are the fundamental building

blocks of arithmetic: they can be added, subtracted, multiplied, and,

except in the case of zero, divided. In linear algebra, scalars play the

role of coefficients, scaling factors, and entries of larger structures

such as vectors and matrices. They provide the weights by which more

complex objects are measured and combined. A vector is an ordered

collection of scalars, arranged either in a row or a column. When the

scalars are real numbers, the vector is said to belong to \emph{real}

\(n\)-dimensional space, written

\[\mathbb{R}^n = \{ (x_1, x_2, \dots, x_n) \mid x_i \in \mathbb{R} \}.\]

An element of \(\mathbb{R}^n\) is called a vector of dimension \(n\) or

an \emph{n}-vector. The number \(n\) is called the dimension of the

vector space. Thus \(\mathbb{R}^2\) is the space of all ordered pairs of

real numbers, \(\mathbb{R}^3\) the space of all ordered triples, and so

on.

Example 1.1.1.

\begin{itemize}

\item

A 2-dimensional vector: \((3, -1) \in \mathbb{R}^2\).

\item

A 3-dimensional vector: \((2, 0, 5) \in \mathbb{R}^3\).

\item

A 1-dimensional vector: \((7) \in \mathbb{R}^1\), which corresponds to

the scalar \(7\) itself.

\end{itemize}

Vectors are often written vertically in column form, which emphasizes

their role in matrix multiplication:

\[\mathbf{v} = \begin{bmatrix} 2 \\ 0 \\ 5 \end{bmatrix} \in \mathbb{R}^3.\]

The vertical layout makes the structure clearer when we consider linear

combinations or multiply matrices by vectors.

\subsubsection{Geometric Interpretation}\label{geometric-interpretation}

In \(\mathbb{R}^2\), a vector \((x_1, x_2)\) can be visualized as an

arrow starting at the origin \((0,0)\) and ending at the point

\((x_1, x_2)\). Its length corresponds to the distance from the origin,

and its orientation gives a direction in the plane. In \(\mathbb{R}^3\),

the same picture extends into three dimensions: a vector is an arrow

from the origin to \((x_1, x_2, x_3)\). Beyond three dimensions, direct

visualization is no longer possible, but the algebraic rules of vectors

remain identical. Even though we cannot draw a vector in

\(\mathbb{R}^{10}\), it behaves under addition, scaling, and

transformation exactly as a 2- or 3-dimensional vector does. This

abstract point of view is what allows linear algebra to apply to data

science, physics, and machine learning, where data often lives in very

high-dimensional spaces. Thus a vector may be regarded in three

complementary ways:

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

As a point in space, described by its coordinates.

\item

As a displacement or arrow, described by a direction and a length.

\item

As an abstract element of a vector space, whose properties follow

algebraic rules independent of geometry.

\end{enumerate}

\subsubsection{Notation}\label{notation}

\begin{itemize}

\item

Vectors are written in boldface lowercase letters:

\(\mathbf{v}, \mathbf{w}, \mathbf{x}\).

\item

The \emph{i}-th entry of a vector \(\mathbf{v}\) is written \(v_i\),

where indices begin at 1.

\item

The set of all \emph{n}-dimensional vectors over \(\mathbb{R}\) is

denoted \(\mathbb{R}^n\).

\item

Column vectors will be the default form unless otherwise stated.

\end{itemize}

\subsubsection{Why begin here?}\label{why-begin-here}

Scalars and vectors form the atoms of linear algebra. Every structure we

will build-vector spaces, linear transformations, matrices,

eigenvalues-relies on the basic notions of number and ordered collection

of numbers. Once vectors are understood, we can define operations such

as addition and scalar multiplication, then generalize to subspaces,

bases, and coordinate systems. Eventually, this framework grows into the

full theory of linear algebra, with powerful applications to geometry,

computation, and data.

\subsubsection{Exercises 1.1}\label{exercises-11}

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Write three different vectors in \(\mathbb{R}^2\) and sketch them as

arrows from the origin. Identify their coordinates explicitly.

\item

Give an example of a vector in \(\mathbb{R}^4\). Can you visualize it

directly? Explain why high-dimensional visualization is challenging.

\item

Let \(\mathbf{v} = (4, -3, 2)\). Write \(\mathbf{v}\) in column form

and state \(v_1, v_2, v_3\).

\item

In what sense is the set \(\mathbb{R}^1\) both a line and a vector

space? Illustrate with examples.

\item

Consider the vector \(\mathbf{u} = (1,1,\dots,1) \in \mathbb{R}^n\).

What is special about this vector when \(n\) is large? What might it

represent in applications?

\end{enumerate}

\subsection{1.2 Vector Addition and Scalar

Multiplication}\label{12-vector-addition-and-scalar-multiplication}

Vectors in linear algebra are not static objects; their power comes from

the operations we can perform on them. Two fundamental operations define

the structure of vector spaces: addition and scalar multiplication.

These operations satisfy simple but far-reaching rules that underpin the

entire subject.

\subsubsection{Vector Addition}\label{vector-addition}

Given two vectors of the same dimension, their sum is obtained by adding

corresponding entries. Formally, if

\[\mathbf{u} = (u_1, u_2, \dots, u_n), \quad

\mathbf{v} = (v_1, v_2, \dots, v_n),\]

then their sum is

\[\mathbf{u} + \mathbf{v} = (u_1+v_1, u_2+v_2, \dots, u_n+v_n).\]

Example 1.2.1.\\

Let \(\mathbf{u} = (2, -1, 3)\) and \(\mathbf{v} = (4, 0, -5)\). Then

\[\mathbf{u} + \mathbf{v} = (2+4, -1+0, 3+(-5)) = (6, -1, -2).\]

Geometrically, vector addition corresponds to the \emph{parallelogram

rule}. If we draw both vectors as arrows from the origin, then placing

the tail of one vector at the head of the other produces the sum. The

diagonal of the parallelogram they form represents the resulting vector.

\subsubsection{Scalar Multiplication}\label{scalar-multiplication}

Multiplying a vector by a scalar stretches or shrinks the vector while

preserving its direction, unless the scalar is negative, in which case

the vector is also reversed. If \(c \in \mathbb{R}\) and

\[\mathbf{v} = (v_1, v_2, \dots, v_n),\]

then

\[c \mathbf{v} = (c v_1, c v_2, \dots, c v_n).\]

Example 1.2.2.\\

Let \(\mathbf{v} = (3, -2)\) and \(c = -2\). Then

\[c\mathbf{v} = -2(3, -2) = (-6, 4).\]

This corresponds to flipping the vector through the origin and doubling

its length.

\subsubsection{Linear Combinations}\label{linear-combinations}

The interaction of addition and scalar multiplication allows us to form

\emph{linear combinations}. A linear combination of vectors

\(\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\) is any vector of the

form

\[c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_k \mathbf{v}_k, \quad c_i \in \mathbb{R}.\]

Linear combinations are the mechanism by which we generate new vectors

from existing ones. The span of a set of vectors-the collection of all

their linear combinations-will later lead us to the idea of a subspace.

Example 1.2.3.\\

Let \(\mathbf{v}_1 = (1,0)\) and \(\mathbf{v}_2 = (0,1)\). Then any

vector \((a,b)\in\mathbb{R}^2\) can be expressed as

\[a\mathbf{v}_1 + b\mathbf{v}_2.\]

Thus \((1,0)\) and \((0,1)\) form the basic building blocks of the

plane.

\subsubsection{Notation}\label{notation-2}

\begin{itemize}

\item

Addition: \(\mathbf{u} + \mathbf{v}\) means component-wise addition.

\item

Scalar multiplication: \(c\mathbf{v}\) scales each entry of

\(\mathbf{v}\) by \(c\).

\item

Linear combination: a sum of the form

\(c_1 \mathbf{v}_1 + \cdots + c_k \mathbf{v}_k\).

\end{itemize}

\subsubsection{Why this matters}\label{why-this-matters}

Vector addition and scalar multiplication are the defining operations of

linear algebra. They give structure to vector spaces, allow us to

describe geometric phenomena like translation and scaling, and provide

the foundation for solving systems of equations. Everything that

follows-basis, dimension, transformations-builds on these simple but

profound rules.

\subsubsection{Exercises 1.2}\label{exercises-12}

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Compute \(\mathbf{u} + \mathbf{v}\) where \(\mathbf{u} = (1,2,3)\) and

\(\mathbf{v} = (4, -1, 0)\).

\item

Find \(3\mathbf{v}\) where \(\mathbf{v} = (-2,5)\). Sketch both

vectors to illustrate the scaling.

\item

Show that \((5,7)\) can be written as a linear combination of

\((1,0)\) and \((0,1)\).

\item

Write \((4,4)\) as a linear combination of \((1,1)\) and \((1,-1)\).

\item

Prove that if \(\mathbf{u}, \mathbf{v} \in \mathbb{R}^n\), then

\((c+d)(\mathbf{u}+\mathbf{v}) = c\mathbf{u} + c\mathbf{v} + d\mathbf{u} + d\mathbf{v}\)

for scalars \(c,d \in \mathbb{R}\).

\end{enumerate}

\subsection{1.3 Dot Product, Norms, and

Angles}\label{13-dot-product-norms-and-angles}

The dot product is the fundamental operation that links algebra and

geometry in vector spaces. It allows us to measure lengths, compute

angles, and determine orthogonality. From this single definition flow

the notions of \emph{norm} and \emph{angle}, which give geometry to

abstract vector spaces.

\subsubsection{The Dot Product}\label{the-dot-product}

For two vectors in \(\mathbb{R}^n\), the dot product (also called the

inner product) is defined by

\[\mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n.\]

Equivalently, in matrix notation:

\[\mathbf{u} \cdot \mathbf{v} = \mathbf{u}^T \mathbf{v}.\]

Example 1.3.1.\\

Let \(\mathbf{u} = (2, -1, 3)\) and \(\mathbf{v} = (4, 0, -2)\). Then

\[\mathbf{u} \cdot \mathbf{v} = 2\cdot 4 + (-1)\cdot 0 + 3\cdot (-2) = 8 - 6 = 2.\]

The dot product outputs a single scalar, not another vector.

\subsubsection{Norms (Length of a

Vector)}\label{norms-length-of-a-vector}

The \emph{Euclidean norm} of a vector is the square root of its dot

product with itself:

\[\|\mathbf{v}\| = \sqrt{\mathbf{v} \cdot \mathbf{v}} = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}.\]

This generalizes the Pythagorean theorem to arbitrary dimensions.

Example 1.3.2.\\

For \(\mathbf{v} = (3, 4)\),

\[\|\mathbf{v}\| = \sqrt{3^2 + 4^2} = \sqrt{25} = 5.\]

This is exactly the length of the vector as an arrow in the plane.

\subsubsection{Angles Between Vectors}\label{angles-between-vectors}

The dot product also encodes the angle between two vectors. For nonzero

vectors \(\mathbf{u}, \mathbf{v}\),

\[\mathbf{u} \cdot \mathbf{v} = \|\mathbf{u}\| \, \|\mathbf{v}\| \cos \theta,\]

where \(\theta\) is the angle between them. Thus,

\[\cos \theta = \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\|\|\mathbf{v}\|}.\]

Example 1.3.3.\\

Let \(\mathbf{u} = (1,0)\) and \(\mathbf{v} = (0,1)\). Then

\[\mathbf{u} \cdot \mathbf{v} = 0, \quad \|\mathbf{u}\| = 1, \quad \|\mathbf{v}\| = 1.\]

Hence

\[\cos \theta = \frac{0}{1\cdot 1} = 0 \quad \Rightarrow \quad \theta = \frac{\pi}{2}.\]

The vectors are perpendicular.

\subsubsection{Orthogonality}\label{orthogonality}

Two vectors are said to be orthogonal if their dot product is zero:

\[\mathbf{u} \cdot \mathbf{v} = 0.\]

Orthogonality generalizes the idea of perpendicularity from geometry to

higher dimensions.

\subsubsection{Notation}\label{notation-3}

\begin{itemize}

\item

Dot product: \(\mathbf{u} \cdot \mathbf{v}\).

\item

Norm (length): \(|\mathbf{v}|\).

\item

Orthogonality: \(\mathbf{u} \perp \mathbf{v}\) if

\(\mathbf{u} \cdot \mathbf{v} = 0\).

\end{itemize}

\subsubsection{Why this matters}\label{why-this-matters-2}

The dot product turns vector spaces into geometric objects: vectors gain

lengths, angles, and notions of perpendicularity. This foundation will

later support the study of orthogonal projections, Gram--Schmidt

orthogonalization, eigenvectors, and least squares problems.

\subsubsection{Exercises 1.3}\label{exercises-13}

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Compute \(\mathbf{u} \cdot \mathbf{v}\) for \(\mathbf{u} = (1,2,3)\),

\(\mathbf{v} = (4,5,6)\).

\item

Find the norm of \(\mathbf{v} = (2, -2, 1)\).

\item

Determine whether \(\mathbf{u} = (1,1,0)\) and

\(\mathbf{v} = (1,-1,2)\) are orthogonal.

\item

Let \(\mathbf{u} = (3,4)\), \(\mathbf{v} = (4,3)\). Compute the angle

between them.

\item

Prove that

\(|\mathbf{u} + \mathbf{v}|^2 = |\mathbf{u}|^2 + |\mathbf{v}|^2 + 2\mathbf{u}\cdot \mathbf{v}\).

This identity is the algebraic version of the Law of Cosines.

\end{enumerate}

\subsection{1.4 Orthogonality}\label{14-orthogonality}

Orthogonality captures the notion of perpendicularity in vector spaces.

It is one of the most important geometric ideas in linear algebra,

allowing us to decompose vectors, define projections, and construct

special bases with elegant properties.

\subsubsection{Definition}\label{definition}

Two vectors \(\mathbf{u}, \mathbf{v} \in \mathbb{R}^n\) are said to be

orthogonal if their dot product is zero:

\[\mathbf{u} \cdot \mathbf{v} = 0.\]

This condition ensures that the angle between them is \(\pi/2\) radians

(90 degrees).

Example 1.4.1.\\

In \(\mathbb{R}^2\), the vectors \((1,2)\) and \((2,-1)\) are orthogonal

since

\[(1,2) \cdot (2,-1) = 1\cdot 2 + 2\cdot (-1) = 0.\]

\subsubsection{Orthogonal Sets}\label{orthogonal-sets}

A collection of vectors is called orthogonal if every distinct pair of

vectors in the set is orthogonal. If, in addition, each vector has norm

1, the set is called orthonormal.

Example 1.4.2.\\

In \(\mathbb{R}^3\), the standard basis vectors

\[\mathbf{e}_1 = (1,0,0), \quad \mathbf{e}_2 = (0,1,0), \quad \mathbf{e}_3 = (0,0,1)\]

form an orthonormal set: each has length 1, and their dot products

vanish when the indices differ.

\subsubsection{Projections}\label{projections}

Orthogonality makes possible the decomposition of a vector into two

components: one parallel to another vector, and one orthogonal to it.

Given a nonzero vector \(\mathbf{u}\) and any vector \(\mathbf{v}\), the

projection of \(\mathbf{v}\) onto \(\mathbf{u}\) is

\[\text{proj}_{\mathbf{u}}(\mathbf{v}) = \frac{\mathbf{u} \cdot \mathbf{v}}{\mathbf{u} \cdot \mathbf{u}} \mathbf{u}.\]

The difference

\[\mathbf{v} - \text{proj}_{\mathbf{u}}(\mathbf{v})\]

is orthogonal to \(\mathbf{u}\). Thus every vector can be decomposed

uniquely into a parallel and perpendicular part with respect to another

vector.

Example 1.4.3.\\

Let \(\mathbf{u} = (1,0)\), \(\mathbf{v} = (2,3)\). Then

\[\text{proj}_{\mathbf{u}}(\mathbf{v}) = \frac{(1,0)\cdot(2,3)}{(1,0)\cdot(1,0)} (1,0)

= \frac{2}{1}(1,0) = (2,0).\]

Thus

\[\mathbf{v} = (2,3) = (2,0) + (0,3),\]

where \((2,0)\) is parallel to \((1,0)\) and \((0,3)\) is orthogonal to

it.

\subsubsection{Orthogonal Decomposition}\label{orthogonal-decomposition}

In general, if \(\mathbf{u} \neq \mathbf{0}\) and

\(\mathbf{v} \in \mathbb{R}^n\), then

\[\mathbf{v} = \text{proj}_{\mathbf{u}}(\mathbf{v}) + \big(\mathbf{v} - \text{proj}_{\mathbf{u}}(\mathbf{v})\big),\]

where the first term is parallel to \(\mathbf{u}\) and the second term

is orthogonal. This decomposition underlies methods such as least

squares approximation and the Gram--Schmidt process.

\subsubsection{Notation}\label{notation-4}

\begin{itemize}

\item

\(\mathbf{u} \perp \mathbf{v}\): vectors \(\mathbf{u}\) and

\(\mathbf{v}\) are orthogonal.

\item

An orthogonal set: vectors pairwise orthogonal.

\item

An orthonormal set: pairwise orthogonal, each of norm 1.

\end{itemize}

\subsubsection{Why this matters}\label{why-this-matters-3}

Orthogonality gives structure to vector spaces. It provides a way to

separate independent directions cleanly, simplify computations, and

minimize errors in approximations. Many powerful algorithms in numerical

linear algebra and data science (QR decomposition, least squares

regression, PCA) rely on orthogonality.

\subsubsection{Exercises 1.4}\label{exercises-14}

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Verify that the vectors \((1,2,2)\) and \((2,0,-1)\) are orthogonal.

\item

Find the projection of \((3,4)\) onto \((1,1)\).

\item

Show that any two distinct standard basis vectors in \(\mathbb{R}^n\)

are orthogonal.

\item

Decompose \((5,2)\) into components parallel and orthogonal to

\((2,1)\).

\item

Prove that if \(\mathbf{u}, \mathbf{v}\) are orthogonal and nonzero,

then \((\mathbf{u}+\mathbf{v})\cdot(\mathbf{u}-\mathbf{v}) = 0\).

\end{enumerate}

\section{Chapter 2. Matrices}\label{chapter-2-matrices}

\subsection{2.1 Definition and

Notation}\label{21-definition-and-notation}

Matrices are the central objects of linear algebra, providing a compact

way to represent and manipulate linear transformations, systems of

equations, and structured data. A matrix is a rectangular array of

numbers arranged in rows and columns.

\subsubsection{Formal Definition}\label{formal-definition}

An \(m \times n\) matrix is an array with \(m\) rows and \(n\) columns,

written

\[A =

\begin{bmatrix}

a_{11} & a_{12} & \cdots & a_{1n} \\

a_{21} & a_{22} & \cdots & a_{2n} \\

\vdots & \vdots & \ddots & \vdots \\

a_{m1} & a_{m2} & \cdots & a_{mn}

\end{bmatrix}.\]

Each entry \(a_{ij}\) is a scalar, located in the \emph{i}-th row and

\emph{j}-th column. The size (or dimension) of the matrix is denoted by

\(m \times n\).

\begin{itemize}

\item

If \(m = n\), the matrix is square.

\item

If \(m = 1\), the matrix is a row vector.

\item

If \(n = 1\), the matrix is a column vector.

\end{itemize}

Thus, vectors are simply special cases of matrices.

\subsubsection{Examples}\label{examples}

Example 2.1.1. A \(2 \times 3\) matrix:

\[A = \begin{bmatrix}

1 & -2 & 4 \\

0 & 3 & 5

\end{bmatrix}.\]

Here, \(a_{12} = -2\), \(a_{23} = 5\), and the matrix has 2 rows, 3

columns.

Example 2.1.2. A \(3 \times 3\) square matrix:

\[B = \begin{bmatrix}

2 & 0 & 1 \\

-1 & 3 & 4 \\

0 & 5 & -2

\end{bmatrix}.\]

This will later serve as the representation of a linear transformation

on \(\mathbb{R}^3\).

\subsubsection{Indexing and Notation}\label{indexing-and-notation}

\begin{itemize}

\item

Matrices are denoted by uppercase bold letters: \(A, B, C\).

\item

Entries are written as \(a_{ij}\), with the row index first, column

index second.

\item

The set of all real \(m \times n\) matrices is denoted

\(\mathbb{R}^{m \times n}\).

\end{itemize}

Thus, a matrix is a function

\(A: {1,\dots,m} \times {1,\dots,n} \to \mathbb{R}\), assigning a scalar

to each row-column position.

\subsubsection{Why this matters}\label{why-this-matters-4}

Matrices generalize vectors and give us a language for describing linear

operations systematically. They encode systems of equations, rotations,

projections, and transformations of data. With matrices, algebra and

geometry come together: a single compact object can represent both

numerical data and functional rules.

\subsubsection{Exercises 2.1}\label{exercises-21}

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Write a \(3 \times 2\) matrix of your choice and identify its entries

\(a_{ij}\).

\item

Is every vector a matrix? Is every matrix a vector? Explain.

\item

Which of the following are square matrices:

\(A \in \mathbb{R}^{4\times4}\), \(B \in \mathbb{R}^{3\times5}\),

\(C \in \mathbb{R}^{1\times1}\)?

\item

Let \(D = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}\). What kind of

matrix is this?

\item

Consider the matrix

\(E = \begin{bmatrix} a & b \\ c & d \end{bmatrix}\). Express

\(e_{11}, e_{12}, e_{21}, e_{22}\) explicitly.

\end{enumerate}

\subsection{2.2 Matrix Addition and

Multiplication}\label{22-matrix-addition-and-multiplication}

Once matrices are defined, the next step is to understand how they

combine. Just as vectors gain meaning through addition and scalar

multiplication, matrices become powerful through two operations:

addition and multiplication.

\subsubsection{Matrix Addition}\label{matrix-addition}

Two matrices of the same size are added by adding corresponding entries.

If

\[A = [a_{ij}] \in \mathbb{R}^{m \times n}, \quad

B = [b_{ij}] \in \mathbb{R}^{m \times n},\]

then

\[A + B = [a_{ij} + b_{ij}] \in \mathbb{R}^{m \times n}.\]

Example 2.2.1.\\

Let

\[A = \begin{bmatrix}

1 & 2 \\

3 & 4

\end{bmatrix}, \quad

B = \begin{bmatrix}

-1 & 0 \\

5 & 2

\end{bmatrix}.\]

Then

\[A + B = \begin{bmatrix}

1 + (-1) & 2 + 0 \\

3 + 5 & 4 + 2

\end{bmatrix}

=

\begin{bmatrix}

0 & 2 \\

8 & 6

\end{bmatrix}.\]

Matrix addition is commutative (\(A+B = B+A\)) and associative

(\((A+B)+C = A+(B+C)\)). The zero matrix, with all entries 0, acts as

the additive identity.

\subsubsection{Scalar Multiplication}\label{scalar-multiplication-2}

For a scalar \(c \in \mathbb{R}\) and a matrix \(A = [[a_{ij}]\), we

define

\[cA = [c \cdot a_{ij}].\]

This stretches or shrinks all entries of the matrix uniformly.

Example 2.2.2.\\

If

\[A = \begin{bmatrix}

2 & -1 \\

0 & 3

\end{bmatrix}, \quad c = -2,\]

then

\[cA = \begin{bmatrix}

-4 & 2 \\

0 & -6

\end{bmatrix}.\]

\subsubsection{Matrix Multiplication}\label{matrix-multiplication}

The defining operation of matrices is multiplication. If

\[A \in \mathbb{R}^{m \times n}, \quad B \in \mathbb{R}^{n \times p},\]

then their product is the \(m \times p\) matrix

\[AB = C = [c_{ij}], \quad c_{ij} = \sum_{k=1}^n a_{ik} b_{kj}.\]

Thus, the entry in the \(i\)-th row and \(j\)-th column of \(AB\) is the

dot product of the \(i\)-th row of \(A\) with the \(j\)-th column of

\(B\).

Example 2.2.3.\\

Let

\[A = \begin{bmatrix}

1 & 2 \\

0 & 3

\end{bmatrix}, \quad

B = \begin{bmatrix}

4 & -1 \\

2 & 5

\end{bmatrix}.\]

Then

\[AB = \begin{bmatrix}

1\cdot4 + 2\cdot2 & 1\cdot(-1) + 2\cdot5 \\

0\cdot4 + 3\cdot2 & 0\cdot(-1) + 3\cdot5

\end{bmatrix}

=

\begin{bmatrix}

8 & 9 \\

6 & 15

\end{bmatrix}.\]

Notice that matrix multiplication is not commutative in general:

\(AB \neq BA\). Sometimes \(BA\) may not even be defined if dimensions

do not align.

\subsubsection{Geometric Meaning}\label{geometric-meaning}

Matrix multiplication corresponds to the composition of linear

transformations. If \(A\) transforms vectors in \(\mathbb{R}^n\) and

\(B\) transforms vectors in \(\mathbb{R}^p\), then \(AB\) represents

applying \(B\) first, then \(A\). This makes matrices the algebraic

language of transformations.

\subsubsection{Notation}\label{notation-5}

\begin{itemize}

\item

Matrix sum: \(A+B\).

\item

Scalar multiple: \(cA\).

\item

Product: \(AB\), defined only when the number of columns of \(A\)

equals the number of rows of \(B\).

\end{itemize}

\subsubsection{Why this matters}\label{why-this-matters-5}

Matrix multiplication is the core mechanism of linear algebra: it

encodes how transformations combine, how systems of equations are

solved, and how data flows in modern algorithms. Addition and scalar

multiplication make matrices into a vector space, while multiplication

gives them an algebraic structure rich enough to model geometry,

computation, and networks.

\subsubsection{Exercises 2.2}\label{exercises-22}

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Compute \(A+B\) for

\end{enumerate}

\[A = \begin{bmatrix} 2 & 3 \\ -1 & 0 \end{bmatrix}, \quad

B = \begin{bmatrix} 4 & -2 \\ 5 & 7 \end{bmatrix}.\]

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Find \(3A\) where

\end{enumerate}

\[A = \begin{bmatrix} 1 & -4 \\ 2 & 6 \end{bmatrix}.\]

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Multiply

\end{enumerate}

\[A = \begin{bmatrix} 1 & 0 & 2 \\ -1 & 3 & 1 \end{bmatrix}, \quad

B = \begin{bmatrix} 2 & 1 \\ 0 & -1 \\ 3 & 4 \end{bmatrix}.\]

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Verify with an explicit example that \(AB \neq BA\).

\item

Prove that matrix multiplication is distributive:

\(A(B+C) = AB + AC\).

\end{enumerate}

\subsection{2.3 Transpose and Inverse}\label{23-transpose-and-inverse}

Two special operations on matrices-the transpose and the inverse-give

rise to deep algebraic and geometric properties. The transpose

rearranges a matrix by flipping it across its main diagonal, while the

inverse, when it exists, acts as the undo operation for matrix

multiplication.

\subsubsection{The Transpose}\label{the-transpose}

The transpose of an \(m \times n\) matrix \(A = [a_{ij}]\) is the

\(n \times m\) matrix \(A^T = [a_{ji}]\), obtained by swapping rows and

columns.

Formally,

\[(A^T)_{ij} = a_{ji}.\]

Example 2.3.1.\\

If

\[A = \begin{bmatrix}

1 & 4 & -2 \\

0 & 3 & 5

\end{bmatrix},\]

then

\[A^T = \begin{bmatrix}

1 & 0 \\

4 & 3 \\

-2 & 5

\end{bmatrix}.\]

Properties of the Transpose.

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

\( (A^T)^T = A\).

\item

\( (A+B)^T = A^T + B^T\).

\item

\( (cA)^T = cA^T\), for scalar \(c\).

\item

\( (AB)^T = B^T A^T\).

\end{enumerate}

The last rule is crucial: the order reverses.

\subsubsection{The Inverse}\label{the-inverse}

A square matrix \(A \in \mathbb{R}^{n \times n}\) is said to be

invertible (or nonsingular) if there exists another matrix \(A^{-1}\)

such that

\[AA^{-1} = A^{-1}A = I_n,\]

where \(I_n\) is the \(n \times n\) identity matrix. In this case,

\(A^{-1}\) is called the inverse of \(A\).

Not every matrix is invertible. A necessary condition is that

\(\det(A) \neq 0\), a fact that will be developed in Chapter 6.

Example 2.3.2.\\

Let

\[A = \begin{bmatrix}

1 & 2 \\

3 & 4

\end{bmatrix}.\]

Its determinant is \(\det(A) = (1)(4) - (2)(3) = -2 \neq 0\). The

inverse is

\[A^{-1} = \frac{1}{\det(A)} \begin{bmatrix}

4 & -2 \\

-3 & 1

\end{bmatrix}

=

\begin{bmatrix}

-2 & 1 \\

1.5 & -0.5

\end{bmatrix}.\]

Verification:

\[AA^{-1} = \begin{bmatrix}

1 & 2 \\

3 & 4

\end{bmatrix}

\begin{bmatrix}

-2 & 1 \\

1.5 & -0.5

\end{bmatrix}

=

\begin{bmatrix}

1 & 0 \\

0 & 1

\end{bmatrix}.\]

\subsubsection{Geometric Meaning}\label{geometric-meaning-2}

\begin{itemize}

\item

The transpose corresponds to reflecting a linear transformation across

the diagonal. For vectors, it switches between row and column forms.

\item

The inverse, when it exists, corresponds to reversing a linear

transformation. For example, if \(A\) scales and rotates vectors,

\(A^{-1}\) rescales and rotates them back.

\end{itemize}

\subsubsection{Notation}\label{notation-6}

\begin{itemize}

\item

Transpose: \(A^T\).

\item

Inverse: \(A^{-1}\), defined only for invertible square matrices.

\item

Identity: \(I_n\), acts as the multiplicative identity.

\end{itemize}

\subsubsection{Why this matters}\label{why-this-matters-6}

The transpose allows us to define symmetric and orthogonal matrices,

central to geometry and numerical methods. The inverse underlies the

solution of linear systems, encoding the idea of undoing a

transformation. Together, these operations set the stage for

determinants, eigenvalues, and orthogonalization.

\subsubsection{Exercises 2.3}\label{exercises-23}

\begin{enumerate}

\def\labelenumi{\arabic{enumi}.}

\item

Compute the transpose of

\end{enumerate}

\[A = \begin{bmatrix} 2 & -1 & 3 \\ 0 & 4 & 5 \end{bmatrix}.\]

\begin{enumerate}