Fuzzy expert systems:

Fuzzy logic
 Introduction, or what is fuzzy thinking?
 Fuzzy sets
 Linguistic variables and hedges
 Operations of fuzzy sets
 Fuzzy rules
 Summary
Introduction, or what is fuzzy thinking?
 Experts rely on common sense when they
solve
problems.
 How can we represent expert knowledge
that
uses vague and ambiguous terms in a
computer?
 Fuzzy logic is not logic that is fuzzy, but
logic that
is used to describe fuzziness. Fuzzy logic is
the
 Boolean logic uses sharp distinctions. It forces us
to draw lines between members of a class and non-
members. For instance, we may say, Tom is tall
because his height is 181 cm. If we drew a line at
180 cm, we would find that David, who is 179 cm,
is small. Is David really a small man or we have
just drawn an arbitrary line in the sand?
 Fuzzy logic reflects how people think. It attempts
to model our sense of words, our decision making
and our common sense. As a result, it is leading to
new, more human, intelligent systems.
 Fuzzy, or multi-valued logic was introduced in the
1930s by Jan Lukasiewicz , a Polish philosopher.
While classical logic operates with only two values
1 (true) and 0 (false), Lukasiewicz introduced logic
that extended the range of truth values to all real
numbers in the interval between 0 and 1. He used a
number in this interval to represent the possibility
that a given statement was true or false. For
example, the possibility that a man 181 cm tall is
really tall might be set to a value of 0.86. It is
likely that the man is tall. This work led to an
inexact reasoning technique often called possibility
theory.
 Later, in 1937, Max Black published a paper called
“Vagueness: an exercise in logical analysis”. In
this paper, he argued that a continuum implies
degrees. Imagine, he said, a line of countless
“chairs”. At one end is a Chippendale. Next to it is
a near-Chippendale, in fact indistinguishable
from the first item. Succeeding “chairs” are less
and less chair-like, until the line ends with a log.
When does a chair become a log? Max Black
stated that if a continuum is discrete, a number
can be allocated to each element. He
accepted vagueness as a matter of
probability.
 In 1965 Lotfi Zadeh, published his famous paper
“Fuzzy sets”. Zadeh extended the work on
possibility theory into a formal system of
mathematical logic, and introduced a new concept
for applying natural language terms. This new logic
for representing and manipulating fuzzy terms was
called fuzzy logic, and Zadeh became the Master of
fuzzy logic.
 Why fuzzy?
As Zadeh said, the term is concrete, immediate and
descriptive; we all know what it means.
However, many people in the West
were repelled by the word fuzzy ,
because
 Why it is usually used in a negative sense.
logic?
Fuzziness rests on fuzzy set theory, and fuzzy logic
is just a small part of that theory.
Fuzzy logic is a set of mathematical
principles
for knowledge representation based
on degrees
of membership.

Unlike two-valued Boolean logic, fuzzy

logic is
multi-valued. It deals with degrees of
membership and degrees of truth.
Fuzzy logic
uses the continuum of logical values
between 0
(completely false) and 1 (completely true).
Range of logical values in Boolean and
fuzzy logic

0 01 0 1 1 0 0 0.2 0.4 0.6 0.8 1 1

(a) Boolean Logic. .
(b) Multi-valued Logic
 Fuzzy sets
 The concept of a set is fundamental to
mathematics.
 However, our own language is also the supreme
expression of sets. For example, car indicates the set
of cars. When we say a car , we mean one out of the
set of cars.
 The classical example in fuzzy sets is tall men.
The elements of the fuzzy set “tall men” are all
men, but their degrees of membership depend on
their height.
Crisp and fuzzy sets of “tall
Degree of
men”
Membership
Crisp Sets
1.0

0.8

0.6

0.4

0.2

0.0
150 160 170 180 190 200 210
Height, cm
Degree of
Fuzzy Sets
Membership
1.0

0.8

0.6

0.4

0.2

0.0
150 160 170 180 190 200 210
Height, cm
 The x-axis represents the universe of discourse –
the range of all possible values applicable to a
chosen variable. In our case, the variable is the man
height. According to this representation, the
universe of men’s heights consists of all tall men.
 The y-axis represents the membership value of the
fuzzy set. In our case, the fuzzy set of “tall men”
maps height values into corresponding membership
values.
 A fuzzy set is a set with fuzzy
boundaries.
 Let X be the universe of discourse and its elements
be denoted as x. In the classical set theory, crisp
set A of X is defined as function fA(x) called the
characteristic function of A
1, if x  A
fA(x): X ® {0, 1}, wheref A ( x) 
0, if x  A

This set maps universe X to a set of two elements.

For any element x of universe X, characteristic
function fA(x) is equal to 1 if x is an element of set
A, and is equal to 0 if x is not an element of A.
 In the fuzzy theory, fuzzy set A of universe X is
defined by function mA(x) called the membership
function of set A
mA(x): X ® [0, 1], where mA(x) = 1 if x is totally in A;
mA (x) = 0 if x is not in A;
0 < mA (x) < 1 if x is partly in A.
This set allows a continuum of possible choices.
For any element x of universe X, membership
function mA(x) equals the degree to which x is an
element of set A. This degree, a value between 0
and 1, represents the degree of membership, also
called membership value, of element x in set A.
How to represent a fuzzy set in a
computer?
 First, we determine the membership functions. In
our “tall men” example, we can obtain fuzzy sets of
tall, short and average men.
 The universe of discourse – the men’s heights –
consists of three sets: short, average and tall men.
As you will see, a man who is 184 cm tall is a
member of the average men set with a degree of
membership of 0.1, and at the same time, he is also
a member of the tall men set with a degree of 0.4.
p and fuzzy sets of short, average and tall m
Degreeof CrispSets
Membership
1.0
0.8 Short Average Tall

0.6
0.4
0.2

0.0
150 160 170 180 190 200 210
Height, cm
Degreeof Fuzzy Sets
Membership
1.0

0.8
0.6 Short Average Tall

0.4
0.2

0.0
150 160 170 180 190 200 210
 Representation of crisp and
fuzzy subsets
 (x)
X FuzzySubsetA
1

0
Crisp SubsetA Fuzziness x

cal functions that can be used to represent a fuz

are sigmoid, gaussian and pi. However, these
ctions increase the time of computation. Therefor
ractice, most applications use linear fit functio
 Linguistic variables and
hedges
 At the root of fuzzy set theory lies the idea of
linguistic variables.
 A linguistic variable is a fuzzy variable. For
example, the statement “John is tall” implies that
the linguistic variable John takes the linguistic value
tall.
In fuzzy expert systems, linguistic variables are used
in fuzzy rules. For example:
IF wind is strong
THEN sailing is good

IF project_duration is long
THEN completion_risk is high

IF speed is slow
THEN stopping_distance is short
 The range of possible values of a linguistic variable
represents the universe of discourse of that variable.
For example, the universe of discourse of the
linguistic variable speed might have the range
between 0 and 220 km/h and may include such
fuzzy subsets as very slow, slow, medium, fast, and
very fast.
 A linguistic variable carries with it the concept
of fuzzy set qualifiers, called hedges.
 Hedges are terms that modify the shape of fuzzy
sets. They include adverbs such as very,
somewhat, quite, more or less and slightly.
 Fuzzy sets with the hedge
Degreeof very
Membership
1.0
Short Short
Tall
0.8

0.6 Average

0.4
Very Shor
t Very
VeryTall
Tall
Tall
0.2

0.0
150 160 170 180 190 200 210
Height, cm
Representation of hedges in
Hedge fuzzy
atical logic
Mathem Graphical Representa
tion
Expression

A little [A(x)]1.3

Slightly [A(x)]1.7

Very [A (x)]2

Extremely [A(x)]3
Representation of hedges in fuzzy
Hedge
logic (continued)
Mathematical
Graphical Representation
Expression

Very very [A ( x)]4

More or less A ( x)

Somewhat A ( x)

2 [A (x )]2
if 0  A  0.5
Indeed
1  2 [1  A ( x)]2
if 0.5 < A  1
 Operations of fuzzy sets
The classical set theory developed in the
late 19th
century by Georg Cantor describes how
crisp sets can
interact. These interactions are called
operations.
 Cantor’s sets
Not A
B

A A
A

Complement Containment

A B A
A B

Intersection Union
 Complement
Crisp Sets: Who does not belong to the set?
Fuzzy Sets: How much do elements not belong
to
the set?
The complement of a set is an opposite of this set.
For example, if we have the set of tall men, its
complement is the set of NOT tall men. When we
remove the tall men set from the universe of
discourse, we obtain the complement. If A is the
fuzzy set, its complement ØA can be found as
follows:
mØ (x) = 1 - m (x)
A A
 Containment
Crisp Sets: Which sets belong to which other sets?
Fuzzy Sets: Which sets belong to other sets?
Similar to a Chinese box, a set can contain other
sets. The smaller set is called the subset. For
example, the set of tall men contains all tall men;
very tall men is a subset of tall men. However, the
tall men set is just a subset of the set of men. In
crisp sets, all elements of a subset entirely belong to
a larger set. In fuzzy sets, however, each element
can belong less to the subset than to the larger set.
Elements of the fuzzy subset have smaller
memberships in it than in the larger set.
 Intersection
Crisp Sets: Which element belongs to both sets?
Fuzzy Sets: How much of the element is in both
sets?
In classical set theory, an intersection between two
sets contains the elements shared by these sets. For
example, the intersection of the set of tall men and
the set of fat men is the area where these sets
overlap. In fuzzy sets, an element may partly
belong to both sets with different memberships. A
fuzzy intersection is the lower membership in both
sets of each element. The fuzzy intersection of two
fuzzy sets A and B on universe of discourse X:
mAÇB(x) = min [mA (x), mB (x)] = mA (x) Ç mB(x),
where xÎX
 Union
Crisp Sets: Which element belongs to either set?
Fuzzy Sets: How much of the element is in either
set?
The union of two crisp sets consists of every element
that falls into either set. For example, the union of
tall men and fat men contains all men who are tall
OR fat. In fuzzy sets, the union is the reverse of the
intersection. That is, the union is the largest
membership value of the element in either set. The
fuzzy operation for forming the union of two fuzzy
sets
m ÈA(x)
and= B on [m
max universe
(x), mX(x)]
can=bem given
(x) Èas:m (x),
A B A B A B
where xÎX
Operations of fuzzy sets
 (x)  (x)
B
1 1 A
A
0 0
x x
B
1 1 A
Not A
0 0
Complement x Containment x

 (x)  (x)

1 1
AB AB
0 0
x x
1 AB 1
AB
0 0
x x
Intersection Union
 Fuzzy rules
In 1973, Lotfi Zadeh published his
second most
influential paper. This paper outlined a
new
approach to analysis of complex systems,
in which
Zadeh suggested capturing human
knowledge in
fuzzy rules.
 What is a fuzzy rule?
A fuzzy rule can be defined as a conditional
statement in the form:

IF x is A
THEN y is B

where x and y are linguistic variables; and A and B

are linguistic values determined by fuzzy sets on the
universe of discourses X and Y, respectively.
 What is the difference between
classical and
fuzzy
A rules?
classical IF-THEN rule uses binary
logic, for example,
Rule: 1 Rule: 2
IF speed is > 100 IF speed is < 40
THEN stopping_distance is long THEN stopping_distance is short

e variable speed can have any numerical value

ween 0 and 220 km/h, but the linguistic variable
pping_distance can take either value long or sho
other words, classical rules are expressed in the
ck-and-white language of Boolean logic.
 We can also represent the stopping
distance rules in a
fuzzy
Rule: 1 form: Rule: 2
IF speed is fast IF speed is slow
THEN stopping_distance is long THEN stopping_distance is short

uzzy rules, the linguistic variable speed also has

range (the universe of discourse) between 0 and
0 km/h, but this range includes fuzzy sets, such a
w, medium and fast. The universe of discourse o
linguistic variable stopping_distance can be
ween 0 and 300 m and may include such fuzzy
s as short, medium and long.
 Fuzzy rules relate fuzzy sets.
 In a fuzzy system, all rules fire to some extent,
or in other words they fire partially. If the
antecedent is true to some degree of
membership, then the consequent is also true to
that same degree.
 Fuzzy sets of tall and heavy men
Degree of Degree of
Membership Membership
1.0 1.0
Tall men Heavy men
0.8 0.8
0.6 0.6

0.4 0.4
0.2 0.2
0.0 0.0
160 180 190 200 70 80 100 120
Height, cm Weight, kg

These fuzzy sets provide the basis for a weight estimation

model. The model is based on a relationship between a
man’s height and his weight:
IF height is tall
THEN weight is heavy
 value of the output or a truth membership grad
rule consequent can be estimated directly from
responding truth membership grade in the
ecedent. This form of fuzzy inference uses a
thod called monotonic selection.
Degree of Degree of
Membership Membership
1.0 1.0
Tall men
0.8 0.8 Heavy men
0.6 0.6

0.4 0.4

0.2 0.2

0.0 0.0
160 180 190 200 70 80 100 120
Height, cm We ight, kg
A fuzzy rule can have multiple antecedents, for
example:

IF project_duration is long
AND project_staffing is large
AND project_funding is inadequate
THEN risk is high

IF service is excellent
OR food is delicious
THEN tip is generous
The consequent of a fuzzy rule can also include
multiple parts, for instance:

IF temperature is hot
THEN hot_water is reduced;
cold_water is increased
Introduction

 It has been long understood that

 learning is a key element of intelligence.
 This holds both for natural intelligence
 we all get smarter by learning - and artificial
intelligence.
The types of machine
learning
 Handwritten digits are a classic case that is
often used when discussing
 why we use machine learning, and we will make
no exception.
 Below you can see examples of handwritten
images from the very commonly used MNIST
dataset.
 The correct label (what digit the writer was
supposed to write) is shown above each image.
 Note that some of the correct class labels are
questionable:
 see for example the second image from left: is
that really a 7, or actually a 4?

 MNIST - M - Modified, and NIST -

National Institute of Standards and
Technology.
Three types of machine
learning
 The roots of machine learning are
 in statistics, which can also be thought of as the
art of extracting knowledge from data.
 Especially methods such as
 linear regression and Bayesian statistics,
 which are both already more than two centuries
old (!), are even today at the heart of machine
learning.
 The area of machine learning is often divided in
subareas according to the kinds of problems
being attacked.
 A rough categorization is as follows:
Supervised learning

 We are given an input,

 for example
 a photograph with a traffic sign, and
 the task is to predict the correct output or label,
 for example which traffic sign is in the picture
(speed limit, stop sign, etc.).
 In the simplest cases, the answers are in the
form of
 yes/no (we call these binary classification
problems).
Unsupervised learning

 There are
 no labels or correct outputs.
 The task is to discover the structure of the data:
 for example,
 grouping similar items to form clusters, or
 reducing the data to a small number of important
dimensions.
 Data visualization can also be considered
unsupervised learning.
Reinforcement
learning
 Commonly used in situations where
 an AI agent like a self-driving car must operate in
an environment and
 where feedback about good or bad choices is
available with some delay.
 Also used in games where the outcome may be
decided only at the end of the game.
Semisupervised
learning
 The categories are somewhat overlapping and
fuzzy,
 so a particular method can sometimes be hard to
place in one category.
 For example, as the name suggests,
 partly supervised and partly unsupervised.
Classification

 When it comes to machine learning,

 we will focus primarily on supervised learning,
and in particular, classification tasks.
 In classification,
 we observe in input,
 such as a photograph of a traffic sign, and
 try to infer its “class”, such as the type of sign
(speed limit 80 km/h, pedestrian crossing, stop sign,
etc.).
 Other examples of classification tasks include:
 identification of fake Twitter accounts
 (input includes the list of followers, and
 the rate at which they have started following the
account, and
 the class is either fake or real account) and
handwritten digit recognition (input is an image,
class is 0,...,9).