BACK PROPAGATION NEURAL NETWORKS
BACKPROPAGATION
This allow you to perform a “backward pass”, which helps tune the weights of the inputs.
Backpropagation performs iterative backward passes which attempt to minimize the “loss”,
or the difference between the known correct prediction and the actual model prediction.
With each backward pass, the weights move towards an optimum that minimizes the loss
function and results in the most accurate prediction
Back Propagation Neural (BPN) is a multilayer neural network consisting of the input layer,
at least one hidden layer and output layer.
Back propagating takes place in the network by calculating the error at the output layer and
comparing the target output and the actual output which will be propagated back towards
the input layer
Neural network or deep learning model training occurs in six stages:
Initialization: initial weights are applied to all the neurons.
Forward propagation: the inputs from a training set are passed through the neural network
and an output is computed.
Error function: because we are working with a training set, the correct output is known. An
error function is defined, which captures the delta between the correct output and the
actual output of the model, given the current model weights (in other words, “how far off” is
the model from the correct result).
Backpropagation: the objective of backpropagation is to change the weights for the
neurons, in order to bring the error function to a minimum.
Weight update: weights are changed to the optimal values according to the results of the
backpropagation algorithm.
Iterate until convergence: because the weights are updated, a small delta step at a time,
several iterations are required in order for the network to learn. After each iteration, the
gradient descent force updates the weights towards less and less global loss function. The
amount of iterations needed to converge depends on the learning rate, the network meta-
parameters, and the optimization method used.
KOHONEN SELF-ORGANIZING MAPS
Self-organizing neural networks are used to cluster input patterns into groups of similar
patterns.
They're called "maps" because they assume a topological structure among their cluster
units; effectively mapping weights to input data.
The Kohonen network is probably the best example, because it's simple, yet introduces the
concepts of self-organization and unsupervised learning easily.
Each weight is representative of a certain input. Input patterns are shown to all neurons
simultaneously
In SOM unsupervised learning, the weights of a neural network is modified without
specifying the desired output for any input patterns
The advantage is that it allows the network to find its own solution, making it more efficient
with pattern association
The disadvantage is that other programs or users have to figure out how to interpret the
output.
The structure of a self-organizing map involves m cluster units, arranged in either a one- or
two-dimensional array, with vectors of n input signals.
The weight vectors define each cluster.
Input patterns are compared to each cluster, and associated with the cluster it best matches.
The comparison is usually based on the square of the minimum Euclidean distance.
When a best match is found, the associated cluster gets its weights and its neighboring units
updated.
Weight vectors are arranged into lines or various grid structures.
Some neighborhoods closer to the ends or edges will have fewer weights, because the
algorithm doesn't wrap around.
INFERENCE SYSTEM
Fuzzy Inference System is the key unit of a fuzzy logic system having decision making as its
primary work.
It uses the “IF…THEN” rules along with connectors “OR” or “AND” for drawing essential
decision rules.
CHARACTERISTICS OF FUZZY INFERENCE SYSTEM
The following are some characteristics of FIS −
The output from FIS is always a fuzzy set irrespective of its input which can be fuzzy
or crisp.
It is necessary to have fuzzy output when it is used as a controller.
A defuzzification unit would be there with FIS to convert fuzzy variables into crisp
variables.
FUNCTIONAL BLOCKS OF FIS
The following five functional blocks will help you understand the construction of FIS −
Rule Base − It contains fuzzy IF-THEN rules.
Database − It defines the membership functions of fuzzy sets used in fuzzy rules.
Decision-making Unit − It performs operation on rules.
Fuzzification Interface Unit − It converts the crisp quantities into fuzzy quantities.
Defuzzification Interface Unit − It converts the fuzzy quantities into crisp quantities.
The block diagram of fuzzy interference system is presented below
Diagram present
WORKING OF FIS
The working of the FIS consists of the following steps −
A fuzzification unit supports the application of numerous fuzzification methods, and
converts the crisp input into fuzzy input.
A knowledge base - collection of rule base and database is formed upon the
conversion of crisp input into fuzzy input.
The defuzzification unit fuzzy input is finally converted into crisp output.
FUZZY QUERY SYSTEM
A fuzzy query system is an interface to users to get information from the database using
(quasi) natural language sentences.
Many fuzzy query implementations have been proposed, resulting in slightly different
languages.
Although there are some variations according to the particularities of different
implementations, the answer to a fuzzy query sentence is generally a list of records, ranked
by the degree of matching
QUANTIFICATION
In modelling natural language statements, quantified statements play an important role.
It means that NL heavily depends on quantifying construction which often includes fuzzy
concepts like “almost all”, “many”, etc.
The following are a few examples of quantifying propositions −
Every student passed the exam.
Every sport car is expensive.
Many students passed the exam.
Many sports cars are expensive.
In the above examples, the quantifiers “Every” and “Many” are applied to the crisp
restrictions “students” as well as crisp scope “(person who) passed the exam” and “cars” as
well as crisp scope” sports”.
Fuzzy logic is applied with great success in various control application.
Almost all the consumer products have fuzzy control. Some of the examples include
controlling your room temperature with the help of air-conditioner, anti-braking system
used in vehicles, control on traffic lights, washing machines, large economic systems, etc
WHY USE FUZZY LOGIC IN CONTROL SYSTEMS
A control system is an arrangement of physical components designed to alter another
physical system so that this system exhibits certain desired characteristics. The following are
some reasons of using Fuzzy Logic in Control Systems −
While applying traditional control, one needs to know about the model and the
objective function formulated in precise terms. This makes it very difficult to apply in
many cases.
By applying fuzzy logic for control we can utilize the human expertise and experience
for designing a controller.
The fuzzy control rules, basically the IF-THEN rules, can be best utilized in designing a
controller
ASSUMPTIONS IN FUZZY LOGIC CONTROL (FLC) DESIGN
While designing fuzzy control system, the following six basic assumptions should be made −
The plant is observable and controllable − It must be assumed that the input,
output as well as state variables are available for observation and controlling
purpose.
Existence of a knowledge body − It must be assumed that there exist a knowledge
body having linguistic rules and a set of input-output data set from which rules can
be extracted.
Existence of solution − It must be assumed that there exists a solution.
‘Good enough’ solution is enough − The control engineering must look for ‘good
enough’ solution rather than an optimum one.
Range of precision − Fuzzy logic controller must be designed within an acceptable
range of precision.
Issues regarding stability and optimality − The issues of stability and optimality
must be open in designing Fuzzy logic controller rather than addressed explicitly
MAJOR COMPONENTS OF FLC
The Following are the major components of the FLC as shown in the above figure −
Fuzzifier − The role of fuzzifier is to convert the crisp input values into fuzzy values.
Fuzzy Knowledge Base − It stores the knowledge about all the input-output fuzzy
relationships. It also has the membership function which defines the input variables
to the fuzzy rule base and the output variables to the plant under control.
Fuzzy Rule Base − It stores the knowledge about the operation of the process of
domain.
Inference Engine − It acts as a kernel of any FLC. Basically it simulates human
decisions by performing approximate reasoning.
Defuzzifier − The role of defuzzifier is to convert the fuzzy values into crisp values
getting from fuzzy inference engine.
STEPS IN DESIGNING FLC
The following are the steps involved in designing FLC −
Identification of variables − Here, the input, output and state variables must be
identified of the plant which is under consideration.
Fuzzy subset configuration − The universe of information is divided into number of
fuzzy subsets and each subset is assigned a linguistic label. Always make sure that
these fuzzy subsets include all the elements of universe.
Obtaining membership function − Now obtain the membership function for each
fuzzy subset that we get in the above step.
Fuzzy rule base configuration − Now formulate the fuzzy rule base by assigning
relationship between fuzzy input and output.
Fuzzification − The fuzzification process is initiated in this step.
Combining fuzzy outputs − By applying fuzzy approximate reasoning, locate the
fuzzy output and merge them.
Defuzzification − Finally, initiate defuzzification process to form a crisp output.
ADVANTAGES OF FUZZY LOGIC CONTROL
Let us now discuss the advantages of Fuzzy Logic Control.
Cheaper − Developing a FLC is comparatively cheaper than developing model based
or other controller in terms of performance.
Robust − FLCs are more robust than PID controllers because of their capability to
cover a huge range of operating conditions.
Customizable − FLCs are customizable.
Emulate human deductive thinking − Basically FLC is designed to emulate human
deductive thinking, the process people use to infer conclusion from what they know.
Reliability − FLC is more reliable than conventional control system.
Efficiency − Fuzzy logic provides more efficiency when applied in control system.
DISADVANTAGES OF FUZZY LOGIC CONTROL
We will now discuss what are the disadvantages of Fuzzy Logic Control.
Requires lots of data − FLC needs lots of data to be applied.
Useful in case of moderate historical data − FLC is not useful for programs much
smaller or larger than historical data.
Needs high human expertise − This is one drawback as the accuracy of the system
depends on the knowledge and expertise of human beings.
Needs regular updating of rules − The rules must be updated with time.
ADAPTIVE FUZZY CONTROLLER
Adaptive Fuzzy Controller is designed with some adjustable parameters along with an
embedded mechanism for adjusting them.
Adaptive controller has been used for improving the performance of controller.
Basic Steps for Implementing Adaptive Algorithm
Let us now discuss the basic steps for implementing adaptive algorithm.
Collection of observable data − The observable data is collected to calculate the
performance of controller.
Adjustment of controller parameters − Now with the help of controller
performance, calculation of adjustment of controller parameters would be done.
Improvement in performance of controller − In this step, the controller parameters
are adjusted to improve the performance of controller.
OPERATIONAL CONCEPTS
Design of a controller is based on an assumed mathematical model that resembles a real
system.
The error between actual system and its mathematical representation is calculated and if it
is relatively insignificant then the model is assumed to work effectively.
A threshold constant that sets a boundary for the effectiveness of a controller, also exists.
The control input is fed into both the real system and mathematical model
Here we assume that x(t) is the output of the real system and y(t) is the output of the
mathematical model
Then the error ϵ (t) can be calculated as follows: ϵ(t)=x(t)-y(t)
Here, x desired is the output we want from the system and µ (t) is the output coming from
controller and going to both real as well as mathematical model.
The following diagram shows how the error function is tracked between output of a real
system and Mathematical model −
PARAMETERS FOR SELECTING AN ADAPTIVE FUZZY CONTROLLER
The following parameters need to be considered for selecting an adaptive fuzzy controller −
Can the system be approximated entirely by a fuzzy model?
If a system can be approximated entirely by a fuzzy model, are the parameters of
this fuzzy model readily available or must they be determined online?
If a system cannot be approximated entirely by a fuzzy model, can it be
approximated piecewise by a set of fuzzy model?
If a system can be approximated by a set of fuzzy models, are these models having
the same format with different parameters or are they having different formats?
If a system can be approximated by a set of fuzzy models having the same format,
each with a different set of parameters, are these parameter sets readily available or
must they be determined online?
FUZZINESS IN NEURAL NETWORKS
Artificial neural network (ANN) is a network of efficient computing systems. The central
theme of which is borrowed from the analogy of biological neural networks.
ANNs are also named as “artificial neural systems,” “parallel distributed processing
systems,” “connectionist systems.”
ANN acquires large collection of units that are interconnected in some pattern to allow
communications between units.
These units, also referred to as nodes or neurons, are simple processors which operate in
parallel.
Every neuron is connected with other neuron through a connection link. Each connection
link is associated with a weight having the information about the input signal.
This is the most useful information for neurons to solve a particular problem because the
weight usually inhibits the signal that is being communicated
WHY USE FUZZY LOGIC IN NEURAL NETWORK
The following are some reasons for using fuzzy logic in neural networks
Fuzzy logic is largely used to define the weights, from fuzzy sets, in neural networks.
When crisp values are not possible to apply, then fuzzy values are used
We have already studied that training and learning help neural networks perform
better in unexpected situations. At that time, fuzzy values would be more applicable
than crisp values.
When we use fuzzy logic in neural networks, the values must not be crisp and the
processing can be done in parallel.
FUZZY COGNITIVE MAP
It is a form of fuzziness in neural networks. Basically FCM is like a dynamic state machine
with fuzzy states (not just 1 or 0).
DIFFICULTY IN USING FUZZY LOGIC IN NEURAL NETWORKS
Despite having numerous advantages, there is also some difficulty while using fuzzy logic in
neural networks.
The difficulty is related with membership rules, the need to build fuzzy system, because it is
sometimes complicated to deduce it with the given set of complex data.
NEURAL-TRAINED FUZZY LOGIC
The reverse relationship between neural network and fuzzy logic, i.e., neural network used
to train fuzzy logic is also a good area of study. The following are two major reasons to build
neural-trained fuzzy logic −
New patterns of data can be learned easily with the help of neural networks hence,
it can be used to pre-process data in fuzzy systems.
Neural network, because of its capability to learn new relationship with new input
data, can be used to refine fuzzy rules to create fuzzy adaptive system.
EXAMPLES OF NEURAL-TRAINED FUZZY SYSTEM
Neural-Trained Fuzzy systems are being used in many commercial applications. Let us now
see a few examples where Neural-Trained Fuzzy system is applied −
The Laboratory for International Fuzzy Engineering Research (LIFE) in Yokohama,
Japan has a back-propagation neural network that derives fuzzy rules. This system
has been successfully applied to foreign-exchange trade system with approximately
5000 fuzzy rules.
Ford Motor Company has developed trainable fuzzy systems for automobile idle-
speed control.
NeuFuz, software product of National Semiconductor Corporation, supports the
generation of fuzzy rules with a neural network for control applications.
AEG Corporation of Germany uses neural-trained fuzzy control system for its water –
and energy conserving machine. It is having total of 157 fuzzy rules.
FUZZY LOGIC - APPLICATIONS
AEROSPACE
In aerospace, fuzzy logic is used in the following areas −
Altitude control of spacecraft
Satellite altitude control
Flow and mixture regulation in aircraft de-icing vehicles
AUTOMOTIVE
In automotive, fuzzy logic is used in the following areas −
Trainable fuzzy systems for idle speed control
Shift scheduling method for automatic transmission
Intelligent highway systems
Traffic control
Improving efficiency of automatic transmissions
BUSINESS
In business, fuzzy logic is used in the following areas −
Decision-making support systems
Personnel evaluation in a large company
DEFENCE
In defence, fuzzy logic is used in the following areas −
Underwater target recognition
Automatic target recognition of thermal infrared images
Naval decision support aids
Control of a hypervelocity interceptor
Fuzzy set modelling of NATO decision making
ELECTRONICS
In electronics, fuzzy logic is used in the following areas −
Control of automatic exposure in video cameras
Humidity in a clean room
Air conditioning systems
Washing machine timing
Microwave ovens
Vacuum cleaners
FINANCE
In the finance field, fuzzy logic is used in the following areas −
Banknote transfer control
Fund management
Stock market predictions
INDUSTRIAL SECTOR
In industrial, fuzzy logic is used in following areas −
Cement kiln controls heat exchanger control
Activated sludge wastewater treatment process control
Water purification plant control
Quantitative pattern analysis for industrial quality assurance
Control of constraint satisfaction problems in structural design
Control of water purification plants
MANUFACTURING
In the manufacturing industry, fuzzy logic is used in following areas −
Optimization of cheese production
Optimization of milk production
MARINE
In the marine field, fuzzy logic is used in the following areas −
Autopilot for ships
Optimal route selection
Control of autonomous underwater vehicles
Ship steering
MEDICAL
In the medical field, fuzzy logic is used in the following areas −
Medical diagnostic support system
Control of arterial pressure during anaesthesia
Multivariable control of anaesthesia
Modelling of neuropathological findings in Alzheimer's patients
Radiology diagnoses
Fuzzy inference diagnosis of diabetes and prostate cancer
SECURITIES
In securities, fuzzy logic is used in following areas −
Decision systems for securities trading
Various security appliances
TRANSPORTATION
In transportation, fuzzy logic is used in the following areas −
Automatic underground train operation
Train schedule control
Railway acceleration
Braking and stopping
PATTERN RECOGNITION AND CLASSIFICATION
In Pattern Recognition and Classification, fuzzy logic is used in the following areas −
Fuzzy logic based speech recognition
Fuzzy logic based
Handwriting recognition
Fuzzy logic based facial characteristic analysis
Command analysis
Fuzzy image search
PSYCHOLOGY
In Psychology, fuzzy logic is used in following areas −
Fuzzy logic based analysis of human behaviour
Criminal investigation and prevention based on fuzzy logic reasoning
