1

INTRODUCTION
TO rfVBRID ARTIFICIAL
INTELLIGENCE SVSTEMS

As comptexity rises, precise statements tose meaning and meaningfut

.. statements tose precision.
Lotfi A. Zadeh

1.1 INTRODUCTION

The term "artificial intelligence" (AI), in its broadest sense, encompasses a

number of technologies that includes, but is not limited to, expert systems,
neural networks, genetic algorithms, fuzzy logic systems, cellular automata,
chaotic systems, and anticipatory systems. Interestingly, most of these tech-
nologies have their origins in biological or behavioral phenomena related to
humans or animaIs, and many of these technologies are simple analogs of
human and animal systems. Hybrid intelligent systems generally involve two,
three, or more of these individual AI technologies that are either used in ..
series or integrated in a way to produce advantageous results through
synergistic interactions. In this book we have placed emphasis on neural
networks and fuzzy systems; to a lesser extent, we have aIso placed emphasis
on genetic algorithms where needed for optimization and expert systems
where they are needed to supervise and implement the other three technolo-
gies. A major emphasis in this book will be on the integration of fuzzy and
neural systems in a synergistic way.
In data andy'or information processing, the objective is generally to gain
an understanding of the phenomena involved and to evaluate relevant
parameters quantitatively. This is usually accomplished through "modeling"
of the systems, either experimentally or analytically (using mathematics and
physical principles). Most hybrid systems relate experimental data to systems
or models. Once we have a model of a system, we can carry out various
 THE PROGRESS IN SOFT COMPUTlNG 3
2 INTRODUCTlON TO HYBRID ARTIFICIAL INTELLlGENCE SYSTEMS

procedures (e.g., sensitivity analysis, statistical regression, etc.) to gain a their interrelationships take the form of well-defined if jthen rules. Zadeh's
better understanding of the system. Such experimentally derived models give ingenious observation that the uncritical pursuit of precision may be not only
insight into the nature of the system behavior that can be used to enhance unnecessary but actually a source of error led him to the notion of a fuzzy
mathematical and physical models. set.
There are, however, many situations in which the phenomena involved are Each of these approaches has its own advantages and disadvantages.
very complex and often not well understood and for which first principles Neural networks can represent (i.e., model) complex nonlinear relationships,
models are not possible. Even more often, physical measurements of the and they are very good at classification of phenomena into preselected
pertinent quantities are very difficult and expensive. These difficulties lead us categories used in the training processo On the other hand, the precision of
to explore the use of neural networks and fuzzy logic systems as a way of the outputs is sometimes limited because the variables are effectively treated
obtaining models based on experimental measurements. as analog variables (even When implemented on a digital computer), and
"minimization of least squares errors" does not mean "zero error." Further-
more, the time required for proper training a neural network using one of the
1.2 NEURAL NETWORK5 ANO FUZZY LOGIC 5Y5TEM5 variations of "backpropagation" training can be substantial (some times hours
or days), Perhaps the "Achilles heel" of neural networks is the need for
In the history of science and technology, new developments often come from substantial data that are representative and cover the entire range over which
observations made from a different perspective. Interrelationships that we the different variables are expected to change.
take for granted today may not have been so obvious in earlier decades. For Fuzzy logic systems address the imprecision of the input and output
instance, we regularly gain insight into the behavior of a dynamic system by variables directly by defining then with fuzzy numbers (and fuzzy sets) that
viewing it as being in the "time domain" andjor the "frequency domain." can be expressed in linguistic terms (e.g., cold, warm, and hot). Furthermore,
However, for the first four decades of the twentieth century, statisticians they allow far greater flexibility in formulating system descriptions at the
dealt with autocorrelation and cross-correlation functions (in the time do- appropriate level of detail. Fuzziness has a lot to do with the parsimony and
main) while electrical engineers dealt with power- and cross-spectral densi- hence the accuracy and efficiency of a description. This means that complex
ties (in the frequency domain) without either group realizing that these two process behavior can be described in general terms without precisely defining
concepts were related to each other through Fourier transformations. the complex (usually nonlinear) phenomena involved. Paraphrasing Occam's
Both the statisticians and the electrical engineers have found that analysis Razor, the philosophical principle holding that more parsimonious descrip-
of the fluctuations in process variables provides useful information about the tions are more representative of nature, we may say that fuzzy descriptions
variables as well as the processes involved. These fluctuations, which result in are more parsimonious and hence easier to formulate and modify, more
uncertainties in measured variables, often are caused by some sort of random tractable, and perhaps more tolerant of change and even failure.
driving function (i.e., fluid turbulence, rotational unbalance, etc.). Investiga- Neural network and fuzzy logic technologies are quite different, and each
tion, and the subsequent understanding of these uncertainties (fluctuations), has unique capabilities that are useful in information processing. Yet, they
led to the development of the field of "random noise analysis" which often can be used to accomplish the same results in different ways. For
spawned such analytical specialties as vibration analysis, seismology, electro- instance, they can speed the unraveling and specifying the mathematical
cardiography, oceanography, and so on. relationships among the numerous variables in a complex dynamic processo
Neural networks and fuzzy systems represent two distinct methodologies Both can be used to control nonlinear systems to a degree not possible with
that deal with uncertainty. Uncertainties that are important include both conventional linear control systems. They perform mappings with some
those in the model or description of the systems involved as well as those in degree of imprecision. However, their unique capabilities can also be com-
the variables. These uncertainties usually arise from system complexity (often bined in a synergistic way. It is this combination of the two technologies (as
including nonlinearities; we think of complexity as a property of system well as combinations with other AI technologies) with the goal of gaining the
description -that is, related to the means of computation or language and advantages of both that is the focus of this book.
not merely a system's complicated nature). Neural networks approach the
modeling representation by using precise inputs and outputs which are used
to "train" a generic model which has sufficient degrees of freedom to 1.3 THE PROGRESS IN 50FT COMPUTING
formulate a good approximation of the complex relationship between the
inputs and the outputs. In fuzzy systems, the reverse situation prevails. The Soft computing refers to computational tools whose distinguishing character-
input and output variables are encoded in "fuzzy" representations, while istic is that they provide approximate solutions to approximately formulated
4 INTRODUCTION TO HYBRID ARTIFICIAL INTELLlGENCE SYSTEMS INTELLlGENT MANAGEMENT OF LARGE COMPLEX SYSTEMS 5

prob1ems (Aminzadeh, 1994). Fuzzy logic, neura1 networks, probabilistic The educational, technological, economic, and social impact and signifi-
reasoning, expert systems, and genetic a1gorithms are some of the con- cance of the compute r as a tool for computation and communication have
stituents of soft computing, all having roots in the fie1d of Artificial Intelli- been continuous1y discussed and debated in the last few decades. In the
gence. Whereas the traditiona1 view of computing considers any imprecision . 1970's Ralph Lapp, in an interesting book called The Logarithmic Century,
and uncertainty undesirab1e, in soft computing some to1erance for impreci- captured the ever-changing and accelerating trend in the development of
sion and uncertainty is exploited in order to develop more tractab1e and techno10gy and economics (Lapp, 1973). Yet, he did not foresee the magni-
robust mode1s of systems, at a lower cost and greater economy of com muni- tude of the impact of advanced compute r technology, especially the role that
cation and computation. communications and information processing wou1d have on society. Perhaps
Few of those who attended the historic 1956 Dartmouth Conference to our Japanese colleagues have a better grasp of the issues involved. In a book
discuss "the potentia1 use of computers and simu1ation in every aspect of entitled The Next Century, Halberstam (1991) reported a conversation with a
1earning and any other feature of intelligence" cou1d have envisioned the retired high officia1 of MITI (Ministry for International Trade and Industry)
evo1ution and growth of the embryonic artificial intelligence fie1d and the who in 1987 said " ... the (Japanese) educational system is in danger
impact it has had on our 1ives. It was there that the term "artificial of ... producing young people who have the intellectua1 capacity of comput-
intelligence" was coined, perhaps because of the emphasis on 1earning and ers but who will be inferior to computers in what they can actually do. The
simu1ation. The term "cybernetics" was in vogue at that time with its computers have caught up."
emphasis on potentia1 contro1 of both man and machines. Vacuum-tube-type Of course, the road of technologica1 change is by no means simp1e.
analog computers had reached a state of maturity that they (along with high Eloquent critics such as Nei1 Postman in his evocative book Technopoly
fide1ity stereo sound systerns) were being marketed as "Heathkits," whi1e the strong1y point out the dangers of subordinating cu1ture and society to an
digital "supercomputer" of the time was an IBM-650 with about 2000 words uncritica1 faith in the machine (Postman, 1993). Indeed, computers cannot
of magnetic drum memory storage that operated at about 2 kHz. magically solve our prob1ems. In today's high1y integrated world, however, a
It was in this environment that Frank Rosenblat developed the Perceptron diverse world popu1ation needs the mu1tip1icityof opportunities provided by
by adding a 1earning capability to the McCulloch-Pitts model of the neuron, the new communications and computer technologies, and soft computing is
Marvin Minsky built the first "learning machine" (using 40 processing e1e- promising to become a powerfu1 means for obtaining quick, yet accurate and
ments, each with six vacuum tubes and a motorjclutchjcontro1 systern), and acceptab1e, solutions to many problems. We, the engineers who work to
Bernard Widrow developed the "Ada1ine" (adaptive linear elernent) that provide and app1y these new soft computing too1s, ardent1y hope that they
even today is used in virtually every high-speed modem and telephone will be used for the benefit of mankind.
switching system to cancel out the echo of reflected signals. Boo1ean algebra
was standard procedure, and John McCarthy and John von Neuman were
putting forth the relative merits of symbolic (LISP) and conventional com- 1.4 INTELLlGENT MANAGEMENT OF LARGE COMPLEX SYSTEMS
puter languages. Although there was litt1e in the way of theoretical bases
providing an understanding of these systems, work proceeded on an experi- The real challenge to soft computing is the intelligent management of 1arge
mental basis that was guided primarily by the genius of the individuals comp1ex systems-that is, organizations operating on the scale of the global
involved. economy and resting on an highly globalized information infrastructure. It· is
Today, some 40 years later, the whole world has changed. The computing perhaps the most important activity facing industrial, educational, military,
capacity of that IBM-650 is now encapsulated in a "wristwatch" computer, and governmental organizations throughout the world today. Management
the Perceptron and Adaline processing elements are instantiated in neura1 decisions made today will reverberate throughout these organizations for
network computing and processing methodologies, learning algorithms are years to come. Management decisions made in the past have shaped these
routinely processed on digital computers of all sizes, Boolean logic and organizations and have made them what they are today. In some cases, large
algebra are being replaced by fuzzy logic concepts, LISP is fading away organizations have made the "right decisions" and have been spectacularly
in favor of object-oriented computer languages for artificial intelligence successful. However, it is clear that the decisions of other large organizations
(e.g., C++), the analog compute r has virtually disappeared, and the modern have not been wise. Multi-hundred million and billion dollar losses, followed
persona1 compute r most of us have on our desks may have more than a by layoffs, restructuring, mergers, and, all too often, bankruptcy are common
gigabyte of memory, operate at a processing rate of 200 MHz or more, and as these organizations pay the price for past mistakes. Why did these
be part of a vast global network of computers capable of sharing on-1ine organizations get into trouble or fail? What steps can be taken to ensure that
information in numerica1, textual, visual and audible forms. decisions today are better than those in the past? The answers to these
6 INTRODUCTION TO HYBRID ARTIFICIAL INTELLlGENCE SYSTEMS STRUCTUREOF THIS BOOK 7

questions are as varied as the nature of the organizations. Typical responses systems, many conservative organizations have elected a "minimum step"
given are as follows: incompetent management, too much attention to the approach -that is, make a decision at the last possible moment that involves
next quarterly earoings, lack of vision, fierce new competition, unfair regula- the least amount of (financial or resource) commitment and produces results
tory practices by goveroments, poor design, failure to keep up with the times, at the earliest possible time. However, this can be a strategy for disaster if
antagonism between labor and management, inadequate research and devel- the basis on which the decision is made is not valido AlI toa often, decisions
opment, and so on. The list goes on and on. AlI of these may be valid must be made in the absence of complete data, which gives rise to uncer-
explanations in individual situations, but correcting these alleged problems tainty in the analysis and a higher probability of an erroneous decision. Even
will not guarantee that an organization will be successful in meeting its goals such a "minimum step" approach requires reliable intelligence, accurate analy-
in the future. The successful strategies and methodologies of the 1980s may sis, valid synthesis, intelligent management, and intelligent communications,
not work in the next century. because there is little margin for error. While a modero digital computer
Large complex systems, as a general class, are often virtually out of cannot guarantee the availability of these five attributes, they simply would
control; indeed they are often deemed to be uncontrollable because of their not be available without the modero digital computers and soft computing.
complexity. The reversal of this situation is absolutely essential in a society in Perhaps the single attribute that gives neurofuzzy systems an advantage in
which systems tend to grow without bound because of the perceived benefits addressing the problems of large complex systems is the ability to perform
of "economy of scale." Indeed, organizations tend to grow until they reach a what in mathematical terms would be called many-to-many mappings. Such
level of inefficiency that inhibits and impedes their growth. Only an organiza- mappings are an inherent part of complex systems, because every single input
tion with virtually unlimited resources or power (i.e., goveromental organiza- to a system can influence every single output; i.e., one significant input
tions) can continue to grow under these conditions. The finite resources of change may generate significant changes in many outputs. Most approaches
the world and of individual nations, as well as the growing population that to systems analysis can only deal with one-to-one or many-to-one mappings-
aspires for improved living conditions, demand improved efficiency. It is that is, with the special class of mathematical mappings that we call
absolutely essential for the benefit of mankind, as well as most modero functions, which have been the premier mathematical relation since the
nations that tend to be dominated by large complex systems, that these Newtonian revolution of the Principia. It is now possible and desirable,
systems be brought under intelligent control and management. The advances however, to effectively compute with more complex mathematical mappings
in digital computer technology (both hardware and software) during the past than functions-that is, with many-to-many relations (see Section 5.1). This
decade, along with the associated development of soft computing, appear, for gives us the hope and the expectation that large complex systems can be dealt
the first time in history, to provide a means of implementing intelligent with in a flexible, reliable, and near-optimal manner.
control of complex systems which are so necessary in delivering the fruits of We do not claim that neurofuzzy systems per se can bring about the
industrial technology and commerce to global society. control of large complex systems. It is clear to us that the integration of many
The personal computers or workstations available on the desk of engineers technologies in a yet indisceroible manner is an essential step in the right
and managers today with its soft computing tools has the power of main-frame direction. Neurofuzzy systems represent an integration of fuzzy logic and
computers just a few years ago. They provide the capability of keeping track neural networks that have capabilities beyond either of these technologies
of what is going on in any organization (intelligence), they can provide the individually (Haykin, 1994; Kartalopoulos, 1996). When we further integrate
tools to examine the data in excruciating detail (analysis), they can provide other technologies, perhaps some not yet discovered, in the decades ahead,
models of the behavior of complex systems (synthesis) which then permits we can look forward to tools with sufficient power to tackle problems such as
predictions into the future, at least into the short-term future, and they can intelligent control of large complex systems.
provide recommendations for specific actions (intelligent managernent) that
can be communicated to those who have a need to act in a form that they can
understand (intelligent communications). To the extent that an organization's 1.5 STRUCTURE OF THIS BOOK
management is willing to utilize these tools correctly, significant progress in
solving some of these problems by making the "right" decisions will follow.
This book is divided into four parts: Part I, entitled "Fuzzy Systems: Con-
Unfortunately, making the "right" decision under the circumstance at the
cepts and FundamentaIs," explores the fundamentaIs of fuzzy Iogic systems
time the decision is made does not guarantee success. It may have been the and includes the following chapters:
"right" decision at the time, but the consequences may be unpredictable
because of the time lag between decision and results in a changing environ-
ment. What is needed is a form of anticipatory control as discussed in Chapter 2. Foundations of Fuzzy Approaches
Chapter 15. In the absence of an ability to predict the future behavior of Chapter 3. Fuzzy Relations
8 INTRODUCTION TO HYBRID ARTIFICIAL INTELLlGENCE SYSTEMS REFERENCES 9

Chapter 4. Fuzzy Numbers 1995; Hanselman, 1996) can be used for demonstrations and solving more
Chapter 5. Lmguistic Descriptions and Their Analytical Forms sophisticated problems. Of course, the Professional Version of MATLAB©
Chapter 6. Fuzzy Control can also be used if it is available.
This supplement was written using the MATLAB© Notebook and .
Part lI, entitled "Neural Networks: Concepts and Fundamentais," explores Microsoft WORD .Version 6.0. The Notebook allows MATLAB© commands
the fundamentais of neural networks and includes the following chapters: to be entered and evaluated while in the WORD environment, which allows
the document to both briefly explain the theoretical details and aIso show
Chapter 7. FundamentaIs of Neural Networks MATLAB© implementations. It also allows the user to experiment with
changing the MATLAB© code fragments in order to gain a better under-
Chapter 8. Backpropagation and Related Training Algorithms
standing of their application.
Chapter 9. Competitive, Associative, and Other Special Neural Networks This supplement contains numerous examples that demonstra te the practi-
Chapter 10. Dynamic Systems and Neural Control cal implementation of relevant techniques using MATLAB©. Although
Chapter 11. Practical Aspects of Using Neural Networks MATLAB© toolboxes for Fuzzy Logic and Neural Networks are available,
they are not required to run the examples given. This supplement should be
Part Ill, entitled "Integrated Neural-Fuzzy Technology," explores the joint considered to be a brief introduction to the MATLAB© implementation of
use of neural networks and fuzzy logic systems. It includes the following neural and fuzzy systems, and we and the author strongly recommend the use
chapters: of Neural Networks and Fuzzy Logic Toolboxes for a more in-depth study of
these information-processing technologies. Many of the m-files and examples
Chapter 12. Fuzzy Methods in Neural Networks are extremely general and portable while other examples will have to be
Chapter 13. Neural Methods in Fuzzy Systems altered significantly for use to solve specific problems.
Chapter 14. Selected Hybrid Neurofuzzy Applications The content of the MATLAB© Supplement is coordinated with Fuzzy and
Neural Approaches in Engineering so that students can use it to enhance their
Chapter 15. Dynamic Hybrid Neurofuzzy Systems
knowledge of fuzzy systems, neural networks, and neurofuzzy systems. In-
deed, it is expected that many instructors will choose to use both this book
Part IV, entitled "Other Artificial Intelligence Systems," reviews other artifi-
and the MATLAB© Supplement together in their classes. Practicing engineers
cial intelligence systems that can be used with neural networks and fuzzy
and scientists in industry who want to use this text to learn about neural,
systems. It includes the following chapters:
fuzzy, and neurofuzzy systems will find this supplement to be a valuable aid
in their self-study.
Chapter 16. Expert Systems in Neurofuzzy Systems
Chapter 17. Genetic Algorithms
Chapter 18. Epilogue REFERENCES

Aminzadeh, F., and Jamshidi, M., Soft Computing, Prentice-Hall, Englewood Cliffs,
1.6 MATLAB©l SUPPLEMENT NJ,1994.
Halberstam, D., The Next Century, William Morrow and Company, New York, 1991.
In this text, we have included problems for students at the end of most Hanselman, D., and LittIefield, B., Mastering MATLAB, Prentice-Hall, Englewood
chapters. Generally, these problems are pedagogical in nature and are Cliffs, NJ, 1996.
intended to be simple enough that they can be solved without the aid of Haykin S., Neural Networks: A Comprehensive Foundation, IEEE Computer Society
computer software. To supplement these exercises, we have enlisted our Press, Macmillan, New York, 1994.
colleague, Dr. J. Wesley Hines of the University of Tennessee, to prepare Kartalopoulos, S., Understanding Neural Networks and Fuzzy Logic, IEEE Press,
a MATLAB© Supplement for Neural and Fuzzy Approaches in Engineering, a New York, 1996.
paperback book of approximately 150 pages published by John Wiley and Lapp, R., The Logarithmic Century, Prentice-Hall, Englewood Cliffs, NJ, 1973.
Sons, in which the Student Edition of MATLAB© (The MathWorks Inc., Mathworks, Inc., The Student Edition of MATLAB~ Users Guide, Natick, MA, 1995.
Postman, N., Technopoly, Vintage Books, New York, 1993.
lMATLAB is copyrighted by MathWorks Inc., of Natick, MA.