International Journal " Information Theories & Applications" Vol.

15 / 2008


21
THE FUZZY-NEURO CLASSIFIER FOR DECISION SUPPORT
Galina Setlak
Abstract: This paper aims at development of procedures and algorithms for application of artificial intelligence
tools to acquire process and analyze various types of knowledge. The proposed environment integrates
techniques of knowledge and decision process modeling such as neural networks and fuzzy logic-based
reasoning methods. The problem of an identification of complex processes with the use of neuro-fuzzy systems is
solved. The proposed classifier has been successfully applied for building one decision support systems for
solving managerial problem.
Keywords: artificial intelligence, artificial neural networks, fuzzy inference systems, classification, decision
support.
ACM Classification Keywords: I. Computing Methodologies, I.2 Artificial Intelligence
Introduction
A managerial decision support system must harness the information embedded in corporate data and apply this
information to problem-solving processes of managers. Information systems required by the factories of the future
must be capable of managing, maintaining and processing all forms of information required in the factory.
Information is considered as being a vital resource of the enterprise because it represents its mind, because it is
the basis for decision making and communication and it forms the basis for new designs.
The traditional artificial intelligence systems, mainly based on the symbolic paradigm, are showed to be efficient
tools for solving exactly and completely stated problems. However, they were ineffective for solving the real life
problems that are described or represented by the imprecise, incomplete, uncertain and linguistic knowledge or
by large amounts of numerical data collected in databases. The foregoing drawbacks of the symbolic paradigm
based artificial intelligence systems have motivated many researches for creating new tools for designing in-
telligent decision support systems. As a result of those efforts the techniques named "computational intelligence"
have been developed. They have been worked out as a joint of three methodologies: artificial neural networks,
fuzzy logic and genetic algorithms. The artificial neural networks bring in the resulting system the ability for
learning, generalizing and processing large amount of numerical data, the fuzzy logic allows the follow-on
systems to represent and process inexact and uncertain information [Zadeh L.A., Kacprzyk J., 1992], and the
genetic algorithm - as a global optimization tool - is used for strengthening the learning abilities of the resulting
tool [Rutkowska D., M.Pilinski, L. Rutkowski, 1997]. As a result of joining the artificial neural networks (ANN),
fuzzy logic, and genetic algorithms we get the system that is a synergistic combination of the three
complementary technologies [Takagi H., 2000], [Rutkowska D., 2000].
Neural computing, genetic algorithms, and fuzzy systems are effective ways to deal with complex problems
efficiently. Each method handles uncertainty and ambiguity differently, and these technologies can often be
blended to utilize the features of each, achieving impressive results. A combination of artificial neural networks
and fuzzy logic can result in synergy that improves speed, fault tolerance, and adaptiveness. Fusion of neural
networks and fuzzy inference systems have attracted the growing interest of researchers in various scientific and
engineering areas due to the growing need of intelligent decision support systems to solve the real world
problems. There are many real-world applications of intelligent systems integration [Takagi H., 2000], [Li S.,
2000], [F. Wong, 1992], [D.Nauck, F. Klawonn, R.Kruse, 1997]. Each intelligent system can be a valuable
component in a decision support system in which each technology can be used in series or in parallel. For
instance, the neural network can identify classes of membership function for the fuzzy system [Jang R., Sun C.T.,
International Journal " Information Theories & Applications" Vol.15 / 2008


22
Mizutani E., 1997], [Takagi H., 2000].The genetic learning method can perform rule discovery in large databases,
with the rules fed into the conventional expert system [Takagi H., 2000].
There are two directions of researches on systems that are built as a combination of neural networks and fuzzy
logic based systems. The first one gives as results so-called fuzzy neural networks build of fuzzy neurons
[Rutkowska D., M.Pilinski, L.Rutkowski, 1997]. The results of the second research course are neuro-fuzzy
systems that use the artificial neural networks within the fuzzy logic systems framework. The most advanced
types of the neuro-fuzzy systems are hybrid ones [Li S., 2000], [Rutkowska D., 2000].
The paper presents the neuro-fuzzy technologies which can be used for designing the rule-based intelligent
decision support systems. In the paper a connectionist neuro-fuzzy system designed for classification problems is
presented. The proposed classifier has been successfully applied for building one decision support systems for
solving managerial problem. Example of classification problems solved by means of this hybrid intelligent system
is illustrated.
The Neuro-Fuzzy Method for Knowledge Modeling
Neural Networks can be used in constructing fuzzy inference systems in ways other than training. They can also
be used for rule selection, membership function determination and in what we can refer to as hybrid intelligent
systems.
Fuzzy systems that have several inputs suffer from the curse of dimensionality. In this paper we will investigate
and apply the Takagi-Hayashi method [Takagi H., Hayashi I., 1991] for the construction and tuning of fuzzy rules,
which is commonly referred to as neural network driven fuzzy reasoning NDF method (see Fig.1). The NDF
method is an automatic procedure for extracting rules and can greatly reduce the number of rules in a high
dimensional problem, thus making the problem tractable.
The NDF method performs three major functions:
Partitions the decision hyperspace into a number of rules. It performs this with a clustering algorithm.
Identifies a rule's antecedent values (left hand side - LHS membership function). It performs this with a
neural network.
Identifies a rule's consequent values (right hand side - RHS membership function) by using a neural
network with supervised training. This part necessitates the existence of target outputs.

Fig.1. Neural Network Driven Fuzzy Reasoning [Takagi H., Hayashi I., 1991].

The above block diagram represents the NDF method of fuzzy rule extraction. This method uses a variation of the
Sugeno fuzzy rule:
IF x
i
is A
i
AND x
2
is A2 AND ... AND x
n
is A
n
THEN y=f(x
1,
x
2
,, x
n
), (1)
where f(.) is a neural network model rather than a mathematical function. This results in a rule of the form:
IF x
i
is A
i
AND x
2
is A
2
AND ... AND x
n
is A
n
THEN y=NN(x
1
, x
2
,, x
n
). (2)
International Journal " Information Theories & Applications" Vol.15 / 2008


23
The NN
mem
calculates the membership of the input to the LHS membership functions and outputs the membership
values. The other neural networks form the RHS of the rules. The LHS membership values weigh the RHS neural
network outputs through a product function. The altered RHS membership values are aggregated to calculate the
NDF system output. The neural networks are standard feed forward multilayer perceptron designs.
The Neuro-Fuzzy System for Classification
Neural networks are widely used as classifiers; see e.g. [Jang R., Sun C.T., Mizutani E., 1997], [Moon Y.B.,
Divers C.K., and H.-J.Kim, 1998], [Takagi H., 2000]. Classification and clustering problems has been addressed
in many problems and by researchers in many disciplines like statistics, machine learning, and data bases. The
basic algorithms of the classification methods are presented in [D.Nauck, F. Klawonn, R.Kruse, 1997],
[Setlak G., 2004]. The application of the clustering procedure can be classified into one of the following
techniques [Jang R., Sun C.T., Mizutani E., 1997] partition in
which a set is divided into m subsets, when m is the input
parameter:
hierarchical form trees in which the leaves represent par-
ticular objects, and the nodes represent their groups. The
higher level concentrations include the lower level
concentrations. In terms of hierarchical methods,
depending on the technique of creating hierarchy classes
(agglomerative methods and divisive methods);
graph-theoretic clustering,
fuzzy clustering,
methods based on evolutionary methods,
methods based on artificial neural networks.
In this work two approaches have been applied to solving of the
classification and clustering problems. As basic method it was
used Self Organizing Map (SOM) of Kohonen, a class of
unsupervised learning neural networks, to perform direct
clustering of parts families and assembly units. Self Organizing
Maps are unsupervised learning neural networks which were
introduced by T. Kohonen [Kohonen T., 1990] in the early 80s.
This type of neural network is usually a two-dimensional lattice of neurons all of which have a reference model
weight vector. SOM are very well suited to organize and visualize complex data in a two dimensional display, and
by the same effect, to create abstractions or clusters of that data. Therefore neural networks of Kohonen are
frequently used in data exploration applications [Kohonen T., 1990], [Takagi H., 2000]. SOM have been applied to
classification of machine elements in group technology [Setlak G., 2004].
The other approach applies fuzzy logic and fuzzy neural systems for classification problems. However, neural
networks work as a "black box", which means that they produce classification results but do not explain their
performance. Thus, we do not know the rules of classification. Neural network weights have no physical
interpretation. Fuzzy and fuzzy neural systems can be employed in order to solve classification problems [Setlak
G., 2000]. The neural-fuzzy systems are rule-based systems that realize fuzzy IF-THEN rules. Some of the major
works in this area are ANFIS [Jang, 1992], [Jang 1997], NEFCLASS [D.Nauck, F. Klawonn, R.Kruse, 1997],
CANFIS [F. Wong, 1992].
ANFIS (Adaptive Neuro-Fuzzy Inference System) [Jang, 1992], [Jang R., Sun C.T., Mizutani E., 1997] (Fig.1) is
a network-structured adaptive fuzzy inference system which has found various applications including control,
system identification, time series prediction, and noise cancellation. A common version of ANFIS uses normalized
International Journal " Information Theories & Applications" Vol.15 / 2008


24
input fuzzy membership functions, product fuzzification, product inference, sum composition and Sugeno-type
linear output functions (and thus needs no defuzzification). The system parameters are tuned using stochastic
gradient descent method for the premise parameters and recursive least square method for the consequent
parameters.
The CANFIS (Co-Active Neuro-Fuzzy Inference System) model integrates fuzzy inputs with modular neural
network to quickly solve poorly defined problems. Fuzzy inference systems are also valuable as they combine the
explanatory nature of rules (membership functions) with the power of black box neural networks.
A hybrid intelligent system for classification can be presented and is shown in Fig.2. The Neuro-Fuzzy
Classifier (NFC) is a neuro-fuzzy system that has a feed-forward network-like structure. The structure of this
system expresses the fuzzy rules base that models the process of decision-making.
The classifier reflects the fuzzy classification rules, called the rule base, described as follows:
R
(k)
: IF x
1
is

G
1
k
and x
2
is G
2
k
and ... and x
n
is G
n
k
THEN (x C
l )
(5)
where x= [x
1
, x
2
..., x
n
]
T
, and x
i
, for i = 1,2,..., n, are linguistic variables, G
i
k
is fuzzy sets for i-th input and k-th
fuzzy rule, C
l,
for l=1,2,...,m, are classes, N denotes the number of rules R
(k)
, for k = 1,..., N.
The crisp input values, presented in Fig.2, constitute the input vector: ]
x
,...,
x
,
x
[
x
n 2 1
T
=
.
The output values,
k
, for k = 1, 2,..., N, represent degrees of rule activation [6], expressed as follows:
)
x
(
n
1 i
i
k
i
k

=
=

,
(6)
where

=
2
k
i
k
i
i
i
k
i
X
x
x
exp ) (

(7)
is the Gaussian membership function, characterized by the center and width parameters,
x
k
i
and

k
i
,
respectively. The neuro-fuzzy network illustrated in Fig.2 performs a classification task based on the values of
k
,
for k = 1,..., N. Each input vector
]
x
,...,
x
,
x
[
n 2 1
T
x = is classified to the class C
l
(where l = 1,2,,m), which
is associated with the maximal degree of rule activation, that is
{ }

k
k
max .
There are five phases of designing the NFC system:
Each input attribute is described by a number of fuzzy sets;
The initial fuzzy rules base is determined;
System training;
Testing the system against test data;
Pruning the system removing weak, superfluous fuzzy rules in order to improve the systems
transparency.
An example of implementing this neuro-fuzzy classifier is given below.
Example: international stock selection
The presented hybrid neuro-fuzzy system has been applied for building Intelligent Decision Support System
(IDSS). As example of a hybrid neuro-fuzzy system we have chosen a method for deriving a stock portfolio plan.
International Journal " Information Theories & Applications" Vol.15 / 2008


25
An international investment company uses a hybrid neuro-fuzzy system to forecast the expected returns from
stocks, cash, bonds, and other assets to determine the optimal allocation of assets. Because the company
invests in global markets, it is first necessary to determine the creditworthiness of various countries, based on
past and estimated performances of key socio-economic ratios, and then select specific stocks based on
company, industry, and economic data. The final stock portfolio must be adjusted according to the forecast of
foreign exchange rates, interest rates, and so forth, which are handled by a currency exposure analysis. The
IDSS includes the following technologies:
Expert system. The system provides the necessary knowledge for both country and stock selection (rule-
based system).
Neural network. The neural network conducts forecasting based on the data included in the database.
Fuzzy logic. The fuzzy logic component supports the assessment of factors for which there are no reliable
data. For example, the credibility of rules in the rule base is given only as a probability. Therefore, the
conclusion of the rule can be expressed either as a probability or as a fuzzy membership degree.
The rule base feeds into IDSS along with data from the database. IDSS is composed of three modules:
membership function generator, neuro-fuzzy inference system (NFIS), and neural network (NN). The modules are
interconnected, and each performs a different task in the decision process.
Performance of an IDSS has been tested on the following input data:
There are three input nodes (n = 3):
X
1
risk of investment, it is defined by a linguistic term G
i
k
, such as high, medium, low.
X
2
clear profit, also it is defined by a linguistic term G
i
k
, such as high, medium and low.
X
3
period refund of investment, it is defined by a linguistic term G
i
k
, such as long, medium and short.
The output values: there are three C
l
classes, where C
1
is defined by a linguistic term very good
investment, C
2
poor investment and C
3
resign.
The fuzzy set is characterized by a membership function ) (
xi
k
i

: R [0,1]. The membership functions for

the fuzzy set are expressed as (7).
Following the procedure described in section 2, the initial shapes of the fuzzy sets describing the input attributes
were defined and the initial fuzzy rules base, containing 144 rules, was generated. The NFC method was used to
classification tasks and results were compared with traditional agglomerative methods. Results were showed in
Table 1.

Table 1. Results of the classification problem obtained using NFC and agglomerative methods

N Price Advertising Volume
of sales
Stimulus
of sale
Neuro-fuzzy
Classifier
Agglomerative
methods
1 385,65 12000 227180 10000 C1 C1
2 397,24 10000 235090 6000 C1 C1
3 452,20 10000 217340 10000 C4 C4
4 478,92 12000 261280 8000 C C
5 493,10 10000 184380 5000 C2,C3 C3
6 526,35 8000 147180 4000 C2 C2
7 583,24 5000 149300 3000 C2,C C2
8 594,93 5000 156520 4000 C1 C1
9 620,70 5000 121280 2000 C2,C3 C2
10 634,56 10000 116530 0 C3,C4 C4
11 663,20 2000 102160 0 C2,C C
12 672,35 0 112510 0 C1,C2,C C
International Journal " Information Theories & Applications" Vol.15 / 2008


26
Conclusions
The fuzzy neural network, used in this paper for classification, has the following features:
each neuron represents one fuzzy IF-THEN rule,
the number of neurons equals to the number of rules in the rule base,
weights of the neurons have an interpretation concerning parameters of the membership functions of the
corresponding neuro-fuzzy system.
Thus, in contrast to classical neural networks, this network does not work as a "black box", it is a rule-based
neural network.
In the paper we have applied basic soft techniques for extracting rules and classification in a high dimensional
managerial problem. The hybrid neuro-fuzzy system briefly presented in the paper was successfully applied for
designing intelligent decision support system.
By using several advanced technologies (combination of fuzzy logic and neural networks) it is possible to handle
a broader range of information and solve more complex problems. The research conducted proves that fuzzy
neural networks are a very effective and useful instrument of implementation of intelligent decision support
systems in management.
Bibliography
[Jang, 1992] Jang R.: Neuro-Fuzzy Modeling: Architectures, Analyses and Applications, PhD Thesis, University of California,
Berkeley, 1992.
[Jang R., Sun C.T., Mizutani E., 1997] Jang S.R., Sun C.T., Mizutani E.: Neurofuzzy and Soft Computing, Prentice-Hall,
Upper Saddle River 1997, p. 245.
[Kohonen T., 1990] Kohonen T.: Self-organizing Maps, Proc. IEEE, 1990, 78, NR.9, pp. 1464-1480.
[Li S., 2000] Li S.: The Development of a Hybrid Intelligent System for Developing Marketing Strategy, Decision Support
Systems, 2000, Vol 27, N4,
[Moon Y.B., Divers C.K., and H.-J. Kim, 1998] Moon Y.B., Divers C.K., and H.-J. Kim: AEWS: An Integrated Knowledge-
based System with Neural Network for Reliability Prediction // Computers in Industry, 1998, Vol.35, N2, pp.312-344.
[D.Nauck, F. Klawonn, R.Kruse, 1997] D.Nauck, F. Klawonn, R.Kruse: Foundations of Neuro-Fuzzy Systems, J.Wiley&Sons,
Chichester, 1997.
[Rutkowska D., 2000] Rutkowska D.: Implication-based neuro-fuzzy architectures - Applied mathematics and computer
science, V.10, N4, 2000, Technical Unieversity Press, Zielona Gora, 675-701.
[Rutkowska D., M.Pilinski, L. Rutkowski, 1997] Rutkowska D., M.Pilinski, L. Rutkowski: Sieci neuronowe, algorytmy
genetyczne i systemy rozmyte, PWN, Warszawa, 1997 r., pp.411.
[Setlak G., 2004] Setlak G.: Intelligent Decision Support System, // LOGOS, Kiev, 2004, (in Rus.), pp. 250.
[Setlak G., 2000] Setlak G.: Neural networks in intelligent decision support systems for management // Journal of Automation
and Information Sciences, Kiev, N1, 2000r., pp. 112-119.
[Takagi H., Hayashi I., 1991] Takagi H., Hayashi I.: NN-Driven Fuzzy Reasoning, //International Journal of Approximate
Reasoning, 1991, Vol.3, p.1376-1389.
[Takagi H., 2000] Takagi H. Fusion technology of neural networks and fuzzy systems //International Journal of Applied
mathematics and computer science, Zielona Gora, 2000, Vol.10, 4, pp.647-675.
[F. Wong, 1992] F. Wong: "Neural Networks, Genetic Algorithms, and Fuzzy Logic for Forecasting," Proceedings,
International Conference on Advanced Trading Technologies, New York, July 1992, pp.504-532.
[Zadeh L.A., Kacprzyk J., 1992] Zadeh L.A., Kacprzyk J.(ed.): Fuzzy logic for the Management of Uncertainty, Wiley, New
York, 1992, p. 492.
Author's Information
Galina Setlak Ph.D., D.Sc, Eng., Associate Professor, Rzeszow University of Technology, Department of
Computer Science, W. Pola 2 Rzeszow 35-959, Poland, Phone: (48-17)- 86-51-433, e-mail: gsetlak@prz.edu.pl