0% found this document useful (0 votes)
64 views24 pages

Cognitive Software-Defined Networking Using Fuzzy Cognitive Maps

Uploaded by

baburao_kodavati
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
64 views24 pages

Cognitive Software-Defined Networking Using Fuzzy Cognitive Maps

Uploaded by

baburao_kodavati
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 24

This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
1

Cognitive Software-Defined Networking Using


Fuzzy Cognitive Maps
Giovanni Baggio, Riccardo Bassoli Member, IEEE, and Fabrizio Granelli, Senior Member, IEEE

Abstract—Future networks are expected to provide improved


support for several different kinds of applications and services.
All these services will have diverse characteristics and require-
ments to be satisfied. A potential technology to upgrade efficiently
and effectively current generation networks is virtualisation via
network ’softwarization’. This approach requires the combi-
nation of software-defined networking and network function
virtualisation. Nevertheless, such a new complex network struc-
ture will raise further issues and challenges to be solved both
reactively and proactively, without human intervention. In order
to achieve that, academia and industry have identified the solution
in the implementation and deployment of machine learning.
Hence, very likely, 5G (and especially beyond 5G) networks will
be cognitive virtualised networks.
In that context, this article proposes a cognitive software-
defined networking architecture based on Fuzzy Cognitive Maps. Fig. 1. The cognition loop adapted from [6].
First, specific design modifications of Fuzzy Cognitive Maps are
proposed to overcome some well-known issues of this learning
paradigm. Second, the efficient integration with a software- ’softwarization’ provides exactly what is missing in current
defined networking architecture is presented and analysed. Fi-
networks:
nally, the emulation of a sample network scenario via Mininet is
provided to validate the effectiveness and the potential of the • the capability to adapt themselves via software reconfig-
new cognitive system and its capability to act and to adapt uration;
independently of human intervention.
• a higher level of centralisation in the control plane, surely
Index Terms—Cognitive networks, Fuzzy cognitive maps, beneficial to implement more efficiently reasoning and
software-defined networking, network virtualisation. learning processes of a cognitive network.
Nevertheless, the path towards actual deployment of the
I. I NTRODUCTION
potential of cognitive networks is still relatively long, as few
cognitive network is a network which implements ’in-
A telligent’ procedures – inspired by the cognition process
of real people – to learn from current operating conditions
works target the core of the problem: the choice of an effective
decision-making procedure to support efficient data mining,
analysis and inference process [2].
in order to adapt and to make future decisions, targeting an Cognitive networks are based on the so called cognitive loop
end-to-end goal. This concept was firstly proposed [1], [2] in (Figure 1). A complete cognitive process is based on Sense-
2006 and 2008 respectively. The main required characteristic Plan-Decide-Act, which is accompanied by learning and final
[1] to enable the deployment of cognitive networks is the ex- environmental feedback. The Sense state is responsible for
istence of software-adaptable networks (SANs), which include information gathering, monitoring and pre-processing. Next,
software-based network functionalities. the Plan state interprets the collected data according to a
During the last few years, growing research on network
model and use the results for planning. The Decide state
function virtualisation (NFV) [3] and especially on software-
identifies the actions to be performed according to policies
defined networking (SDN) [4], provided a suitable infrastruc-
and processed information, and evaluates eventual alternatives.
ture to pave the way towards an actual implementation of
Finally, the Act state realises the actual action on the network.
cognitive networks. Moreover, the importance of NFV and
The environmental information is used as input to re-start the
SDN grew towards the design and implementation of future
loop and helps in the final check of the effects: it can also
5G cellular networks [5]: in fact, they are seen as the main
send alarms and warnings. All the previous states provide
technological enablers in order to achieve higher data-rate,
a feedback, which helps the learning algorithm to improve
lower latency and adaptivity for their diverse service scenar-
the performance and responsiveness according to the defined
ios. Indeed, in the cognitive networking paradigm, network
policies.
G. Baggio is with FBK CREATE-NET, WiN research unit, Trento, Italy Among the initial works about cognitive networks, some
(e-mail: g.baggio@fbk.eu). significant ones theorised about the best tools a cognitive entity
R. Bassoli and F. Granelli are with the Department of Information En-
gineering and Computer Science, at the University of Trento, Trento, Italy should use to carry out their optimisation tasks ([1], [7]), while
(e-mail: {riccardo.bassoli, fabrizio.granelli}@unitn.it). others, instead, adopted specific reasoning techniques: anyway,

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
2

often they did not clearly justify the reason for the choice knowledge of the network status. In fact, by considering
among those techniques [6]. In the last decade, several publi- the taxonomy of ML-based SDN [13], ML approaches
cations have investigated the deployment of various machine generally employ so called training datasets, i.e. human
learning (ML) algorithms to enhance the limited management supervision.
capabilities of legacy SDN. Nevertheless, in 2011 [8], the • the design of an actual and effective architecture, where
authors of this paper were the first to propose a suitable the proposed enhanced FCMs are integrated into SDN-
reasoning formalism for cognitive networks based on Fuzzy based networks. That is followed by an actual validation
Cognitive Maps (FCMs), which can be used to perform causal of this proposed system on an emulated testbed, therefore
reasoning. The innovative aspect lied in the adoption of such proposing one of the first complete implementations of
a tool to explicitly exploit cause-effect relationships among FCM-based cognitive networking to date.
variables in the protocol stack of network nodes. The choice of The rest of the paper is organised as follows. Subsection
FCMs was based on the idea that variables at different layers of I-A justifies the choice of FCMs for SDN-based cognitive
the protocol stack share cause-effect relationships. Moreover, networks, by showing their advantages in respect of other
FCMs allow the presence of loops, which are present in several ML algorithms. Section II describes the mathematical nota-
aspects of networking (see, for example, closed-loop operation tion and problem statement, as originally proposed in [12].
of flow and congestion control in TCP, automatic rate fallback Furthermore, it briefly presents the Hebbian-based algorithm
in WiFi, etc.). used for learning. Next, Section III fully discusses the novel
Fuzzy cognitive maps [9] have significantly attracted the enhanced model for Hebbian-based FCMs, which improves
attention of the research community in the last fifteen years. the theoretical framework previously developed in [12]. Our
The main limitations of FCMs, which have been identified, are theoretical solutions improve the correctness/performance of
[10]: edges’ weights are linear, they cannot represent logical the algorithm towards efficient deployment in real cognitive
operators between ingoing nodes, they cannot model multi- networks. Section IV deals with the design of a cognitive
meaning environments, they do not include multi-state con- software-defined networking architecture based on our pro-
cepts, they cannot handle more than one relationship between posed FCMs. This section considers architectural analysis
nodes, they are symmetric or monotonic (that does not happen and evaluation of overhead and latency introduced in SDN
for various real causal relations), and their first order dynamics paradigm by our enhanced Hebbian-based FCMs. Finally,
cannot handle randomness associated with complex domains. Section V provides results to verify the effectiveness of our
Learning algorithms for FCMs can be organised into three FCM-based SDN in two exemplar virtualised networks, which
main families [11]: Hebbian-based, population-based and hy- have been implemented in Mininet environment.
brid approaches. The first type uses the basis of experts’
knowledge to lead FCMs to converge into an acceptable
solution. The second employs historical data without any A. Related Works and Motivation
expert’s intervention. The third is based on both experts’ input Regarding unsupervised learning and SDN, the exhaustive
and historical data. The first detailed implemented architecture survey about ML-base SDN [13] identifies two main ap-
for FCM-based cognitive networks presented in [8] and [12] proaches in the literature: k-Means SDN and self-organising
applied a Hebbian-based learning algorithm. These works pro- map (SOM) SDN. The former is a popular method to classify
vided a framework to design FCMs for cognitive networks by unlabelled data, so it make it useful both for classical and QoS-
overcoming the above limitations, which could have negatively aware traffic classification in SDN-based networks. It is mainly
affected their deployment in the context of communications limited to clustering problem solving and it is computationally
networks. However, the authors of [8], [12] have not clearly expensive, in particular for large maps with big amount of
defined a criterion for the FCM-based algorithm to detect when training data. Recently, k-means have also been applied to
the inference process reaches a satisfying solution. Basically, SDN to solve controller placement problem [14], [15], [16],
those works provided evidence of the potential of FCMs for which is an important issue in presence of multiple SDN
cognitive networking, without achieving a general solution in controllers. The latter [17] is also principally employed to
order to predict when the system gets its final status. Still, they solve clustering problems: its data mapping is easier to be
represent a relevant milestone toward an actual architecture understood and capable to handle big datasets, while it is also
able to support cognitive networking. computationally expansive mainly for large training datasets.
The contribution of this article includes: Side by side, reinforcement learning (RL) [18] and deep
• the definition of an enhanced theoretical model for FCMs reinforcement learning (DRL) [19], [20] are additional ML
applied to wired networks. This model capitalises on pre- paradigms, in which an ’unsupervised agent’ interacts with its
vious results in [8], [12] to improve FCMs towards their environment to learn the best action to perform in order to
effective deployment in cognitive networks. In particular, maximise its long-term reward.
the article focuses on some critical aspects affecting clas- Moreover, the taxonomy of works about ML applied to
sical FCMs’ inference process, which – according to the SDN reveals that ML algorithms are very often limited to
best of authors’ knowledge – have not been investigated specific aspects of SDN network management such as traffic
yet in the scientific literature. classification, Quality-of-Service (QoS), routing optimisation,
• the theoretical and practical proof that our enhanced resource management or security: there has been no approach
Hebbian-based FCM can work effectively without a-priori yet, which has tried to use unsupervised ML/RL to realise a

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
3

sort of SDN-based autonomic network (i.e. an SDN network convergence of belief propagation algorithms [12]. On the
that manages itself without human intervention). other hand, the FCM inference procedure is also effective to
Regarding routing optimisation, the authors of [21] designed solve problems involving no loops. In particular, inference is
a centralised Cognitive Routing Engine (CRE), based on guaranteed to converge to either a fixed point or a limit cycle,
Random Neural Networks with Reinforcement Learning, was provided that concepts take their values in any finite discrete
to find the optimal network paths. In 2017, [22] applied RL set [34].
to improve the performance of routing protocols in SDN. Next, when neural networks are employed to model dy-
The same year, [23] developed a DRL agent to optimise namic systems, the obtained solution does not necessarily
routing towards a decrease in network latency. Next, [24] reflect the actual relationships among its system variables [35].
designed DRL paradigm for routing optimisation in SDN- On the other hand, the edges of an FCM faithfully represent
based environments. Reference [25] proposed solution based those relations and are, therefore, more appropriate to analyse
on k-means to optimise routing protocol in SDN. both direct and indirect cross-layer interactions [33], [12].
In the context of resource management, article [26] designed Further advantage is the possibility of exchanging/merging
a content delivery framework based on RL and SDN to select multiple FCMs, resembling operations people do when they
the optimal protocol. Next, [27] deployed RL to minimise the exchange their opinions [12]. This aspect has its roots in
long-term reconfiguration cost of roadside unit cloud network the primary purpose for which FCMs were created, i.e. to
based on SDN. In 2017, [28] applied RL and game theory for allow experts to represent their causal knowledge about some
activation of servers in mobile edge computing based on SDN. situation. Different people may have different opinions about
In parallel, [29] applied RL and game theory for resource the same matter, and may encode differently their beliefs,
allocation and management in distributed environment based hence drawing conflicting FCMs. Merging helps to smooth
on SDN. Recently, [30] applied DRL for multimedia traffic (possibly divergent) beliefs and biases, thereby reducing the
control in SDN in order to achieve high quality of experience. possibility of biased reasoning. Moreover, weights can be
Given the above context, this article is the first one to design employed to give more or less credit to each FCM. Finally,
in detail an FCM-based SDN system for unsupervised virtual the ’composed FCM’ can contain potentially non-overlapping
network management while providing a quantitative study of FCMs, thus, enabling the exchange of knowledge in case the
its performances. Another pivotal characteristic of this work domain of knowledge of cognitive entities is different, which
is the scope of ML paradigm: the aim of our FCM-based undoubtedly is an advantageous feature when dealing with
controller is not to optimise single aspects of networking but to uncertain scenarios such as communication networks.
guarantee efficient and effective self management of the SDN- The main drawback of legacy FCMs is the automatic
based network. Thus, the cognitive controller has to control synthesis of the maps: FCMs were not originally designed for
and to manage various aspects of networking to guarantee being constructed starting from observational data but they
good routing management while enhancing QoS provisioning were initially devised in the social science field as a tool
via effective management of resources. to help experts to express their beliefs about a given matter
The choice of FCMs comes from a major aspect in the area [33]. For this reason, self-synthesis of FCMs is hard, mostly
of unsupervised autonomic networks: human monitoring. In because, for non-humans, cause/effect relationships between
fact, humans have to be able to monitor and to understand variables are generally more complex to detect than simple
easily how the cognitive controller thinks and acts: that to correlations [33].
prevent unwanted bad events. One of the main advantages of Finally, the results obtained by our designed and imple-
FCM is exactly the clear and easy-to-understand representation mented FCM-based SDN keep comparable delays with legacy
of knowledge [9], which does not happen with solutions SDN Open Network Operating System (ONOS), thus the
based on k-Means, SOMs and RL. This is important since reasoning and acting procedures are performed almost trans-
in unsupervised systems humans have to be able to control parently. Moreover, our proposed cognitive SDN system can
how the unsupervised system thinks and make decisions. achieve 100% accuracy in average thus overcoming all the
Furthermore, while our design of FCMs is mainly focused legacy ML-based SDN solutions [13].
on resource/routing management and QoS optimisation (in this
article), the FCM-based controller can easily be employed to II. P RELIMINARIES AND BACKGROUND
consider and to manage other aspects of networking by adding
further variables to the FCM (e.g. variables referred to actions A. Software-defined Networking
and quality). Software-defined networking is a virtualisation paradigm,
Fuzzy Cognitive Maps have more advantages than other which permits a software-based control of data-paths and
reasoning techniques such as neural networks, Bayesian and routing strategies of networks. This technology aims at de-
Markov networks. Unlike both Bayesian and Markov net- taching control and data plane of communication networks.
works, the inference procedure of FCMs is less complex since The central entity of SDN virtual architecture is the SDN
it only involves vector-by-matrix multiplications and thresh- controller, which updates flow tables and policies at so called
olding operations [31]. Moreover, FCMs are more effective SDN switches. Moreover, an application layer is responsible
to represent causality loops in a problem; for these kind of to handle different networking applications such as control of
problems Bayesian networks cannot be applied [32], [33]. In data paths, user authentication and management of mobility.
Markov networks, the presence of loops cannot guarantee the Among various protocols proposed for controller-switches

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
4

Fig. 2. Structure of SDN virtual network (left side) and structure of SDN
architecture (left side), which specifies the logic interfaces of the architecture.

communications, the standard protocol currently in use is


Openflow: its aim is to change flow tables of Openflow SDN
switches, which contains rules, actions and policies related to
data traffic. Next, SDN switches collect statistics to be sent to
SDN controller for basic network control.
Figure 2 depicts SDN network structure (left side), which
has previously been described. The same figure (right side)
shows the logic architecture of SDN and its principal inter-
faces. In particular, the controllers (if multiple are allowed)
communicate each other via the eastbound/westbound inter-
face. Next, the northbound interface is the one that translates Fig. 3. Example of FCM structure and its respective matricial representation.
the different third-party application requirements into network
commands. Finally, the southbound interface allows the con-
trollers to set up the flow tables of SDN switches via the
Openflow protocol to allow the creation of virtual networks
according to the needs of different services.

B. Introduction to Fuzzy Cognitive Maps


The ’cognition loop’, graphically represented in Figure 1,
is central to any cognitive architecture. Cognitive networking
undoubtedly needs a cross-layer approach to operate, which
represents the means to provide optimisation, while the cog-
nitive engine deals with learning, adaptation and decision
process to achieve end-to-end goals [12].
FCMs are mathematical structures proposed in 1986 [9] as
a means for modelling dynamic systems through the causal
relationships, which characterise them. Figure 3 shows a Fig. 4. Example of how the inference process of an FCM works. Thresholding
considers the threshold value 0.5.
simple example of graphical and matricial representation of
an FCM. The former is a directed graph, in which a node
represents a generic concept and an edge is the causal relation cannot cause themselves. Thus, the diagonal of the adjacency
between two connected concepts: especially, the cause is the matrix F of an FCM only has zeros.
starting node of the directed edge. The state of an FCM-based system with n distinct concepts
The domains of nodes and the weights of edges can be either is a vector of dimensions 1 × n. Then, the inference process
discrete or continuous sets. The latter is used in more complex consists in various multiplications of this vector by the FCM
FCMs, where a high level of detail needs to be achieved. For matrix. Next, the result is thresholded each time, until it
example, by extending a continuous set from [0, 1] to [−1, 1], converges either to a fixed point or to a limit cycle. Figure
it generally results in a greater flexibility of the model. The 4 depicts a simple example of this inference procedure.
use of simpler ranges of the same model is normally suited to
obtain a preliminary representation of a problem. If a concept
is zero, it means the concept is ’off’, ’inactive’, in a ’low-state’ C. Fuzzy Cognitive Maps for Cognitive Networks
or is completely neglected. On the other hand, a concept set FCMs have been proposed to implement reasoning and
to one means it is ’high’ or ’active’. Edge weights measure learning in cognitive networks [12]. In order to target an end-
the degree of causality; values such as +1 or −1 denote a to-end goal, the FCM should consider parameters associated
strong causal relationship, either positive or negative. An edge to different layers, and the FCM concepts should be mapped
labelled with zero means that two concepts are not causally to communication protocol internals. As proposed in [12], the
related. Clearly, the graph cannot have loops since concepts concepts can be grouped into three main categories according

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
5

to their relationship with Quality-of-Service (QoS) metrics, Moreover, let fÛi,t j label the variation of the edge’s value,
to environmental characteristics in which the cognitive entity connecting concept Ci with concept C j at time instant t. Then,
is, and to the set of actions that a protocol can perform. the differential Hebbian law states that
Mathematically, these concepts respectively form the triple of
fÛi,t j = − fi,t−1
j + Ci C j
Ût Ût (1)
vectors (q, e, a), which represents the system state vector s.
The objective of the FCM is the convergence to a solution where the derivatives represent changes in two generic con-
state s∗ = (q, e, a∗ ), achieved by finding a vector a∗ such that cepts and the result of their product reveals the correlation
the constraints expressed by q are satisfied before environmen- among them.
tal conditions e change. Since vectors q and e are known, the Expression (1) shows that a variation of concept Ci occurs
search space of the algorithm is limited to the elements of a: before the variation of concept C j , the other way around if
that avoids the problem to be NP-hard. C j was the first concept changing then the edge to be updated
However, in order to map the concepts in the most effective would be f j,i . The negative value of the edge at time t − 1 is
way, it is fundamental to choose the right domain. Concepts included in the right-hand side of equation (1) to prevent that
can be expressed either by discrete sets or continuous sets. In a spurious simultaneous variation impacts indefinitely.
particular, it is normally better to avoid continuous sets be- Next, the edge at step t is computed as the value it had at
cause they can results into chaotic behaviour [12]. Continuous time step t − 1 plus the variation:
ranges of values can be converted into discrete sets by ap-
plying a thresholding procedure. However, finding the optimal j + fi, j = Ci C j
fi,t j = fi,t−1 Ût Ût Ût (2)
threshold for an application may not be straightforward. Thanks to the term − fi,t−1
j in expression (1), the value on the
On the other hand, the size of a discrete set can be chosen edge can be set to zero when one of the concepts do not
according to the needs. For example, concepts that convey change, thereby indicating that the previous causal relationship
an ON-OFF relationship can use the set {0, 1}, and the ones no longer holds.
requiring to invert causal relationship can use the set {−1, 1}. As a result, the learning process does not update the
After defining the concepts and their domains, it is impor- knowledge considering only the current causality relationships
tant to select a learning algorithm for FCMs. The follow- degree but also keeping in consideration the past value.
ing subsection provides some details on one of the existing Depending on the application, such update may be preferred
learning algorithms. That should be considered as a concrete to be more responsive or vice versa, the responsiveness of
example of how learning can be implemented, but other the algorithm can be modified by introducing a parameter
methods are available and can be used without architectural η in the previous formula known either as learning rate or
differences with respect to the solution presented in this paper. as decreasing learning coefficient. In other words, it can be
seen as a smoothing parameter, capable to avoid significant
D. Differential Hebbian Learning Algorithm oscillation in the values of the edges (the amplitude of these
variations depends on the magnitude of the coefficient value).
In our work we borrow the problem statement proposed
Moreover, parameter η is generally defined in the set (0, 1]:
in [12]: hence, we apply a Hebbian-based algorithm called
values close to zero result in a slowly changing FCM, while
differential Hebbian learning (DHL) algorithm [36]. The main
values close to the unity produce a highly responsive FCM.
drawback of this method is that the formula updates weights
Finally, the learning formula becomes
between each pair of concepts without considering the influ-
ence of other concepts. j + η(− fi, j + Ci C j )
fi,t j = fi,t−1 t−1 Ût Ût (3)
In order to explain how DHL algorithm works, let’s consider
a simple FCM with two concepts named A and B. If both A It is worth notice that if η is set to one meaning that the past
and B change their values from 0 to 1, then it can be inferred value on the edge is not considered when it is updated, then
that the positive variation of one concept has caused a positive edges can assume values in the discrete set (−1, 0, 1) while a
variation of the other. Similarly, if both A and B change their different η results in continuous range of values.
values from 1 to 0, it can be inferred that the negative variation
of A has produced a negative variation of B. In both cases the III. E NHANCED F UZZY C OGNITIVE M APS FOR C OGNITIVE
sign of the variation is positive for these two concepts because N ETWORKS
a positive causal relationship is present. The work in [12] proposed FCMs based on discrete domains
Let’s now suppose that A changes its value to 1 and B, instead of continuous ones and introduced thresholding in
which before the variation of A had the value set to 1, goes to form of pre-processing in order to map values into discrete
0. In this case, there is a negative causality relationship because domains. This section demonstrates the potential of using
a variation of the first concept led to the opposite variation of continuous ranges of values, by avoiding thresholding. In
the second, thus the edge connecting the two concepts is now particular, we show the significant advantages obtained by
negative (for simplicity let’s assume the system is memoryless, interpreting continuous values in the range [−1, 1] as ’directed’
meaning that the algorithm does not consider the previous (signed) probabilities.
states). Moreover, we also show it is actually possible to acquire
Formally, let Ci and C j denote two generic concepts, and let knowledge not only related to the causality degrees among
CÛi and CÛ j denote their variations over time (time derivatives). nodes but also to create, modify and delete persisting causal

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
6

relationships among them. In other words, the system is able


to start from an empty matrix, which means that no causal
relationships have been inserted a-priori (i.e. no knowledge
is provided to the system), and to build the edges during run
time by analysing the environmental cause-effect relationships
(i.e. the system can learn during run time). That would allow
a relevant capability ’to learn’ in an autonomous manner and,
in perspective, leading to a significant reduction in human
intervention in managing the network.

A. Finding the Desired State (a)


First of all, it is necessary to identify an effective way
to detect either when the inference process has reached a
satisfying solution or when the process should continue. In
[12], this aspect was not completely investigated. Thus, this
subsection explains in detail how the inference process works
and the meaning of each of the steps performed during its
operation.
Figure 5 shows an example of an extremely simple FCM
and, on the right, the log of the inference process is reported.
A mathematical way to describe the connections between
concepts is the use of adjacency matrix since an FCM can
be represented as a directed graph. The example in Figure 5
has two concepts so it produces a 2 × 2 adjacency matrix.
The only connection between those concepts is the directed
line from Concept 0 to Concept 1. Then, the corresponding
adjacency matrix becomes (b)
!
0 1 Fig. 5. (a) Simple FCM with positive causality (b) Description of FCM
(4) inference process. The end of the inference process happens when there is a
0 0 repetition of two identical vectors at the end of each trial.

Since there are no loops, the algorithm ends with a zero-


value array, which means that no further step has to be taken: B. Belief Propagation for Fuzzy Cognitive Maps
in fact, the knowledge involved in the variables and their The inference process implementation of [12] suggests that
relationships have already been exploited. In order to simplify at each step the output of the vector-by-matrix multiplications
the explanation, all of the examples reported are based on the has to be rounded with a threshold, which maps negative
assumption that quality concepts are considered in a bad shape values to zero and positive values to one. There are two main
or alert state when their value is high. The example of Figure problems related to this approach.
5 shows a positive causal relationship from an action to a First, it is not possible to propagate negative causal relation-
quality concept. Once the latter enters in an undesired state, ships because zero-valued relations stop the inference process.
the inference process is triggered and the consequences for the The solution of this issue is to avoid thresholding. Figure 6
two possible states of the action variable are investigated. In depicts an example where there is a negative causal relation-
order to keep this example simple, the causality relationships ship from the action to the quality concept. The corresponding
are supposed to be deterministic, thus we work with integer adjacency matrix is !
numbers: however, the same procedure would have also been 0 −1
applied in the continuous case. (5)
0 0
By looking at the log (Figure 5, on the right), the repetition
of two identical vectors at the end of each trial are produced Since negative values in the output of the vector-by-matrix
by the inference process in order to automatically understand multiplication are brought to zero it is unavoidable that causal
when no further steps have to be performed, i.e. when the relationships are interrupted.
knowledge of the FCM has already been fully exploited for Second, precious information stored in the continuous set
making a decision. The process finishes with the conclusion [−1, 1] is lost. According to the formula presented in the
that Concept 0 has to be set to a low value because this choice learning part, the edges can acquire values in the continuous
– according to the information embedded in the FCM – will set [−1, 1], which can be treated as signed probabilities: in
move the system in a safer condition, so a low state for the particular, the sign on these values means that there is a
quality concept. certain probability (the absolute value of the edge) that either

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
7

(a) (a)

(b)

Fig. 6. (a) Simple FCM with negative causality (b) Description of FCM
inference process. Example that shows the system cannot come to a decision
if thresholding is applied.

a positive or a negative relationship is present. As an example,


let’s consider three concepts A, B and C such that A implies
B with a value on the edge equal to 0.7, and B implies C
with a strength of −0.5; then the inference process will reveal
that A indirectly implies C with a value of −0.35, as we can (b)
see the negative causal relationships are preserved thanks to
this approach. Moreover, since the values in the matrix of Fig. 7. Example that shows the interpretation of continuous values as
probabilities (a) Example of an FCM (b) Description of FCM inference
the FCM system represent the past experience, referred to a process.
particular cause triggering a consequence with a certain degree
of confidence, we can claim that the absolute values on the
edges represent probabilities. Hence, in the previous example, on the edge. The adjacency matrix of this FCM is
the value −0.35 is the probability resulting by the product
of the probabilities of the two edges. If A implies B with a 0 0.8 0 0 0
© ª
positive causal relation of probability 0.7 and B implies C ­
­ 0 0 −0.7 0 0 ®®
with a negative causal relation of probability 0.5 (note that ­
­ 0 0 0 −0.5 0 ®
®
(6)
here the negative sign is missing because it is just related ­
­ 0 0 0 0
®
−1 ®®
to the causal relation of the probability), then an action on ­
A will result in a consequence to C with a negative causal « 0 0 0 0 0 ¬
relation of probability 0.35. Then, since the obtained value By interpreting the edges’ values as probabilities, it is possible
is negative, the target concept C has to be considered low. to come to the final decision through a belief propagation path,
Furthermore, if two probabilities with negative sign had been which provides the highest confidence.
involved, their product would have been resulted in a positive In order to define the meaning of the variable score dis-
causal relationship. played in Figure 7, let’s focus on the example reported in
Figure 7 shows an example of a chain composed by five Figure 8. There are two causality chains, which both lead to
concepts connected in a row, with arbitrary values in [−1, 1] the same quality parameter and enforce the same decision (the

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
8

distinguishes between the two possible solutions according to


the causality path, which ends with the higher score. This
result is a direct consequence of the proposed novel inter-
pretation, which changes how the inference process works.
Furthermore, this probabilistic approach enables the system to
choose among different possible results. Moreover, the value
of the score is strictly related to the confidence that an action
will get specific effects: thus allowing to predict the probability
the system will be able to recover from an alert state.
By avoiding thresholding and by employing probabilistic
weights for the edges of the FCMs, the inference process can
(a)
now choose the statuses of the action variables. Then, it can
achieve – according to the knowledge contained in the FCM –
the best result, even when a trade-off is mandatory. In Figure
9, the FCM contains two quality concepts named (Q) Concept
2 and (Q) Concept 3. Its adjacency matrix is
0 0.4 0 0.3
© ª
­ 0 0 −0.9 0 ®
­
­ 0 0
® (8)
­ 0 0 ®
®
« 0 0 0 0 ¬
By looking at the causal relationships among the concepts,
it is easy to understand that it is impossible to satisfy both
the constraints due to the quality parameters (regardless of
the status imposed to the action variable). These situations
are very common in real world, when actions can frequently
have positive consequences but also unwanted ones. However,
a decision has to be made: in this example, the decision is do
not act, thus the action variable remains zero, and maintains
the variable (Q) Concept 2 in an alert state. Then, the system
has decided to fulfil the request of the latter variable, despite
the second quality concept.
The decision-making process, implemented in the inference
algorithm, chooses for the action that has the higher proba-
bility to lower the value of at least one quality variable. This
(b)
result may seem controversial because no relevance has been
considered for any of the two quality concepts. Moreover, in
Fig. 8. Example that shows how combination of paths works (a) Example of this scenario, the system will keep changing the value of the
an FCM, which has two causality chains, which both lead to the same quality
parameter and enforce the same decision (b) Description of FCM inference
action variable in a continuous loop.
process. According to the authors, there are possible solutions for
this problem, and an appropriate choice has to be found
according to the specific system requirements. First, the system
action variable has to be set to a high state in this case). The can find a solution by noting that quality variables are treated
adjacency matrix becomes in the same manner: in particular, let’s differentiate among
quality variables by introducing a weight parameter, associated
0 0.8 0 0 −0.1
© ª to each of them. Thanks to these parameters, the inference
­
­ 0 0 −0.7 0 0 ®® process can now assign the scores to each solution. In this
0 0 0 −0.5 0 ® (7) way, the FCM can provide a higher degree of priority to
­ ®
­
a quality parameter with respect to the other. However, this
­ ®
­ 0 0 0 0 −1 ®®
simple approach it requires a-priori knowledge since these
­
0 0 0 0 0 ¬
« weight coefficients have to be manually set.
The weight associated to each path is used for comparison: The second solution should be the consideration of fluc-
in the output of the implementation reported near the FCM tuations in the response of the system due to the inference
scheme, it is specified a score value, which is always positive. process, as a special feature rather than a limitation. This
This value represents a metric that can be used in order to intrinsic behaviour is caused by the system, which tries to
determine which causality chain better supports the choice of satisfy the request of the first quality concept, at a price of an
a specific decision. It is possible to notice that the system undesired state for the second quality concept. So the request

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
9

C. Combination of Different Paths


In the previous examples, we saw that – when there are two
paths from an action to a quality concept – the system decides
according to their relevance: in particular, this relevance is
related to the probability that an action will generate the
desired effects.
At this point, it may be useful to wonder if it is possible
to improve the decision procedure of Figure 8. Actually, that
can be achieved by combining the two possible paths instead
of singularly selecting them by adding up their weights to get
a more accurate decision. Moreover, it is important to prove
(a)
whether the standard inference process already performs the
combination of the contributions coming from different paths
and, in particular, if such behaviour happens regardless the
configuration of the FCM.
In Figure 9, the FCM singularly considered different paths.
Nevertheless, Figure 11 and Figure 10 show that this may not
be always the right method and, more important, it cannot be
possible to define a-priori which one of the two methods a
generic FCM has to use.
Figure 11 and Figure 10 represent two FCMs, which slightly
differ in their own structure: the first has unequal length
(expressed as number of hops, concept by concept from the
action to the quality variable) of the two chains, while the
second has equal lengths. Given the assumption that the
numbers of nodes and edges should not affect the decision
process methodology, if the two cases are similar then the
outcome should be equivalent.
By looking at the logs of the inference process (right side
Figure 11 and Figure 10), it is possible to see the previous
assumption has not been satisfied: in fact, the final scores are
different. It is also worth notice that the score in the second
case (Figure 10) could have also been greater than one, if
(b) only the values on the edges where slightly different. These
inaccurate outcomes cannot be accepted during a reasoning
Fig. 9. Example that shows the presence of multiple quality parameters in
the FCM (a) Example of an FCM, in which, not thresholding and employing mechanism of a cognition system, thus a solution has to be
probabilistic weights, allows the inference process to find the best trade-off found.
for the action variable (b) Description of FCM inference process. In the first case (Figure 11), the two causality chains are
asymmetric (i.e. they do not enter in the final node at the same
time): this leads the system to consider the two contributions
separately, and to keep the most convenient one. However, in
the second case (Figure 10) – since the two causality chains
of the second quality concept is satisfied but the result becomes are symmetric – they infer on the quality parameter during
that the first quality concept will go back to a bad condition. the same step, hence resulting in the combination of their
contributions. That means the two probabilities, coming from
Nevertheless, the FCM with the proposed new character- the end of the two causality chains, are added up when they
istics can still keep a certain physical magnitude between enter in the final node (the quality parameter).
two bounding levels. Let’s suppose a low value of the latter Each case (Figure 11 and Figure 10) shows a different
mentioned quantity triggers the high state of the first quality example of reasoning error: in the first case, the contribution
parameter, while a high value would have activated an alert of the least significant path is not considered due to the
state of the second one. Our implementation maintains this chain asymmetry, while, in the second, both contributions are
value between these two boundaries. In fact, given these two combined thanks to the chain symmetry, but the final value
alerting situations, the system reacts by activating/deactivating comes from the simple addition of these two contributions.
the action variable, which can invert the current trend of the Even if the values propagating in the map during the
physical magnitude. An additional important effect of this execution of the protocol are considered as probabilities, such
novel proposed approach is also the capability to discover the a behaviour has to be expected since the inference process
most appropriate action by trials and errors, so that it can is based on vector-by-matrix multiplication. This problem,
operate without any a-priori knowledge. which has been discovered, comes from the union of two

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
10

(a)

(a)

(b)

(b)

Fig. 10. Example to show the advantage of combining different paths (a)
Example of an FCM, which has unequal lengths (b) Description of FCM
inference process, which shows that values are erroneously added.

contradictory concepts. First, FCMs are supposed to be simple


and their knowledge easily understandable by humans, and
the vector-by-matrix multiplications necessary in the inference
process has become one of the most known feature, which is
supposed to represent the simplicity of this approach. On the
other hand, a cognition process has to be able to fully exploit
the stored knowledge in order to perform actions [12].
Then, we can understand that the problem lies in the nature
of FCMs. In order to correctly adapt FCMs to solve this issue,
(c)
we need to introduce some modifications in the inference
process implementation: Fig. 11. Example to show the advantage of combining different paths (a)
Example of an FCM, which has unequal lengths (b) Description of FCM
• If n branches, leading to the same concept, are not inference process,which does not wait the outcome of the longest path (c)
composed of the same number of nodes (asymmetric Description of FCM inference process showing that inference always finds
optimal result.

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
11

paths) then, during the inference process, inference on by interpreting weights of edges as probabilities, there is the
the shorter path has to wait until the one on the longest need to propose other solutions to face loops.
has arrived. First, it would be possible to set a limit in order to determine
• When n concepts lead to the same node, their contribu- whether the inference process has to continue through the
tions to the final value have to be treated like probabilities, chain of concepts or it has to stop: in particular when the
i.e. they cannot be simply added up. Hence, the shared absolute value of the probability reported along a chain of
target node is inferred with a probability, calculated as causal relationships is smaller than the threshold, the latter
the sum of independent random variables. action has to be performed. Moreover, the value of this limit
The first condition imposes that – regardless of the concept should be close to zero to allow the system to explore possible
class type – when there is more than one arrow entering a solutions when weak knowledge is present on the edges, thus
node, a decision for the final value of this variable only has to resulting in small absolute values.
be computed when all the contributions – coming from other The idea behind this first approach comes from the fact
nodes – are available. The second condition instead is related that loops in FCMs make the inference algorithm spinning
to how these contributions have to be combined together: as continuously across the involved nodes and this process never
discussed before, the values propagating on the edges during stops because a steady solution cannot be found. It is worth
the algorithm execution are treated like probabilities, so that notice that cycle after cycle, the output of the inference process
the sum is not allowed because it could possibly lead to values converges to an interval close to zero: by definition, if the
out of [0, 1). causality values on the edges are bounded by the continuous
As an example, let’s consider two causality chains, reaching set [0, 1), then the absolute value of the output will decrease
a certain target node with probability 0.7 and 0.8 respectively. iteration after iteration. Given that, the introduction of the
As a consequence, their sum is 1.5, which it may be approxi- proposed limit allows the algorithm to indirectly realise when
mated to 1.0. Then, a causal relationship, represented with the it is stuck in a loop.
unity value, reflects a fully deterministic connection, which is The second proposed approach (the one applied in the rest
a nonsense if we consider that the ones closer to the sources of this work) is based on a preprocessing operation, performed
are not. before the inference process starts. During this stage, the
So, the proposed solution overcomes this issue by treating cognitive engine runs an algorithm similar to the Spanning tree
probabilities as they are and by performing the addition protocol: that can detect and remove cognition loops from the
accordingly. The sum of independent random-variable formula FCM. This idea is supported by the consideration that, in the
fits the bill and provides an output in the continuous set [0, 1). matrix representing the knowledge, loops do not supply any
It is worth notice that every contribution coming from each additional useful information to the final result of the inference
causality chain positively contributes to the final outcome. It process, thus they can be removed with confidence.
is also worth to notice that before combining the contributions, According to this method, there is no need for a limit, which
the positive and negative causal relationships are separated and may not be straightforward to be set to an appropriate value.
only at the end are summed up. In particular, the proposed This second approach should be preferred to the first because
expression is only used among contribution with the same sign. it avoids useless iteration across the same concepts forming
Formally, given n variables the loop. However, the drawback of this method is a higher
implementation complexity and an additional overhead in the
y[n] = y[n − 1] + x[n] − y[n − 1] · x[n]
(
inference process.
(9)
y[0] = 0 Figure 12 depicts the FCM before and after the execution of
the procedure of loop removal. The adjacency matrix referred
Referring to examples in Figure 11 and Figure 10, the score to Figure 12(a) is
variable is now meaningless once there is only one quality
variable because all the contributions, coming from different 0 −0.1 0 0 0 0 0
© ª
paths, are combined together, thus resulting in a single score ­ 0 0 0.6 0 0 0.4 0 ®
­ ®
value. It is possible to see that the outcomes of the two FCMs ­
­ 0 0 0 0 0 0.9 0 ®
®
are now equal as expected: in the inference process log, it ­
0 0 0.5 0 0 0 0
®
­ ® (10)
is clearly visible that the implementation waits for the first ­ ®
branch on the left to advance and, once they are ready to
­
­ 0 0 0 0 0 1 0 ®
®
­ ®
enter together in the quality variable, the algorithms proceed ­ 0 0 0 0 0 0 0.6 ®
­ ®
with the sum of their probabilities. 0 0 0 0 0 0 0
« ¬
Especially, Figure 12(b) reveals that Concept 3 has been re-
D. Causality Loops Removal moved from the collections of the considered nodes during the
If FCMs perform thresholding and map values in the dis- inference process because it does not provide any information
crete range [0, 1], the inference process can produce matrices since it is part of a loop.
with redundant vectors. Then, once the algorithm recognises The modifications applied so far on the FCMs’ inference
that the current outcome has already been found in the past process resulted in an augmented complexity of the FCM-
steps it just stops. Nevertheless, by avoiding thresholds and based system. The first proposed approach can search faster

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
12

to the suggestion reported in [12], if no action has been found


during the inference process, the system has to randomise the
possible actions in order to discover what may lead to any
effect. However, there was no indication related to how such
a mechanism should be performed.
The task related to the discovery of relationships from the
action to the quality variables is particularly critical, mainly
because communication networks are dynamic systems rapidly
changing, and a certain action may only be effective for a
short period of time. By this consideration, it is a fundamental
requirement to deploy a system, which can discover the correct
set of actions as fast as possible, before the environmental
conditions change. An exhaustive search may not be feasible
because it could take a long time, since the number of sets
increases exponentially with the number of action variables.
(a) On the other hand, if each action is tested independently from
the others, then the risk is to neglect possible effects, which are
only manifested when a combination of actions is examined.
To solve the above issue, this work proposes a novel ap-
proach, which contemplates both the previous considerations.
The main idea is to select a set of combinations of actions
capable to stimulate the system in a relatively small number
of steps. That is inspired by how humans perform when they
do not have any knowledge about the functions of a certain
device, so they can solely rely on their common sense. For
example a typical behaviour of a user that does not know how
to use a radio, is to push many buttons at the same time until
he is able to achieve the desired goal. The user is aware that
he will only need to press few buttons in order to reach his
objective. However, since he does not know which ones belong
to the correct combination, he will continuously try different
possibilities.
(b) According to the aforementioned requirements, there is the
Fig. 12. (a) Starting FCM, before loop removal (b) Ending FCM after loop
need to operate quickly over the network through actions in
removal procedure. order to discover new causal relationships. The formula used
in this article to generate k sequences of actions when in the
system there are N action variables is
and more time-effective compared to the second: actually, the
choice of the first would lead to longer and harder results to yk,n = (−1)k+ bc c (11)
understand because the output of the inference process would where c = n−1 b k2
c , 1 ≤ n ≤ N and 1 ≤ k ≤ b2(dlog Ne +1)c.
N 2 2
be affected by many irrelevant values, related to the spin of In particular, it represents the starting point, which is used by
the algorithm around the loop. The system would properly the cognitive engine to generate action matrices to determine
work anyway and the final decision would not change: then, and to set the initial value of each action variable.
for completeness, both the approaches have been discussed in This formula has been derived from the generic family
this subsection. of discrete transforms; in our scenario, the one-dimensional
sequences at different frequencies are treated as scrambled ac-
E. Multiple Action Variables tion patterns. Actually, the authors have proposed this formula
Besides the entities devoted to reasoning and learning, an because of its interesting following properties:
FCM-based system also contains two fundamental modules, in • half of the generated sequences are equal to the other half
charge of performing the transcription of measurements from but inverted, and there are always at least bN/2c active
the real world to the logical one and vice-versa: these units actions at each iteration to boost the action impact on the
are called sensors and actuators. This subsection will discuss system;
the latter. • regardless of the position in the actions array, every
In an FCM, there might be more than one variable belonging iteration each action has the same chance to be activated;
to the action class: especially, when many causal relationships • the number of action sequences grows logarithmically
are missing in the matrix, the decision-making process might with the length of actions array.
have to react to a bad-shaped situation of the system without Next, expression (11) produces the output matrices in Figure
the possibility to rely on the necessary information. According 13, which are particularly effective to train different combi-

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
13

Fig. 13. Matrices to numerically derive the formula to generate k sequences


of actions when in the system there are N action variables.

nations in a small amount of iterations. As just discussed,


the number of steps is now logarithmic hence the system
can be probed in reasonable amount of time. In particular,
it is bounded by a range that is larger compared to the case
where each single action is singularly tested, but smaller (a)
compared to the case of exhaustive search, which is an NP-
hard problem. The performance achieved by the cognitive
SDN system presented in Section IV are satisfactory, using
relation (11). However, the training of proposed FCMs may
be further optimised and improved: that is out of the scope of
this work. Further methods of FCMs’ training can be found
in [11].

F. Environmental Variables as Blocking Conditions


As previously explained in Subsection III-E, it is possible
to have more than one action variable, for which different
combinations are tested in order to discover the best set
of action values, according to the knowledge stored in the
map. However, in real world performing and acting or, more
generally, changing the value of a variable has an impact and
a cost.
Let’s suppose the controller of a cellular base station decides (b)
to increase the output power of an antenna to provide a
Fig. 14. Inference process where the blocking environmental concept BSs
better service for a set of users. That choice can be made nearby is not active (a) Fuzzy cognitive map (b) Description of FCM inference
because in that particular area there are many customers, process.
who are consuming services characterised by a high demand
of bandwidth. Anyway, the situation is hardly sustainable
are no neighbouring base stations: thus it is possible to increase
by the serving base stations because it is not possible to
the power, moving the system to a safe condition where the
deploy modulations with higher spectrum efficiency, due to
state of the variable Poor QoS is low. On the other hand, the
the distance. As a consequence, the FCM-based system can
second case has a high environmental parameter (see the log),
enter in a bad-shape situation, and the inference process can
so the inference process stops and communicates to the system
be run to find a solution. A proposed action may be to increase
that it is better not to act. Indeed in the last scenario, the Poor
the output power of the antenna because the system has learnt
QoS alert state of the variable could not have been resolved
in the past that it is possible to improve the QoS by increasing
by acting since it would be resulted in a worse situation.
the feeding current. Nevertheless, when a base station operates
with more power it can provide a better service to the users,
but there is also the risk that this increment will interfere with G. Negative Nodes Values
other base stations located nearby. Despite significant part of the literature referred to FCMs,
Figure 14 displays an FCM, which already yields this this work proofs that adding negatives values to concepts can
knowledge, represented by a number on the relative edge, boost the performance of the system. Instead of range [0, 1],
whose absolute value is particularly high: that means the the idea is to use the discrete set [−1, 0, 1], particularly suitable
system is aware that the increase of output power through an for modelling concepts, for which low states (zero values)
antenna leads to interference very often. Figure 14 and Figure yield a double meaning.
15 report the outputs of the two executions of the inference According to the authors, the zero value should be assigned
process. In the first case, the blocking environmental concept to a concept when its state is inactive. Next, a negative value
BSs nearby is not active, representing a scenario where there should be acquired by the variable when a concept has an

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
14

(a)
Fig. 16. Reference architecture for Cognitive SDN based on FCMs.

IV. A C OGNITIVE S OFTWARE -D EFINED N ETWORKING


A RCHITECTURE
The SDN concept of centralising the control plane and
separating it from the data plane [4] sets up fertile ground
in order to effectively include the cognition processes in the
networks. This section clarifies why it has been decided to
locate cognition processes in the control plane of SDN. Figure
16 depicts the reference architecture of the proposed cognitive
SDN architecture.
By considering cloud-computing vision, system manage-
ment is centralised: therefore, the idea is to exploit this char-
acteristic of SDN by placing an FCM-based Cognitive Engine
over the SDN controller. In this way, the cognitive module
(b)
gains direct access to all network information processed by the
Fig. 15. Inference process that takes into account neighbouring base stations network manager, hence allowing learning and acting through
as blocking environmental concept (a) Fuzzy cognitive map (b) Description
of FCM inference process.
a unique interface. The network manager’s interface with the
FCM-based cognitive engine is used to exchange network
statistics, and commands to update network characteristics.
Recent developments in SDN theory have shown the need
effect on the system, so when its influence is the inverse of of designing intent-based interfaces, for exposing network
the high state case. As an example, let’s suppose there is a functionalities to the upper layers. An intent-based approach
positive causal relationship between the concepts Financial starts from the idea that a tenant should not obtain both
Crisis and Poverty, where the first is the cause and the second network details and access to the controller; nevertheless, a
is the effect. The inference process of the FCM-based system tenant should only specify what the scope is, and then the
will reveal that, if the cause is in high state, then we can expect controller should perform effective operations to achieve the
the propagation of this status to lead to a high state also in the requirements.
poverty concept; on the other hand, when the Financial Crisis This approach has been widely adopted by OpenFlow
variable takes the low value then the Poverty concept should controllers [37]. Among those, it is particularly interesting
acquire the same status as well. the work that has been performed on the ONOS controller
This example conveys that the zero-valued concept may [38], which contains an intent-based interface that is capable
often be related to the fact that the economy is in a steady- to perform advanced reasoning operations.
state condition, thus we cannot claim anything about poverty; The controller of the proposed cognitive SDN system has
on the other hand, low Financial Crisis value may specifically an intent-based interface to implement ’controller neutrality’.
represent a growing economy, which has an active negative In fact, users’ requests represent the input requirements from
impact on Poverty concept. In other words, in particular the tenant. The intent-based controller is capable to translate
scenarios, it is important to distinguish between an inactive (compile) intents into data plane rules according to the current
and a negative-state of a concept otherwise it would not be network conditions: basically, once it retrieves the intention
possible to accurately model the parameters of the system. from the user, it tries its best to accomplish the designated

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
15

a congestion happens. This event provides different statistics to


the SDN systems, which are given to the FCM to be analysed.
That results in further learning and subsequent autonomous
adaptation to the new network conditions. The last part of the
signalling has the same structure but shows the procedures
applied when congestion is solved.

A. Latency and Overhead


In order to react to network events, a certain amount of time
is needed for the cognition process to find a solution: this can
be split in
• the time required for the messages exchange between the
cognitive engine and the controller, plus the time required
for the inference process to complete,
• the time that has to be waited because the cognition
process is still evaluating whether the application of
certain actions are having any effect in solving an alarm
state.
In the first case the FCM has already enough knowledge for
applying a set of actions in response to an alarm state, the
reaction time of the cognitive engine is usually very short
and only depends on the computation and message exchange
capabilities of the system.
The latency overhead due to message exchange strongly
depends on the application-specific design choices. In the most
general case, each variable update (cognitive engine input)
and action (cognitive engine output) requires one message
exchange with the network controller; in case of large FCM
maps some optimizations could be introduced, for example
by performing the threshold comparison of the measurements
in the controller and notify the cognitive engine only when a
change on status has occurred.
Regarding the computational overhead, there are two main
activities that are performed by the cognitive process: learn-
ing and reasoning; the following measurements are aimed
Fig. 17. Example of signalling between the FCM-based cognitive engine and
at showing the impact of these tasks to the overall system
SDN (controller and data plane). performances. It is worth to report that these measurements
have been obtained with a non-optimized cognitive engine
implementation, meaning that these absolute latency values
task, and it manages to react to the events in the network (e.g. can be further reduced with proper implementation choices.
broken link) in order to maintain the intent alive. Especially, in From performance perspective, the goal is to discover how the
the latter case, the controller recompiles the intent in order to proposed system scales when more complex network scenarios
adapt the corresponding actions on the network to the current with many variables are considered. Nevertheless, the order of
environmental situation. These operations cannot be performed magnitude of these values can be considered suitable for most
by following a rigid analytical model of the controlled system, networking scenarios, as these never exceed few milliseconds
since very often the control logic can rely to little a-priori with a 100-variable FCM with many causal relationships.
knowledge, and the effectiveness of certain actions strongly Figure 18 depicts the measured amount of time required
depends on the infrastructure characteristics and the operating for completing the FCM update process, by considering also
context, that have to be empirically discovered. the size of the map. The graph shows three boxplots related
Figure 17 shows an example of signalling between cognitive to the time spent for the update for three FCM sizes, from a
engine and SDN system. The process starts with the user, small FCM with only 10 variables to a more complex with
who sends the requirements to the SDN controller. Next, SDN 100. The boxplot representation has been chosen because it
controller updates OpenFlow tables at the data plane. The data allows to show in a compact manner statistical information
plane sends network statistics to the SDN controller, which are of a set of data (in this case 160 samples for each FCM
interpreted and converted into variables for the FCM. Then, size has been used), obtained by feeding the cognitive engine
these values are passed to the FCM-based cognitive engine, (the actual implementation) with random variable changes, in
which uses them for the learning/decision process. Suddenly, order to simulate a complex network with many events. As

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
16

Fig. 18. Time needed for the FCM updating process by considering the size
of the map. The boxplots are related to the time spent for the update for three Fig. 20. Time needed for the FCM reasoning process by considering the
FCM sizes, from a small FCM with only 10 variables to a more complex one number of causal relationships, due to the small FCM size (10) there is no
with 100. On each box, the bottom and top blue edges of each box indicate relevant differences in the computational time.
the 25th and 75th percentiles respectively, the central red mark indicates the
median, and the two horizontal bars comprising the box are the minimum and
maximum values of the range of samples.

Fig. 19. At the top of the table, the maximum amount of cognitive
relationships is reported for each FCM size. Then, these values are used
as references for comparing them with three percentage levels of FCM link
filling.

previously stated, the main information obtained is related to


the increase of processing time when the FCM grows; results
show that time spent for the computation has approximately
doubled with a FCM size change from 10 to 100 variables, Fig. 21. Similarly to Figure 20 it has been reported the processing time
proving that the updating process can scale well. required for different amounts of causal relationships, because of the greater
FCM size (50) their values show some small differences among them.
As results show, these operations does not demand high
resources, since the update process only involves one variable
status change and others that happened within the same per- the computation is reported for different amount of information
sistence time interval. Indeed, the reason behind the increase stored in the FCM. For each FCM size, the maximum number
of processing time with the FCM size is related to the higher of causal relationships that can be established changes: it is
number of causal relationships that are usually present. worth to notice that this value is given by N 2 · 5/9 because
Different from the learning process is the reasoning one. not all the causal relationships are stored where N is the FCM
The exploitation of the available information contained in the matrix size (or number of variables). For example, it is not
FCM requires the execution of the inference process which possible to establish a causal relationship between two action
however, from the computational perspective, does not contain variables. Table 19 helps to map the absolute number of links
any exponential algorithm, meaning that it is expected to scale. into the corresponding percentage of FCM information; these
values have been chosen in order to allow a fair comparison,
Figure 20, Figure 21 and Figure 22 report the time required since for each FCM size it is reported the time spent for the
to find a suitable solution to an alarm state by considering inference process when the FCM is full at 20%, 50% and 90%
both the number of FCM variables and the number of causal approximately. The obtained results show that the proposed
relationships. For these tests, it has also been considered the reasoning technique is capable to provide action plans in short
number of causal relationships, since it proves to impact the amounts of time, which is aligned with the one reported by
final performance measurements. Each graph is related to a ONOS.
fixed number of variables (from left to right, 10, 50, 100 Figure 23 shows the time spent by the inference processes
respectively, with 20 samples per box) where the time spent for for different FCM sizes and different amounts of learnt

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
17

Fig. 22. With the greatest reasonable FCM size (100) the differences in
the processing times depending on the number of causal relationships have
increased with respect to Figure 20 and Figure 21. However, it is worth
to notice that these have never exceeded the half millisecond, proving the
suitability of the proposed approach.

Fig. 23. Time spent by the inference process, considering both different FCM
sizes and different amounts of learnt relationships.

relationships. The results reveal that there is a small time


difference between an almost-empty FCM and an almost-full
one, only for large FCMs; on the other hand, smaller FCMs
shows negligible difference in time spent.
Figure 24 depicts the comparison between the latency due
to legacy ONOS-SDN networks with different characteristics
and the additional overhead (in time) due to set up of new
proposed FCM-based SDN system (with different FCM sizes).
This temporal analysis comes from the fact that 5G and beyond
5G networks will have latency as a main requirement: thus a
cognitive SDN framework, managing a virtual network, should Fig. 24. Representation of delay of legacy ONOS-SDN systems and of
not affect significantly the latency of legacy SDN systems. The the additional latency introduced by our Hebbian-based FCM algorithms.
picture clearly shows how negligible the overhead (additional The comparison of their orders of magnitude shows that our FCM solution
introduces negligible delay in legacy ONOS-SDN networks. The horizontal
delay) of the FCM inference process is with respect to axis considers the changing size of FCM, thus it is referred to FCM line (other
operations in legacy SDN networks. For example, with a large ONOS-referred lines remain constant). In order to increase the readability of
100-variables FCM, the impact on the delay for setting up an the picture, in the legend, the respective values of vertical axis are reported
between round brackets.
end-to-end flow through 300 switches is of (3.63 ms / 3273 ms)
≈ 0.11%, meaning the overall time required by the controller
to act on the network is less than 3277 ms.
The goal of comparing order of magnitudes is to show that
the proposed FCM-based SDN paradigm introduces negligible
temporal overhead when the Cognitive Engine is implemented

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
18

on top legacy SDN controller. Indeed, the proposed approach


is not meant to replace existing SDN solution, but rather to add
new more effective cognitive functionalities. Figure 24 reports
the average processing time required by the FCM inference
process, and other time-based measurements obtained from
some ONOS performance tests [39]. By looking at the picture,
the impact of FCM operations on ONOS-SDN is in the order
of [10−4, 10−2 ] ms (for FCM size = 100) and in the order of
[10−6, 10−4 ] ms (for FCM size = 10). It is worth to notice
that FCM-based performances depend on factors such as the
FCM size and the number of relationships, while the delay
of ONOS-SDN controllers mainly depends on SDN-specific
tasks, where network latency between controllers and switches
plays a relevant role. The same horizontal axis on the graph
is referred to FCM size and only affects the curve related
to FCM-based SDN. In fact, setting up an OpenFlow rule Fig. 25. Scheme of the implementation of the cognition process.
through a long path of switches has no correlation with having
a large FCM, since a single action variable on the FCM can
trigger the actuation of many configuration at a time. It is
also important to remember that absolute delay times strongly
depends on the implementation choices and optimisations:
from this perspective, with respect to the software that has Fig. 26. Differential Hebbian Learning behaviour.
been developed for validation, there is room for improving
the implemented FCM-based solution, which can lead to a
further reduction of processing time. their asynchronous nature, i.e. as they occur in the real world,
allows the system to be more reactive to the changes and
thanks to this approach it is possible to provide the ability
B. Learning and Reasoning
to find the whole causal relationships happening among the
Both sensing and acting operations require accurate design concepts. On the contrary, with the previous approach, some
choice in order to correctly translate the quantities from logical of these relationships would have been lost. In Figure 26 and
to real world; this stage is particularly critical because it Figure 27, a comparison between the two systems is provided:
deeply affects system performance. At this point, we can in the old method, it is clearly visible that the relationships
start analysing how an FCM-based system implementation between events E1 and E2 has not been considered. Instead
should work with the concepts and their changes of status, the causality connections between the couples E1→E3 and
from the theoretical point of view of Differential Hebbian E3→E4 have been registered even though. It is present a
Learning. When it comes to implement such approach, other large duration difference between the two cases (the first
parameters have to be introduced. The first question that has lasted much longer than the second). With our novel approach
to be answered is "if A changes its status, how long should instead, as we can see in Figure 27, two events are considered
the system wait for event(s) X(es) for considering it(them) the cause-effect when the difference of their timestamps is smaller
effect(s) of cause A"? The correct interval time ∆ depends on than ∆ time and greater than µ · ∆, with 0 < µ < 1. The
the application, some a-priori knowledge about the reactivity parameter µ is mandatory in order to prevent the enforcement
of the system may help the designer in choosing the more of false causal relationships, indeed if two events occurred
appropriate value, otherwise this parameter should be set after very close one to respect of the other: then it is likely that
some trials: if no edges are built in the FCM when the they have to be considered contemporary meaning that there
supervisor is aware that concepts has changed their status, is no relations between the two. In the example, the causal
then a higher ∆ value should be chosen. Figure 25 reports relationships E1→E2 and E2→E3 have been registered since
the scheme of implementation of the cognitive process: in the duration of each of them is shorter than ∆ and longer than
particular the diagram shows that, apart from the learning µ · ∆.
process, represented in the first block, the system is triggered Finally, the delay is related to the learning coefficient and
when events related to quality concepts enter in a bad shape
condition i.e. there is an alarm. The Φ time is related to
the persistence of the selected action(s): especially, in order
to detect whether an action(s) has(ve) been successful, it is
mandatory to wait for a certain amount of time in order to
allow the effects of such an action to happen. Usually it is
better if the Φ value is smaller than the ∆ time, because
otherwise it would not be possible for the FCM to connects
the effects to the actions. Processing the events maintaining Fig. 27. Behaviour of our new FCM-based SDN method.

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
19

the persistence timer: in order to learn new causality relation-


ships, the cognitive engine has to wait for events to happen
within a temporal window and for reacting it has to wait for
discovering whether the selected actions set is effective. An
excessive reduction of the values of these parameters prevents
the cognitive process from establishing causal relationships,
or it leads to the establishment of false ones. The choice of
the most appropriate parameters is necessary to tune properly
system performances while a trade-off between reactivity and
correctness has to be found. These parameters and their impact
are expressed in Figure 19.

C. Number of Network Variables


In the cognitive engine, variables represent the network
environment that has to be managed; the number of variables
to be defined has an impact on the reasoning granularity, which
eventually affects the system performances. A small number of
variables prevents the cognitive process from discovering more
subtle relationships, leaving only the most trivial ones (for Fig. 28. First scenario to test the performance of cognitive SDN.
example, a system with only ”Move users” and ”Congestion”
variables would not provide information about other aspects,
for example the user distribution and the network load). composed by data and control plane. The reason to maintain
On the other hand, many variables could be hard to de- the two components independent – even in the validation
fine and could possibly overload the cognitive engine with process – relies on the fact that the cognitive process can
reasoning operations with small significance; for example, a actually work regardless of the controlled infrastructure.
concept that has no correlation with the considered network Figure 28 depicts the first testing environment. As stated
environment should be discarded. There is not a particular above, the SDN network is emulated in the virtual machine of
bound on the number of variables in an FCM, however, the Mininet, which exploits the network name-spaces functionality
authors propose to start with a large number of variables, of the Linux kernel and virtualises the full-network stack of
and then remove the ones that have not been part of any many virtual hosts. Such virtual hosts are the nodes of the
causality relationship. In any case, there is the indication that network: each of them runs a Java program, that connects
a large FCM is composed by ”dozens of concepts”, meaning to the controller via the Control Network. The role of that
that the maximum size of 100 variables, contemplated for Java software is to manage the activity of the nodes, i.e.
the performance evaluation, is close to the highest reasonable this agent can start an instance of Iperf and can generate
number. traffic towards another node if requested by the controller.
Moreover, it can periodically report the current throughput to
D. Convergence the controller. Next, given these measurements, the controller
The proposed cognitive engine, with its capability to build can detect whether a congestion state is occurring, i.e. it gets
new knowledge and change it according to network conditions, information for the cognitive engine to perform reasoning.
requires a specific definition of convergence. The learning The reason to separate the Experimental from the Control
process can be considered successfully completed when, for Network, is related to the fact that the experiments, which
the same input provided to the cognitive engine, there is a run on the former network, must not interfere with the service
set of optimal actions that can be obtained by simply using information, exchanged between the nodes and the controller.
the pre-discovered knowledge: these actions lead to the same This design choice is particularly important: let’s consider the
outcome that was already learnt. In this condition of ”static controller runs tests on the Experimental Network and deeply
environment”, the causal relationships are enforced and the stresses the network infrastructure, where links are often at
cognitive process converges. However, the main objective of their maximum capacity. Meanwhile, the control traffic must
the proposed work is to design a controlling system capable to not interfere with the measurements related to the experimental
react to the dynamics of real-life networks, where some actions network activity.
are only effective for a certain amount of time, after which new The whole virtual network is created and managed by
solutions must be found. From this perspective, it is safe to the Controller, which is connected to a Python agent that
claim that the cognitive process never converges because the controls the Mininet API, and sets up and configures the actual
controlled network continues to change its characteristics, it networks in real time. In particular, in the initialisation stage,
is a continuous adaptation to new conditions. the former component configures the network topology on the
experimental network. Besides that, the controller is also in
V. R ESULTS AND D ISCUSSIONS charge of leading the experiment, so it can act on the network
In order to validate the FCM-based SDN, proposed in infrastructure. Furthermore, the controller can both redistribute
Section III, a test network has been developed in Mininet [40], the users in the network (changing the switch to which users

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
20

(a)

Fig. 30. Scenario to test the performance of cognitive SDN, where users’
connections with switches have been changed.

(b)

Fig. 29. Fuzzy Cognitive Maps referred to scenario in Figure 28 (a) This FCM
underlines that system starts with no a-priori knowledge (b) The Cognitive
Engine learnt the effectiveness of reducing bandwidth.

are associated), and reduce the throughput of each users: these


two elements represent the FCM’s Action variables of the
cognitive system.
Side by side, the FCM’s Environmental variables are: Bad
User Distribution, which is related to a non-homogeneous
users’ distribution among the switches, and High Load, related
to the overall network utilisation. Finally, the FCM’s Quality
variable is Congestion, which is triggered whenever at least an
(a)
user cannot achieve a throughput greater or equal to its profile
bandwidth (minimum guaranteed bandwidth). The users can
also randomly move across the network switches, and inter-
mittently start and stop the Iperf generated throughput: this
feature is useful to simulate when users move and not always
transmit.
The remainder of this section shows the results, which
demonstrate how the Cognitive Engine is actually capable
autonomously ’to unlearn’ old beliefs once they do not hold to
the current environmental situation any more, and immediately
after, to learn the new effective countermeasures for an alarm
state. In Figure 28, the congestion state is obtained since all
the clients transmit to the receiving node h9.
(b)
Next, Figure 29(a) represents the related FCM, which high-
lights that the controller has no a-priori knowledge. The con- Fig. 31. Fuzzy Cognitive Maps referred to scenario in Figure 30 (a) Cognitive
Engine erased past knowledge and now goes on by trials. The objective is
troller starts by activating the Move Users variable, but since to get safe state for (Q) Congestion (b) The Cognitive Engine learnt that
hosts are already homogeneously distributed it has no effect; redistributing users among switches is effective. Creation of new relationship
once it realises that this action is not suitable to the current between (A) Move Users and (Q) Congestion.
network condition, it tries to activate the Reduce Bandwidth
variable, which becomes effective since the congestion state

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
21

Fig. 32. Graph which highlights the three main moments of the experiment. For clarity, some values have been slightly changed in order to allow a better
representation: in reality the response of the Cognitive Engine to an event is much quicker than what seems to be from the graph. The two actions available
at the Cognitive Engine are reported by the red and black plots, while the blue one reports the congestion state. Next, the light-green and light-blue lines are
related to the two environmental variables. For all of them, the ’on’ and ’off’ values are related to the active and non-active variable status respectively. The
feedback to the cognitive system is retrieved each time: after a change of status of some action variables, there are other environmental or quality variables
that change their status.

is not present any more. was previously ineffective – has now proved to be able to
At this point, a new causal relationship is created between reduce the congestion, thus a new causal relationship between
variables Reduce Bandwidth and Congestion, and it is enforced this action variable and Congestion is created and enforced
each time the controller acquires new evidences about the (see Figure 31(b)).
effectiveness of the action (see Figure 29(b)). Furthermore, a In order to let the controller strengthen the new belief, we
new relationship between Reduce Bandwidth and High Load have manually forced the users to move towards a single
is discovered: here, the latter is triggered when there is a switch, resulting in new congestion states being triggered.
congestion because it is in this period that the bandwidth on The final result not only contains information between the
the gateway is saturated. After few minutes, the bandwidth on currently working action variable for solving the congestion
the gateway is increased to 100 Mb/s, resulting in a network state, but also includes certain causal relationships among
condition where congestion disappears. As depicted in Figure different environmental variables. Figure 32 shows the overall
32, the Cognitive Engine do not act any more, but the acquired experiment variables statuses over time. Especially, it is possi-
knowledge is preserved. ble to recognise three main moments: first, Reduce Bandwidth
At this point of the experiment, we manually move the is the effective action, second, the latter action is not working
users among the switches, resulting in the network topology any more due to the changed network topology, third the
of Figure 30: the variable Congestion is triggered due to the Cognitive Engine realises that the new effective action was
bottleneck in the link s4–s5 (h9 remains the receiver), and Move Users.
the Cognitive Engine – which is completely unaware of the During the entire test duration, the Cognitive Engine has
new change in the infrastructure – has to react. It first tries to mixed the learning and testing stages: at the beginning, for the
enable the Reduce Bandwidth action variable, that FCM has first 40 seconds, due to the complete absence of a-priori knowl-
proven to be effective in the past, but, since the hosts have edge the cognitive process has learnt by test and trials: the only
already reached the lowest guaranteed throughput, this action effective action to solve the congestion state was to reduce the
cannot solve the issue. bandwidth. In the next period, this belief has been enforced by
After few trials, during which the causal relationship be- the successive congestion states that raised and for which the
tween Reduce Bandwidth and Congestion variables is weak- Cognitive Engine has managed to successfully handle them
ened, the controller returns in its original situation (Figure (i.e. the congestion variable has switched from ’on’ to ’off’
31(a)), where it has to discover which action is more suitable state after the action had triggered). This phase continues until
for bringing the quality variable in a safe state. Then, the the 330th second, after which the network scenario completely
controller discovers that Move Users action variable – which changes: due to deep variations of users’ displacement, the

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
22

in detail the proposed architecture for cognitive SDN. The


characteristics of this architecture have been studied and its
performance has been evaluated in some sample scenarios.
It is important to notice the capability of proposed cognitive
SDN to adapt and to learn, also without human intervention.
Moreover, the presented general cognitive architecture can also
work with different learning techniques.
To the best of authors’ knowledge, this is the first work
describing in detail FCM-based SDN networks, with a specific
learning algorithm and with exhaustive tests. Future works will
be focused on the following points: (i) further optimisation of
FCMs, by using both more advanced learning techniques and
training methods; (ii) the experimentation of other learning
Fig. 33. Comparison of upper bounds achieved by ML-based SDN solutions. technologies alternative to FCMs; (iii) the evaluation and test
Upper bounds on accuracy of legacy solutions come from [13, Table VIII]. of FCM-based SDN in further more complex scenarios.
The underlined paradigms are belonging to either unsupervised ML or RL.

R EFERENCES
reduction of bandwidth becomes not effective. Once again, [1] R. W. Thomas, D. H. Friend, L. A. DaSilva, and A. B. Mackenzie,
the Cognitive Engine has to learn that by acting (period 330- “Cognitive networks: adaptation and learning to achieve end-to-end
520 seconds) during which the congestion state remains to performance objectives,” IEEE Communications Magazine, vol. 44,
no. 12, pp. 51–57, Dec. 2006.
an active level. Finally, the-out-of-dated knowledge model is [2] B. S. Manoj, R. R. Rao, and M. Zorzi, “CogNet: a cognitive complete
demolished and the Cognitive Engine is ready to experiment knowledge network system,” IEEE Wireless Communications, vol. 15,
with new action patterns; it tries by moving the users which, no. 6, pp. 81–88, Dec. 2008.
[3] R. Mijumbi, J. Serrat, J. L. Gorricho, N. Bouten, F. D. Turck, and
differently from the initial scenario, is now an effective action R. Boutaba, “Network function virtualization: State-of-the-art and re-
(see from the 520th seconds till the end). Finally, excluding the search challenges,” IEEE Communications Surveys Tutorials, vol. 18,
periods during which knowledge had to be built either for lack no. 1, pp. 236–262, Firstquarter 2016.
[4] B. A. A. Nunes, M. Mendonca, X. N. Nguyen, K. Obraczka, and
or out-of-dated knowledge, the system has shown an accuracy T. Turletti, “A survey of software-defined networking: Past, present,
of 100% (7 out of 7 congestion status in the first phase and 5 and future of programmable networks,” IEEE Communications Surveys
out of 5 in the second one). Tutorials, vol. 16, no. 3, pp. 1617–1634, Third 2014.
[5] M. Agiwal, A. Roy, and N. Saxena, “Next generation 5g wireless
Figure 33 depicts a comparison of upper bound of accuracy networks: A comprehensive survey,” IEEE Communications Surveys
of legacy ML-based SDN and the proposed FCM-based SDN. Tutorials, vol. 18, no. 3, pp. 1617–1655, thirdquarter 2016.
In the context of routing optimisation, [41], [42], [43] calcu- [6] C. Fortuna and M. Mohorcic, “Trends in the development of
communication networks: Cognitive networks,” Computer Networks,
lated the accuracy of neural networks applied to SDN. In the vol. 53, no. 9, pp. 1354 – 1376, 2009. [Online]. Available:
context of resource management, [44] calculated the accuracy http://www.sciencedirect.com/science/article/pii/S1389128609000085
of some supervised ML algorithms (Naive Bayes, Linear [7] J. R. P. Mähönen, M. Petrova and M. Wellens, “Cognitive wireless
networks: Your network just became a teenager,” in 25th Conference
and radial support vector machine and k-neural networks) on Computer Communications, 2006.
and [45] evaluated the accuracy of unsupervised ML and [8] C. Facchini, F. Granelli, and N. da Fonseca, “Cognitive service-oriented
RL applied to SDN. As it is possible to see, the proposed infrastructures,” Journal of Internet Engineering, vol. 4, no. 1, pp. 269–
278, 2010.
algorithm can reach the maximum of accuracy overpassing the [9] B. Kosko, “Fuzzy cognitive maps,” Int. J. Man-Mach. Stud.,
legacy ML-based SDN paradigms employing either supervised vol. 24, no. 1, pp. 65–75, Jan. 1986. [Online]. Available:
ML, unsupervised ML or RL. Furthermore, the novel FCM- http://dx.doi.org/10.1016/S0020-7373(86)80040-2
based SDN handles both routing optimisation and resource [10] E. I. Papageorgiou and J. L. Salmeron, “A review of fuzzy cognitive
maps research during the last decade,” IEEE Transactions on Fuzzy
management while other legacy solutions only address one of Systems, vol. 21, no. 1, pp. 66–79, Feb 2013.
them. [11] E. I. Papageorgiou, “Learning algorithms for fuzzy cognitive maps – a
review study,” IEEE Transactions on Systems, Man, and Cybernetics,
Part C (Applications and Reviews), vol. 42, no. 2, pp. 150–163, Mar.
VI. C ONCLUSION 2012.
[12] C. Facchini, “Bridging the gap between theory and implementation in
The main scope of the work is to propose the design and the cognitive networks: developing reasoning in today’s networks,” Ph.D.
evaluation of a cognitive SDN architecture, employing FCMs dissertation, University of Trento, Dec. 2011.
as learning algorithm. First, by borrowing the model presented [13] J. Xie, F. R. Yu, T. Huang, R. Xie, J. Liu, and Y. Liu, “A survey
of machine learning techniques applied to software defined networking
in [12], the authors have discussed and proposed novel char- (sdn): Research issues and challenges,” IEEE Communications Surveys
acteristics required by FCMs to enable their deployment in Tutorials, pp. 1–1, 2018.
the context of cognitive networks. Of particular interest, the [14] B. P. R. Killi, E. A. Reddy, and S. V. Rao, “Cooperative game theory
based network partitioning for controller placement in sdn,” in 2018
authors have proposed the interpretation of continuous edge 10th International Conference on Communication Systems Networks
values as ’signed’ probabilities to adapt optimally FCMs to (COMSNETS), Jan. 2018, pp. 105–112.
the communication networks’ environment and relationships [15] H. Kuang, Y. Qiu, R. Li, and X. Liu, “A hierarchical k-means algorithm
for controller placement in sdn-based wan architecture,” in 2018 10th
among the related concepts. An efficient method for FCMs’ International Conference on Measuring Technology and Mechatronics
training has also been defined. Next, the authors has showed Automation (ICMTMA), Feb. 2018, pp. 263–267.

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
23

[16] M. Aslan and A. Matrawy, “A clustering-based consistency adaptation [38] ONOS. Onos overview white paper. [Online]. Avail-
strategy for distributed sdn controllers,” in 2018 4th IEEE Conference on able: http://onosproject.org/wp-content/uploads/2014/11/Whitepaper-
Network Softwarization and Workshops (NetSoft), Jun. 2018, pp. 441– ONOS-final.pdf
448. [39] ——, “Open network operating system (ONOS) wiki,” Sep. 2017.
[17] T. Kohonen, “Essentials of the self-organizing map,” Neu- [Online]. Available: https://wiki.onosproject.org/
ral Networks, vol. 37, pp. 52 – 65, 2013, twenty- [40] Mininet. Mininet – an instant virtual network on your laptop (or other
fifth Anniversay Commemorative Issue. [Online]. Available: pc). [Online]. Available: http://mininet.org/
http://www.sciencedirect.com/science/article/pii/S0893608012002596 [41] L. Yanjun, L. Xiaobo, and Y. Osamu, “Traffic engineering framework
[18] R. S. Sutton and A. G. Barto, The Problem. MITP, 1998. [Online]. with machine learning based meta-layer in software-defined networks,”
Available: https://ieeexplore.ieee.org/document/6282959 in 2014 4th IEEE International Conference on Network Infrastructure
[19] Y. Li, “Deep reinforcement learning: An overview,” and Digital Content, Sep. 2014, pp. 121–125.
CoRR, vol. abs/1701.07274, 2017. [Online]. Available: [42] C. Chen-xiao and X. Ya-bin, “Research on load balance method in
http://arxiv.org/abs/1701.07274 SDN,” International Journal of Grid and Distributed Computing, vol. 9,
[20] K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, pp. 25–36, 01 2016.
“Deep reinforcement learning: A brief survey,” IEEE Signal Processing [43] A. Azzouni, R. Boutaba, and G. Pujolle, “Neuroute: Predictive dynamic
Magazine, vol. 34, no. 6, pp. 26–38, Nov. 2017. routing for software-defined networks,” CoRR, vol. abs/1709.06002,
[21] F. Francois and E. Gelenbe, “Optimizing secure SDN-enabled inter- 2017. [Online]. Available: http://arxiv.org/abs/1709.06002
data centre overlay networks through cognitive routing,” in 2016 IEEE [44] D. D. Testa, M. Danieletto, G. M. D. Nunzio, and M. Zorzi, “Esti-
24th International Symposium on Modeling, Analysis and Simulation of mating the number of receiving nodes in 802.11 networks via machine
Computer and Telecommunication Systems (MASCOTS), Sep. 2016, pp. learning techniques,” in 2016 IEEE Global Communications Conference
283–288. (GLOBECOM), Dec. 2016, pp. 1–7.
[22] S. Sendra, A. Rego, J. Lloret, J. M. Jimenez, and O. Romero, “Includ- [45] J. Bendriss, I. G. B. Yahia, and D. Zeghlache, “Forecasting and anticipat-
ing artificial intelligence in a routing protocol using software defined ing SLO breaches in programmable networks,” in 2017 20th Conference
networks,” in 2017 IEEE International Conference on Communications on Innovations in Clouds, Internet and Networks (ICIN), Mar. 2017, pp.
Workshops (ICC Workshops), May 2017, pp. 670–674. 127–134.
[23] G. Stampa, M. Arias, D. Sanchez-Charles, V. Muntés-Mulero, and
A. Cabellos, “A deep-reinforcement learning approach for software-
defined networking routing optimization,” CoRR, vol. abs/1709.07080,
2017. [Online]. Available: http://arxiv.org/abs/1709.07080
[24] C. Yu, J. Lan, Z. Guo, and Y. Hu, “DROM: Optimizing the routing
in software-defined networks with deep reinforcement learning,” IEEE
Access, vol. 6, pp. 64 533–64 539, 2018.
[25] K. K. Budhraja, A. Malvankar, M. Bahrami, C. Kundu, A. Kundu, and
M. Singhal, “Risk-based packet routing for privacy and compliance-
preserving SDN,” in 2017 IEEE 10th International Conference on Cloud Giovanni Baggio received his B.Sc. degree in Elec-
Computing (CLOUD), Jun. 2017, pp. 761–765. tronics and Telecommunications Engineering and his
[26] R. Haw, M. G. R. Alam, and C. S. Hong, “A context-aware content M.Sc. degree in Telecommunications Engineering
delivery framework for QoS in mobile cloud,” in The 16th Asia-Pacific from University of Trento (Italy) in 2012 and 2015
Network Operations and Management Symposium, Sep. 2014, pp. 1–6. respectively. He is currently employed as Expert Re-
[27] M. A. Salahuddin, A. Al-Fuqaha, and M. Guizani, “Software-defined search Engineer in the research unit "WiN" at FBK
networking for RSU clouds in support of the internet of vehicles,” IEEE CREATE-NET. His current and past activities have
Internet of Things Journal, vol. 2, no. 2, pp. 133–144, Apr. 2015. spanned over multiple topics including Software-
[28] S. Ranadheera, S. Maghsudi, and E. Hossain, “Mobile edge Defined Networks, Network Functions Virtualiza-
computation offloading using game theory and reinforcement tion, Service Orchestration, Wi-Fi and LTE/5G re-
learning,” CoRR, vol. abs/1711.09012, 2017. [Online]. Available: lated technologies.
http://arxiv.org/abs/1711.09012
[29] S. D’Oro, L. Galluccio, S. Palazzo, and G. Schembra, “A game theoretic
approach for distributed resource allocation and orchestration of soft-
warized networks,” IEEE Journal on Selected Areas in Communications,
vol. 35, no. 3, pp. 721–735, Mar. 2017.
[30] X. Huang, T. Yuan, G. Qiao, and Y. Ren, “Deep reinforcement learning
for multimedia traffic control in software defined networking,” IEEE
Network, vol. 32, no. 6, pp. 35–41, Nov. 2018.
[31] P. V. Klaine, M. A. Imran, O. Onireti, and R. D. Souza, “A survey of ma- Riccardo Bassoli (riccardobassoli.com) received his
chine learning techniques applied to self-organizing cellular networks,” B.Sc. and M.Sc. degrees in Telecommunications
IEEE Communications Surveys Tutorials, vol. 19, no. 4, pp. 2392–2431, Engineering from University of Modena and Reggio
Fourthquarter 2017. Emilia (Italy) in 2008 and 2010 respectively. Next,
[32] Z.-Q. Liu, “Causation, bayesian networks, and cognitive maps,” Zi- he received his Ph.D. degree from 5G Innovation
donghua Xuebao/Acta Automatica Sinica, vol. 27, 12 2000. Centre (5GIC) at University of Surrey (UK), in
[33] D. Fletcher, D. Nguyen, and K. Cios, “Autonomous synthesis of fuzzy 2016, with his theses entitled ’Network Coding for
cognitive maps from observational data: Preliminaries,” in 2005 IEEE Efficient Vertical Handovers’. He is currently post-
Aerospace Conference, Mar. 2005, pp. 1–9. doctoral researcher at Department of Information
[34] A. K. Tsadiras, “Comparing the inference capabilities of binary, trivalent Engineering and Computer Science (DISI), at Uni-
and sigmoid fuzzy cognitive maps,” Information Sciences, vol. 178, versity of Trento (Italy). He was a Marie Curie Early
no. 20, pp. 3880 – 3894, 2008, special Issue on Industrial Applications Stage Researcher in the GREENET project, funded by European Commission.
of Neural Networks. During that period he worked at Instituto de Telecomunicações (Portugal) and
[35] T. R. Newman, B. A. Barker, A. M. Wyglinski, A. Agah, J. B. Evans, and he was visiting researcher at Airbus Defence and Space (France). Now, he
G. Minden, “Cognitive engine implementation for wireless multicarrier is actively involved in Dynamic Architecture based on UAVs Monitoring for
transceivers,” Wireless Communications and Mobile Computing, vol. 7, border Security and Safety (DAVOSS) project, funded by NATO. His main
pp. 1129 – 1142, 11 2007. research interests include theoretical aspects of software-defined networking,
[36] J. A. Dickerson and B. Kosko, “Virtual worlds as fuzzy cognitive maps,” network function virtualisation, network slicing and autonomic networks.
in Proceedings of IEEE Virtual Reality Annual International Symposium,
Sep. 1993, pp. 471–477.
[37] F. Hu, Q. Hao, and K. Bao, “A survey on software-defined network
and openflow: From concept to implementation,” IEEE Communications
Surveys Tutorials, vol. 16, no. 4, pp. 2181–2206, Fourthquarter 2014.

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
24

Fabrizio Granelli is Associate Professor at the


Dept. of Information Engineering and Computer Sci-
ence (DISI) of the University of Trento (Italy). From
2012 to 2014, he was Italian Master School Coor-
dinator in the framework of the European Institute
of Innovation and Technology ICT Labs Consortium.
He was Delegate for Education at DISI in 2015-2016
and he is currently member of the Executive Com-
mittee of the Trentino Wireless and Optical Testbed
Lab. He was IEEE ComSoc Distinguished Lecturer
for 2012-15 and IEEE ComSoc Director for Online
Content in 2016-17. Prof. Granelli is IEEE ComSoc Director for Educational
Services for 2018-19 and coordinator of the research and didactical activities
on computer networks within the degree in Telecommunications Engineering.
He was advisor of more than 80 B.Sc. and M.Sc. theses and 8 Ph.D. theses.
He is author or co-author of more than 200 papers published in international
journals, books and conferences in networking, with particular reference to
performance modelling, cross-layering, wireless networks, cognitive radios
and networks, green networking and smart grid communications.

2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

You might also like