Cognitive Software-Defined Networking Using Fuzzy Cognitive Maps
Cognitive Software-Defined Networking Using Fuzzy Cognitive Maps
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
1
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
2
often they did not clearly justify the reason for the choice knowledge of the network status. In fact, by considering
among those techniques [6]. In the last decade, several publi- the taxonomy of ML-based SDN [13], ML approaches
cations have investigated the deployment of various machine generally employ so called training datasets, i.e. human
learning (ML) algorithms to enhance the limited management supervision.
capabilities of legacy SDN. Nevertheless, in 2011 [8], the • the design of an actual and effective architecture, where
authors of this paper were the first to propose a suitable the proposed enhanced FCMs are integrated into SDN-
reasoning formalism for cognitive networks based on Fuzzy based networks. That is followed by an actual validation
Cognitive Maps (FCMs), which can be used to perform causal of this proposed system on an emulated testbed, therefore
reasoning. The innovative aspect lied in the adoption of such proposing one of the first complete implementations of
a tool to explicitly exploit cause-effect relationships among FCM-based cognitive networking to date.
variables in the protocol stack of network nodes. The choice of The rest of the paper is organised as follows. Subsection
FCMs was based on the idea that variables at different layers of I-A justifies the choice of FCMs for SDN-based cognitive
the protocol stack share cause-effect relationships. Moreover, networks, by showing their advantages in respect of other
FCMs allow the presence of loops, which are present in several ML algorithms. Section II describes the mathematical nota-
aspects of networking (see, for example, closed-loop operation tion and problem statement, as originally proposed in [12].
of flow and congestion control in TCP, automatic rate fallback Furthermore, it briefly presents the Hebbian-based algorithm
in WiFi, etc.). used for learning. Next, Section III fully discusses the novel
Fuzzy cognitive maps [9] have significantly attracted the enhanced model for Hebbian-based FCMs, which improves
attention of the research community in the last fifteen years. the theoretical framework previously developed in [12]. Our
The main limitations of FCMs, which have been identified, are theoretical solutions improve the correctness/performance of
[10]: edges’ weights are linear, they cannot represent logical the algorithm towards efficient deployment in real cognitive
operators between ingoing nodes, they cannot model multi- networks. Section IV deals with the design of a cognitive
meaning environments, they do not include multi-state con- software-defined networking architecture based on our pro-
cepts, they cannot handle more than one relationship between posed FCMs. This section considers architectural analysis
nodes, they are symmetric or monotonic (that does not happen and evaluation of overhead and latency introduced in SDN
for various real causal relations), and their first order dynamics paradigm by our enhanced Hebbian-based FCMs. Finally,
cannot handle randomness associated with complex domains. Section V provides results to verify the effectiveness of our
Learning algorithms for FCMs can be organised into three FCM-based SDN in two exemplar virtualised networks, which
main families [11]: Hebbian-based, population-based and hy- have been implemented in Mininet environment.
brid approaches. The first type uses the basis of experts’
knowledge to lead FCMs to converge into an acceptable
solution. The second employs historical data without any A. Related Works and Motivation
expert’s intervention. The third is based on both experts’ input Regarding unsupervised learning and SDN, the exhaustive
and historical data. The first detailed implemented architecture survey about ML-base SDN [13] identifies two main ap-
for FCM-based cognitive networks presented in [8] and [12] proaches in the literature: k-Means SDN and self-organising
applied a Hebbian-based learning algorithm. These works pro- map (SOM) SDN. The former is a popular method to classify
vided a framework to design FCMs for cognitive networks by unlabelled data, so it make it useful both for classical and QoS-
overcoming the above limitations, which could have negatively aware traffic classification in SDN-based networks. It is mainly
affected their deployment in the context of communications limited to clustering problem solving and it is computationally
networks. However, the authors of [8], [12] have not clearly expensive, in particular for large maps with big amount of
defined a criterion for the FCM-based algorithm to detect when training data. Recently, k-means have also been applied to
the inference process reaches a satisfying solution. Basically, SDN to solve controller placement problem [14], [15], [16],
those works provided evidence of the potential of FCMs for which is an important issue in presence of multiple SDN
cognitive networking, without achieving a general solution in controllers. The latter [17] is also principally employed to
order to predict when the system gets its final status. Still, they solve clustering problems: its data mapping is easier to be
represent a relevant milestone toward an actual architecture understood and capable to handle big datasets, while it is also
able to support cognitive networking. computationally expansive mainly for large training datasets.
The contribution of this article includes: Side by side, reinforcement learning (RL) [18] and deep
• the definition of an enhanced theoretical model for FCMs reinforcement learning (DRL) [19], [20] are additional ML
applied to wired networks. This model capitalises on pre- paradigms, in which an ’unsupervised agent’ interacts with its
vious results in [8], [12] to improve FCMs towards their environment to learn the best action to perform in order to
effective deployment in cognitive networks. In particular, maximise its long-term reward.
the article focuses on some critical aspects affecting clas- Moreover, the taxonomy of works about ML applied to
sical FCMs’ inference process, which – according to the SDN reveals that ML algorithms are very often limited to
best of authors’ knowledge – have not been investigated specific aspects of SDN network management such as traffic
yet in the scientific literature. classification, Quality-of-Service (QoS), routing optimisation,
• the theoretical and practical proof that our enhanced resource management or security: there has been no approach
Hebbian-based FCM can work effectively without a-priori yet, which has tried to use unsupervised ML/RL to realise a
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
3
sort of SDN-based autonomic network (i.e. an SDN network convergence of belief propagation algorithms [12]. On the
that manages itself without human intervention). other hand, the FCM inference procedure is also effective to
Regarding routing optimisation, the authors of [21] designed solve problems involving no loops. In particular, inference is
a centralised Cognitive Routing Engine (CRE), based on guaranteed to converge to either a fixed point or a limit cycle,
Random Neural Networks with Reinforcement Learning, was provided that concepts take their values in any finite discrete
to find the optimal network paths. In 2017, [22] applied RL set [34].
to improve the performance of routing protocols in SDN. Next, when neural networks are employed to model dy-
The same year, [23] developed a DRL agent to optimise namic systems, the obtained solution does not necessarily
routing towards a decrease in network latency. Next, [24] reflect the actual relationships among its system variables [35].
designed DRL paradigm for routing optimisation in SDN- On the other hand, the edges of an FCM faithfully represent
based environments. Reference [25] proposed solution based those relations and are, therefore, more appropriate to analyse
on k-means to optimise routing protocol in SDN. both direct and indirect cross-layer interactions [33], [12].
In the context of resource management, article [26] designed Further advantage is the possibility of exchanging/merging
a content delivery framework based on RL and SDN to select multiple FCMs, resembling operations people do when they
the optimal protocol. Next, [27] deployed RL to minimise the exchange their opinions [12]. This aspect has its roots in
long-term reconfiguration cost of roadside unit cloud network the primary purpose for which FCMs were created, i.e. to
based on SDN. In 2017, [28] applied RL and game theory for allow experts to represent their causal knowledge about some
activation of servers in mobile edge computing based on SDN. situation. Different people may have different opinions about
In parallel, [29] applied RL and game theory for resource the same matter, and may encode differently their beliefs,
allocation and management in distributed environment based hence drawing conflicting FCMs. Merging helps to smooth
on SDN. Recently, [30] applied DRL for multimedia traffic (possibly divergent) beliefs and biases, thereby reducing the
control in SDN in order to achieve high quality of experience. possibility of biased reasoning. Moreover, weights can be
Given the above context, this article is the first one to design employed to give more or less credit to each FCM. Finally,
in detail an FCM-based SDN system for unsupervised virtual the ’composed FCM’ can contain potentially non-overlapping
network management while providing a quantitative study of FCMs, thus, enabling the exchange of knowledge in case the
its performances. Another pivotal characteristic of this work domain of knowledge of cognitive entities is different, which
is the scope of ML paradigm: the aim of our FCM-based undoubtedly is an advantageous feature when dealing with
controller is not to optimise single aspects of networking but to uncertain scenarios such as communication networks.
guarantee efficient and effective self management of the SDN- The main drawback of legacy FCMs is the automatic
based network. Thus, the cognitive controller has to control synthesis of the maps: FCMs were not originally designed for
and to manage various aspects of networking to guarantee being constructed starting from observational data but they
good routing management while enhancing QoS provisioning were initially devised in the social science field as a tool
via effective management of resources. to help experts to express their beliefs about a given matter
The choice of FCMs comes from a major aspect in the area [33]. For this reason, self-synthesis of FCMs is hard, mostly
of unsupervised autonomic networks: human monitoring. In because, for non-humans, cause/effect relationships between
fact, humans have to be able to monitor and to understand variables are generally more complex to detect than simple
easily how the cognitive controller thinks and acts: that to correlations [33].
prevent unwanted bad events. One of the main advantages of Finally, the results obtained by our designed and imple-
FCM is exactly the clear and easy-to-understand representation mented FCM-based SDN keep comparable delays with legacy
of knowledge [9], which does not happen with solutions SDN Open Network Operating System (ONOS), thus the
based on k-Means, SOMs and RL. This is important since reasoning and acting procedures are performed almost trans-
in unsupervised systems humans have to be able to control parently. Moreover, our proposed cognitive SDN system can
how the unsupervised system thinks and make decisions. achieve 100% accuracy in average thus overcoming all the
Furthermore, while our design of FCMs is mainly focused legacy ML-based SDN solutions [13].
on resource/routing management and QoS optimisation (in this
article), the FCM-based controller can easily be employed to II. P RELIMINARIES AND BACKGROUND
consider and to manage other aspects of networking by adding
further variables to the FCM (e.g. variables referred to actions A. Software-defined Networking
and quality). Software-defined networking is a virtualisation paradigm,
Fuzzy Cognitive Maps have more advantages than other which permits a software-based control of data-paths and
reasoning techniques such as neural networks, Bayesian and routing strategies of networks. This technology aims at de-
Markov networks. Unlike both Bayesian and Markov net- taching control and data plane of communication networks.
works, the inference procedure of FCMs is less complex since The central entity of SDN virtual architecture is the SDN
it only involves vector-by-matrix multiplications and thresh- controller, which updates flow tables and policies at so called
olding operations [31]. Moreover, FCMs are more effective SDN switches. Moreover, an application layer is responsible
to represent causality loops in a problem; for these kind of to handle different networking applications such as control of
problems Bayesian networks cannot be applied [32], [33]. In data paths, user authentication and management of mobility.
Markov networks, the presence of loops cannot guarantee the Among various protocols proposed for controller-switches
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
4
Fig. 2. Structure of SDN virtual network (left side) and structure of SDN
architecture (left side), which specifies the logic interfaces of the architecture.
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
5
to their relationship with Quality-of-Service (QoS) metrics, Moreover, let fÛi,t j label the variation of the edge’s value,
to environmental characteristics in which the cognitive entity connecting concept Ci with concept C j at time instant t. Then,
is, and to the set of actions that a protocol can perform. the differential Hebbian law states that
Mathematically, these concepts respectively form the triple of
fÛi,t j = − fi,t−1
j + Ci C j
Ût Ût (1)
vectors (q, e, a), which represents the system state vector s.
The objective of the FCM is the convergence to a solution where the derivatives represent changes in two generic con-
state s∗ = (q, e, a∗ ), achieved by finding a vector a∗ such that cepts and the result of their product reveals the correlation
the constraints expressed by q are satisfied before environmen- among them.
tal conditions e change. Since vectors q and e are known, the Expression (1) shows that a variation of concept Ci occurs
search space of the algorithm is limited to the elements of a: before the variation of concept C j , the other way around if
that avoids the problem to be NP-hard. C j was the first concept changing then the edge to be updated
However, in order to map the concepts in the most effective would be f j,i . The negative value of the edge at time t − 1 is
way, it is fundamental to choose the right domain. Concepts included in the right-hand side of equation (1) to prevent that
can be expressed either by discrete sets or continuous sets. In a spurious simultaneous variation impacts indefinitely.
particular, it is normally better to avoid continuous sets be- Next, the edge at step t is computed as the value it had at
cause they can results into chaotic behaviour [12]. Continuous time step t − 1 plus the variation:
ranges of values can be converted into discrete sets by ap-
plying a thresholding procedure. However, finding the optimal j + fi, j = Ci C j
fi,t j = fi,t−1 Ût Ût Ût (2)
threshold for an application may not be straightforward. Thanks to the term − fi,t−1
j in expression (1), the value on the
On the other hand, the size of a discrete set can be chosen edge can be set to zero when one of the concepts do not
according to the needs. For example, concepts that convey change, thereby indicating that the previous causal relationship
an ON-OFF relationship can use the set {0, 1}, and the ones no longer holds.
requiring to invert causal relationship can use the set {−1, 1}. As a result, the learning process does not update the
After defining the concepts and their domains, it is impor- knowledge considering only the current causality relationships
tant to select a learning algorithm for FCMs. The follow- degree but also keeping in consideration the past value.
ing subsection provides some details on one of the existing Depending on the application, such update may be preferred
learning algorithms. That should be considered as a concrete to be more responsive or vice versa, the responsiveness of
example of how learning can be implemented, but other the algorithm can be modified by introducing a parameter
methods are available and can be used without architectural η in the previous formula known either as learning rate or
differences with respect to the solution presented in this paper. as decreasing learning coefficient. In other words, it can be
seen as a smoothing parameter, capable to avoid significant
D. Differential Hebbian Learning Algorithm oscillation in the values of the edges (the amplitude of these
variations depends on the magnitude of the coefficient value).
In our work we borrow the problem statement proposed
Moreover, parameter η is generally defined in the set (0, 1]:
in [12]: hence, we apply a Hebbian-based algorithm called
values close to zero result in a slowly changing FCM, while
differential Hebbian learning (DHL) algorithm [36]. The main
values close to the unity produce a highly responsive FCM.
drawback of this method is that the formula updates weights
Finally, the learning formula becomes
between each pair of concepts without considering the influ-
ence of other concepts. j + η(− fi, j + Ci C j )
fi,t j = fi,t−1 t−1 Ût Ût (3)
In order to explain how DHL algorithm works, let’s consider
a simple FCM with two concepts named A and B. If both A It is worth notice that if η is set to one meaning that the past
and B change their values from 0 to 1, then it can be inferred value on the edge is not considered when it is updated, then
that the positive variation of one concept has caused a positive edges can assume values in the discrete set (−1, 0, 1) while a
variation of the other. Similarly, if both A and B change their different η results in continuous range of values.
values from 1 to 0, it can be inferred that the negative variation
of A has produced a negative variation of B. In both cases the III. E NHANCED F UZZY C OGNITIVE M APS FOR C OGNITIVE
sign of the variation is positive for these two concepts because N ETWORKS
a positive causal relationship is present. The work in [12] proposed FCMs based on discrete domains
Let’s now suppose that A changes its value to 1 and B, instead of continuous ones and introduced thresholding in
which before the variation of A had the value set to 1, goes to form of pre-processing in order to map values into discrete
0. In this case, there is a negative causality relationship because domains. This section demonstrates the potential of using
a variation of the first concept led to the opposite variation of continuous ranges of values, by avoiding thresholding. In
the second, thus the edge connecting the two concepts is now particular, we show the significant advantages obtained by
negative (for simplicity let’s assume the system is memoryless, interpreting continuous values in the range [−1, 1] as ’directed’
meaning that the algorithm does not consider the previous (signed) probabilities.
states). Moreover, we also show it is actually possible to acquire
Formally, let Ci and C j denote two generic concepts, and let knowledge not only related to the causality degrees among
CÛi and CÛ j denote their variations over time (time derivatives). nodes but also to create, modify and delete persisting causal
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
6
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
7
(a) (a)
(b)
Fig. 6. (a) Simple FCM with negative causality (b) Description of FCM
inference process. Example that shows the system cannot come to a decision
if thresholding is applied.
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
8
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
9
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
10
(a)
(a)
(b)
(b)
Fig. 10. Example to show the advantage of combining different paths (a)
Example of an FCM, which has unequal lengths (b) Description of FCM
inference process, which shows that values are erroneously added.
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
11
paths) then, during the inference process, inference on by interpreting weights of edges as probabilities, there is the
the shorter path has to wait until the one on the longest need to propose other solutions to face loops.
has arrived. First, it would be possible to set a limit in order to determine
• When n concepts lead to the same node, their contribu- whether the inference process has to continue through the
tions to the final value have to be treated like probabilities, chain of concepts or it has to stop: in particular when the
i.e. they cannot be simply added up. Hence, the shared absolute value of the probability reported along a chain of
target node is inferred with a probability, calculated as causal relationships is smaller than the threshold, the latter
the sum of independent random variables. action has to be performed. Moreover, the value of this limit
The first condition imposes that – regardless of the concept should be close to zero to allow the system to explore possible
class type – when there is more than one arrow entering a solutions when weak knowledge is present on the edges, thus
node, a decision for the final value of this variable only has to resulting in small absolute values.
be computed when all the contributions – coming from other The idea behind this first approach comes from the fact
nodes – are available. The second condition instead is related that loops in FCMs make the inference algorithm spinning
to how these contributions have to be combined together: as continuously across the involved nodes and this process never
discussed before, the values propagating on the edges during stops because a steady solution cannot be found. It is worth
the algorithm execution are treated like probabilities, so that notice that cycle after cycle, the output of the inference process
the sum is not allowed because it could possibly lead to values converges to an interval close to zero: by definition, if the
out of [0, 1). causality values on the edges are bounded by the continuous
As an example, let’s consider two causality chains, reaching set [0, 1), then the absolute value of the output will decrease
a certain target node with probability 0.7 and 0.8 respectively. iteration after iteration. Given that, the introduction of the
As a consequence, their sum is 1.5, which it may be approxi- proposed limit allows the algorithm to indirectly realise when
mated to 1.0. Then, a causal relationship, represented with the it is stuck in a loop.
unity value, reflects a fully deterministic connection, which is The second proposed approach (the one applied in the rest
a nonsense if we consider that the ones closer to the sources of this work) is based on a preprocessing operation, performed
are not. before the inference process starts. During this stage, the
So, the proposed solution overcomes this issue by treating cognitive engine runs an algorithm similar to the Spanning tree
probabilities as they are and by performing the addition protocol: that can detect and remove cognition loops from the
accordingly. The sum of independent random-variable formula FCM. This idea is supported by the consideration that, in the
fits the bill and provides an output in the continuous set [0, 1). matrix representing the knowledge, loops do not supply any
It is worth notice that every contribution coming from each additional useful information to the final result of the inference
causality chain positively contributes to the final outcome. It process, thus they can be removed with confidence.
is also worth to notice that before combining the contributions, According to this method, there is no need for a limit, which
the positive and negative causal relationships are separated and may not be straightforward to be set to an appropriate value.
only at the end are summed up. In particular, the proposed This second approach should be preferred to the first because
expression is only used among contribution with the same sign. it avoids useless iteration across the same concepts forming
Formally, given n variables the loop. However, the drawback of this method is a higher
implementation complexity and an additional overhead in the
y[n] = y[n − 1] + x[n] − y[n − 1] · x[n]
(
inference process.
(9)
y[0] = 0 Figure 12 depicts the FCM before and after the execution of
the procedure of loop removal. The adjacency matrix referred
Referring to examples in Figure 11 and Figure 10, the score to Figure 12(a) is
variable is now meaningless once there is only one quality
variable because all the contributions, coming from different 0 −0.1 0 0 0 0 0
© ª
paths, are combined together, thus resulting in a single score 0 0 0.6 0 0 0.4 0 ®
®
value. It is possible to see that the outcomes of the two FCMs
0 0 0 0 0 0.9 0 ®
®
are now equal as expected: in the inference process log, it
0 0 0.5 0 0 0 0
®
® (10)
is clearly visible that the implementation waits for the first ®
branch on the left to advance and, once they are ready to
0 0 0 0 0 1 0 ®
®
®
enter together in the quality variable, the algorithms proceed 0 0 0 0 0 0 0.6 ®
®
with the sum of their probabilities. 0 0 0 0 0 0 0
« ¬
Especially, Figure 12(b) reveals that Concept 3 has been re-
D. Causality Loops Removal moved from the collections of the considered nodes during the
If FCMs perform thresholding and map values in the dis- inference process because it does not provide any information
crete range [0, 1], the inference process can produce matrices since it is part of a loop.
with redundant vectors. Then, once the algorithm recognises The modifications applied so far on the FCMs’ inference
that the current outcome has already been found in the past process resulted in an augmented complexity of the FCM-
steps it just stops. Nevertheless, by avoiding thresholds and based system. The first proposed approach can search faster
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
12
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
13
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
14
(a)
Fig. 16. Reference architecture for Cognitive SDN based on FCMs.
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
15
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
16
Fig. 18. Time needed for the FCM updating process by considering the size
of the map. The boxplots are related to the time spent for the update for three Fig. 20. Time needed for the FCM reasoning process by considering the
FCM sizes, from a small FCM with only 10 variables to a more complex one number of causal relationships, due to the small FCM size (10) there is no
with 100. On each box, the bottom and top blue edges of each box indicate relevant differences in the computational time.
the 25th and 75th percentiles respectively, the central red mark indicates the
median, and the two horizontal bars comprising the box are the minimum and
maximum values of the range of samples.
Fig. 19. At the top of the table, the maximum amount of cognitive
relationships is reported for each FCM size. Then, these values are used
as references for comparing them with three percentage levels of FCM link
filling.
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
17
Fig. 22. With the greatest reasonable FCM size (100) the differences in
the processing times depending on the number of causal relationships have
increased with respect to Figure 20 and Figure 21. However, it is worth
to notice that these have never exceeded the half millisecond, proving the
suitability of the proposed approach.
Fig. 23. Time spent by the inference process, considering both different FCM
sizes and different amounts of learnt relationships.
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
18
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
19
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
20
(a)
Fig. 30. Scenario to test the performance of cognitive SDN, where users’
connections with switches have been changed.
(b)
Fig. 29. Fuzzy Cognitive Maps referred to scenario in Figure 28 (a) This FCM
underlines that system starts with no a-priori knowledge (b) The Cognitive
Engine learnt the effectiveness of reducing bandwidth.
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
21
Fig. 32. Graph which highlights the three main moments of the experiment. For clarity, some values have been slightly changed in order to allow a better
representation: in reality the response of the Cognitive Engine to an event is much quicker than what seems to be from the graph. The two actions available
at the Cognitive Engine are reported by the red and black plots, while the blue one reports the congestion state. Next, the light-green and light-blue lines are
related to the two environmental variables. For all of them, the ’on’ and ’off’ values are related to the active and non-active variable status respectively. The
feedback to the cognitive system is retrieved each time: after a change of status of some action variables, there are other environmental or quality variables
that change their status.
is not present any more. was previously ineffective – has now proved to be able to
At this point, a new causal relationship is created between reduce the congestion, thus a new causal relationship between
variables Reduce Bandwidth and Congestion, and it is enforced this action variable and Congestion is created and enforced
each time the controller acquires new evidences about the (see Figure 31(b)).
effectiveness of the action (see Figure 29(b)). Furthermore, a In order to let the controller strengthen the new belief, we
new relationship between Reduce Bandwidth and High Load have manually forced the users to move towards a single
is discovered: here, the latter is triggered when there is a switch, resulting in new congestion states being triggered.
congestion because it is in this period that the bandwidth on The final result not only contains information between the
the gateway is saturated. After few minutes, the bandwidth on currently working action variable for solving the congestion
the gateway is increased to 100 Mb/s, resulting in a network state, but also includes certain causal relationships among
condition where congestion disappears. As depicted in Figure different environmental variables. Figure 32 shows the overall
32, the Cognitive Engine do not act any more, but the acquired experiment variables statuses over time. Especially, it is possi-
knowledge is preserved. ble to recognise three main moments: first, Reduce Bandwidth
At this point of the experiment, we manually move the is the effective action, second, the latter action is not working
users among the switches, resulting in the network topology any more due to the changed network topology, third the
of Figure 30: the variable Congestion is triggered due to the Cognitive Engine realises that the new effective action was
bottleneck in the link s4–s5 (h9 remains the receiver), and Move Users.
the Cognitive Engine – which is completely unaware of the During the entire test duration, the Cognitive Engine has
new change in the infrastructure – has to react. It first tries to mixed the learning and testing stages: at the beginning, for the
enable the Reduce Bandwidth action variable, that FCM has first 40 seconds, due to the complete absence of a-priori knowl-
proven to be effective in the past, but, since the hosts have edge the cognitive process has learnt by test and trials: the only
already reached the lowest guaranteed throughput, this action effective action to solve the congestion state was to reduce the
cannot solve the issue. bandwidth. In the next period, this belief has been enforced by
After few trials, during which the causal relationship be- the successive congestion states that raised and for which the
tween Reduce Bandwidth and Congestion variables is weak- Cognitive Engine has managed to successfully handle them
ened, the controller returns in its original situation (Figure (i.e. the congestion variable has switched from ’on’ to ’off’
31(a)), where it has to discover which action is more suitable state after the action had triggered). This phase continues until
for bringing the quality variable in a safe state. Then, the the 330th second, after which the network scenario completely
controller discovers that Move Users action variable – which changes: due to deep variations of users’ displacement, the
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
22
R EFERENCES
reduction of bandwidth becomes not effective. Once again, [1] R. W. Thomas, D. H. Friend, L. A. DaSilva, and A. B. Mackenzie,
the Cognitive Engine has to learn that by acting (period 330- “Cognitive networks: adaptation and learning to achieve end-to-end
520 seconds) during which the congestion state remains to performance objectives,” IEEE Communications Magazine, vol. 44,
no. 12, pp. 51–57, Dec. 2006.
an active level. Finally, the-out-of-dated knowledge model is [2] B. S. Manoj, R. R. Rao, and M. Zorzi, “CogNet: a cognitive complete
demolished and the Cognitive Engine is ready to experiment knowledge network system,” IEEE Wireless Communications, vol. 15,
with new action patterns; it tries by moving the users which, no. 6, pp. 81–88, Dec. 2008.
[3] R. Mijumbi, J. Serrat, J. L. Gorricho, N. Bouten, F. D. Turck, and
differently from the initial scenario, is now an effective action R. Boutaba, “Network function virtualization: State-of-the-art and re-
(see from the 520th seconds till the end). Finally, excluding the search challenges,” IEEE Communications Surveys Tutorials, vol. 18,
periods during which knowledge had to be built either for lack no. 1, pp. 236–262, Firstquarter 2016.
[4] B. A. A. Nunes, M. Mendonca, X. N. Nguyen, K. Obraczka, and
or out-of-dated knowledge, the system has shown an accuracy T. Turletti, “A survey of software-defined networking: Past, present,
of 100% (7 out of 7 congestion status in the first phase and 5 and future of programmable networks,” IEEE Communications Surveys
out of 5 in the second one). Tutorials, vol. 16, no. 3, pp. 1617–1634, Third 2014.
[5] M. Agiwal, A. Roy, and N. Saxena, “Next generation 5g wireless
Figure 33 depicts a comparison of upper bound of accuracy networks: A comprehensive survey,” IEEE Communications Surveys
of legacy ML-based SDN and the proposed FCM-based SDN. Tutorials, vol. 18, no. 3, pp. 1617–1655, thirdquarter 2016.
In the context of routing optimisation, [41], [42], [43] calcu- [6] C. Fortuna and M. Mohorcic, “Trends in the development of
communication networks: Cognitive networks,” Computer Networks,
lated the accuracy of neural networks applied to SDN. In the vol. 53, no. 9, pp. 1354 – 1376, 2009. [Online]. Available:
context of resource management, [44] calculated the accuracy http://www.sciencedirect.com/science/article/pii/S1389128609000085
of some supervised ML algorithms (Naive Bayes, Linear [7] J. R. P. Mähönen, M. Petrova and M. Wellens, “Cognitive wireless
networks: Your network just became a teenager,” in 25th Conference
and radial support vector machine and k-neural networks) on Computer Communications, 2006.
and [45] evaluated the accuracy of unsupervised ML and [8] C. Facchini, F. Granelli, and N. da Fonseca, “Cognitive service-oriented
RL applied to SDN. As it is possible to see, the proposed infrastructures,” Journal of Internet Engineering, vol. 4, no. 1, pp. 269–
278, 2010.
algorithm can reach the maximum of accuracy overpassing the [9] B. Kosko, “Fuzzy cognitive maps,” Int. J. Man-Mach. Stud.,
legacy ML-based SDN paradigms employing either supervised vol. 24, no. 1, pp. 65–75, Jan. 1986. [Online]. Available:
ML, unsupervised ML or RL. Furthermore, the novel FCM- http://dx.doi.org/10.1016/S0020-7373(86)80040-2
based SDN handles both routing optimisation and resource [10] E. I. Papageorgiou and J. L. Salmeron, “A review of fuzzy cognitive
maps research during the last decade,” IEEE Transactions on Fuzzy
management while other legacy solutions only address one of Systems, vol. 21, no. 1, pp. 66–79, Feb 2013.
them. [11] E. I. Papageorgiou, “Learning algorithms for fuzzy cognitive maps – a
review study,” IEEE Transactions on Systems, Man, and Cybernetics,
Part C (Applications and Reviews), vol. 42, no. 2, pp. 150–163, Mar.
VI. C ONCLUSION 2012.
[12] C. Facchini, “Bridging the gap between theory and implementation in
The main scope of the work is to propose the design and the cognitive networks: developing reasoning in today’s networks,” Ph.D.
evaluation of a cognitive SDN architecture, employing FCMs dissertation, University of Trento, Dec. 2011.
as learning algorithm. First, by borrowing the model presented [13] J. Xie, F. R. Yu, T. Huang, R. Xie, J. Liu, and Y. Liu, “A survey
of machine learning techniques applied to software defined networking
in [12], the authors have discussed and proposed novel char- (sdn): Research issues and challenges,” IEEE Communications Surveys
acteristics required by FCMs to enable their deployment in Tutorials, pp. 1–1, 2018.
the context of cognitive networks. Of particular interest, the [14] B. P. R. Killi, E. A. Reddy, and S. V. Rao, “Cooperative game theory
based network partitioning for controller placement in sdn,” in 2018
authors have proposed the interpretation of continuous edge 10th International Conference on Communication Systems Networks
values as ’signed’ probabilities to adapt optimally FCMs to (COMSNETS), Jan. 2018, pp. 105–112.
the communication networks’ environment and relationships [15] H. Kuang, Y. Qiu, R. Li, and X. Liu, “A hierarchical k-means algorithm
for controller placement in sdn-based wan architecture,” in 2018 10th
among the related concepts. An efficient method for FCMs’ International Conference on Measuring Technology and Mechatronics
training has also been defined. Next, the authors has showed Automation (ICMTMA), Feb. 2018, pp. 263–267.
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
23
[16] M. Aslan and A. Matrawy, “A clustering-based consistency adaptation [38] ONOS. Onos overview white paper. [Online]. Avail-
strategy for distributed sdn controllers,” in 2018 4th IEEE Conference on able: http://onosproject.org/wp-content/uploads/2014/11/Whitepaper-
Network Softwarization and Workshops (NetSoft), Jun. 2018, pp. 441– ONOS-final.pdf
448. [39] ——, “Open network operating system (ONOS) wiki,” Sep. 2017.
[17] T. Kohonen, “Essentials of the self-organizing map,” Neu- [Online]. Available: https://wiki.onosproject.org/
ral Networks, vol. 37, pp. 52 – 65, 2013, twenty- [40] Mininet. Mininet – an instant virtual network on your laptop (or other
fifth Anniversay Commemorative Issue. [Online]. Available: pc). [Online]. Available: http://mininet.org/
http://www.sciencedirect.com/science/article/pii/S0893608012002596 [41] L. Yanjun, L. Xiaobo, and Y. Osamu, “Traffic engineering framework
[18] R. S. Sutton and A. G. Barto, The Problem. MITP, 1998. [Online]. with machine learning based meta-layer in software-defined networks,”
Available: https://ieeexplore.ieee.org/document/6282959 in 2014 4th IEEE International Conference on Network Infrastructure
[19] Y. Li, “Deep reinforcement learning: An overview,” and Digital Content, Sep. 2014, pp. 121–125.
CoRR, vol. abs/1701.07274, 2017. [Online]. Available: [42] C. Chen-xiao and X. Ya-bin, “Research on load balance method in
http://arxiv.org/abs/1701.07274 SDN,” International Journal of Grid and Distributed Computing, vol. 9,
[20] K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, pp. 25–36, 01 2016.
“Deep reinforcement learning: A brief survey,” IEEE Signal Processing [43] A. Azzouni, R. Boutaba, and G. Pujolle, “Neuroute: Predictive dynamic
Magazine, vol. 34, no. 6, pp. 26–38, Nov. 2017. routing for software-defined networks,” CoRR, vol. abs/1709.06002,
[21] F. Francois and E. Gelenbe, “Optimizing secure SDN-enabled inter- 2017. [Online]. Available: http://arxiv.org/abs/1709.06002
data centre overlay networks through cognitive routing,” in 2016 IEEE [44] D. D. Testa, M. Danieletto, G. M. D. Nunzio, and M. Zorzi, “Esti-
24th International Symposium on Modeling, Analysis and Simulation of mating the number of receiving nodes in 802.11 networks via machine
Computer and Telecommunication Systems (MASCOTS), Sep. 2016, pp. learning techniques,” in 2016 IEEE Global Communications Conference
283–288. (GLOBECOM), Dec. 2016, pp. 1–7.
[22] S. Sendra, A. Rego, J. Lloret, J. M. Jimenez, and O. Romero, “Includ- [45] J. Bendriss, I. G. B. Yahia, and D. Zeghlache, “Forecasting and anticipat-
ing artificial intelligence in a routing protocol using software defined ing SLO breaches in programmable networks,” in 2017 20th Conference
networks,” in 2017 IEEE International Conference on Communications on Innovations in Clouds, Internet and Networks (ICIN), Mar. 2017, pp.
Workshops (ICC Workshops), May 2017, pp. 670–674. 127–134.
[23] G. Stampa, M. Arias, D. Sanchez-Charles, V. Muntés-Mulero, and
A. Cabellos, “A deep-reinforcement learning approach for software-
defined networking routing optimization,” CoRR, vol. abs/1709.07080,
2017. [Online]. Available: http://arxiv.org/abs/1709.07080
[24] C. Yu, J. Lan, Z. Guo, and Y. Hu, “DROM: Optimizing the routing
in software-defined networks with deep reinforcement learning,” IEEE
Access, vol. 6, pp. 64 533–64 539, 2018.
[25] K. K. Budhraja, A. Malvankar, M. Bahrami, C. Kundu, A. Kundu, and
M. Singhal, “Risk-based packet routing for privacy and compliance-
preserving SDN,” in 2017 IEEE 10th International Conference on Cloud Giovanni Baggio received his B.Sc. degree in Elec-
Computing (CLOUD), Jun. 2017, pp. 761–765. tronics and Telecommunications Engineering and his
[26] R. Haw, M. G. R. Alam, and C. S. Hong, “A context-aware content M.Sc. degree in Telecommunications Engineering
delivery framework for QoS in mobile cloud,” in The 16th Asia-Pacific from University of Trento (Italy) in 2012 and 2015
Network Operations and Management Symposium, Sep. 2014, pp. 1–6. respectively. He is currently employed as Expert Re-
[27] M. A. Salahuddin, A. Al-Fuqaha, and M. Guizani, “Software-defined search Engineer in the research unit "WiN" at FBK
networking for RSU clouds in support of the internet of vehicles,” IEEE CREATE-NET. His current and past activities have
Internet of Things Journal, vol. 2, no. 2, pp. 133–144, Apr. 2015. spanned over multiple topics including Software-
[28] S. Ranadheera, S. Maghsudi, and E. Hossain, “Mobile edge Defined Networks, Network Functions Virtualiza-
computation offloading using game theory and reinforcement tion, Service Orchestration, Wi-Fi and LTE/5G re-
learning,” CoRR, vol. abs/1711.09012, 2017. [Online]. Available: lated technologies.
http://arxiv.org/abs/1711.09012
[29] S. D’Oro, L. Galluccio, S. Palazzo, and G. Schembra, “A game theoretic
approach for distributed resource allocation and orchestration of soft-
warized networks,” IEEE Journal on Selected Areas in Communications,
vol. 35, no. 3, pp. 721–735, Mar. 2017.
[30] X. Huang, T. Yuan, G. Qiao, and Y. Ren, “Deep reinforcement learning
for multimedia traffic control in software defined networking,” IEEE
Network, vol. 32, no. 6, pp. 35–41, Nov. 2018.
[31] P. V. Klaine, M. A. Imran, O. Onireti, and R. D. Souza, “A survey of ma- Riccardo Bassoli (riccardobassoli.com) received his
chine learning techniques applied to self-organizing cellular networks,” B.Sc. and M.Sc. degrees in Telecommunications
IEEE Communications Surveys Tutorials, vol. 19, no. 4, pp. 2392–2431, Engineering from University of Modena and Reggio
Fourthquarter 2017. Emilia (Italy) in 2008 and 2010 respectively. Next,
[32] Z.-Q. Liu, “Causation, bayesian networks, and cognitive maps,” Zi- he received his Ph.D. degree from 5G Innovation
donghua Xuebao/Acta Automatica Sinica, vol. 27, 12 2000. Centre (5GIC) at University of Surrey (UK), in
[33] D. Fletcher, D. Nguyen, and K. Cios, “Autonomous synthesis of fuzzy 2016, with his theses entitled ’Network Coding for
cognitive maps from observational data: Preliminaries,” in 2005 IEEE Efficient Vertical Handovers’. He is currently post-
Aerospace Conference, Mar. 2005, pp. 1–9. doctoral researcher at Department of Information
[34] A. K. Tsadiras, “Comparing the inference capabilities of binary, trivalent Engineering and Computer Science (DISI), at Uni-
and sigmoid fuzzy cognitive maps,” Information Sciences, vol. 178, versity of Trento (Italy). He was a Marie Curie Early
no. 20, pp. 3880 – 3894, 2008, special Issue on Industrial Applications Stage Researcher in the GREENET project, funded by European Commission.
of Neural Networks. During that period he worked at Instituto de Telecomunicações (Portugal) and
[35] T. R. Newman, B. A. Barker, A. M. Wyglinski, A. Agah, J. B. Evans, and he was visiting researcher at Airbus Defence and Space (France). Now, he
G. Minden, “Cognitive engine implementation for wireless multicarrier is actively involved in Dynamic Architecture based on UAVs Monitoring for
transceivers,” Wireless Communications and Mobile Computing, vol. 7, border Security and Safety (DAVOSS) project, funded by NATO. His main
pp. 1129 – 1142, 11 2007. research interests include theoretical aspects of software-defined networking,
[36] J. A. Dickerson and B. Kosko, “Virtual worlds as fuzzy cognitive maps,” network function virtualisation, network slicing and autonomic networks.
in Proceedings of IEEE Virtual Reality Annual International Symposium,
Sep. 1993, pp. 471–477.
[37] F. Hu, Q. Hao, and K. Bao, “A survey on software-defined network
and openflow: From concept to implementation,” IEEE Communications
Surveys Tutorials, vol. 16, no. 4, pp. 2181–2206, Fourthquarter 2014.
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCCN.2019.2920593, IEEE
Transactions on Cognitive Communications and Networking
24
2332-7731 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.