Ref.

Ares(2016)2069377 - 30/04/2016

30-04-2016

Deliverable D13.1
Research and Study Network Technologies
– White Paper

Deliverable D13.1

Contractual Date: 30-04-2016

Actual Date: 30-04-2016
Grant Agreement No.: 691567
Activity: 13/JRA1>
Task Item: Task 1
Nature of Deliverable: R (Report)
Dissemination Level: PU (Public)
Lead Partner: NORDUnet
Document Code: GN4-1-16-7D602
Authors: H. Wessing (NORDUnet); K. Bozorgebrahimi (NORDUnet); J. Vojtech (CESNET); P. Skoda
(CESNET); S. Naegele-Jackson (DFN); A. Metz (DFN); M. N. Petersen (NORDUnet); L. Velasco
(RedIRIS); N. Sambo (CNIT/GARR); V. Olifer (Jisc); R. Lund (NORDUnet); P. Rydlichowski (PSNC)

© GEANT Limited on behalf of the GN4 Phase 1 project.

The research leading to these results has received funding from the European Union’s Horizon 2020 research and
innovation programme under Grant Agreement No. 691567 (GN4-1).

Abstract
This deliverable reports on the work carried out by GN4-1 Joint Research Activity 1 Future Network Topologies, Task 1
Current and Future Network Technologies to investigate the trends and technologies in optical transport networks and
how these can be managed to help deliver the concept of zero-touch connectivity. It covers increasing utilisation of the
photonic layer; spectral sharing and alien waves; frequency and time distribution; and network dynamicity.
Table of Contents
Executive Summary 1

1 Introduction 3

2 Increasing Utilisation of the Photonic Layer 5

2.1 Growing Capacity Trends 5
2.1.1 New Formats and Emerging Modulations 6
2.1.2 Extending the Optical Spectrum Beyond the C Band 8
2.1.3 High-Bitrate Standardisation and Beyond 100G 11
2.1.4 NREN Partner Experiments 14
2.1.5 Conclusions 16
2.2 Flexible Optical Networks 17
2.2.1 Results of GN3plus REACTION 17
2.2.2 Recent Developments on FON 19
2.2.3 Transponder Interoperability 20
2.2.4 FON Vendor Available Implementations 23
2.2.5 Conclusions 24

3 Spectral Sharing and Alien Waves 25

3.1 Advantages of Alien Waves and Shared Spectrum 25
3.2 Modelling Tool Development 26
3.2.1 Introduction to the Tool 26
3.2.2 Guidelines on How to Use the Tool 27
3.2.3 Basic Principles, Capabilities and Assumptions of the Tool 28
3.2.4 Extensions to the Tool 29
3.3 Best Practices from Other Projects 30
3.3.1 General Best Practice 30
3.3.2 SUNET Alien Wave-Based Network 30
3.3.3 UNINETT 100G Alien Wavelength Cisco–Juniper Interoperability
Test 31
3.3.4 Operator Experiences with Alien Waves 34
3.4 Experimental Scenarios and Key Results 34
3.4.1 Experiments in NORDUnet 35
3.5 Conclusions 37

4 Frequency and Time Distribution 38

4.1 Distribution on the Lowest Layer 38
4.1.1 Physical Layer 39

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602
Contents

4.1.2 Experimental Results 42

4.1.3 Conclusions: Management of Waves for Frequency Distribution 47
4.2 Time Distribution in Layer 3 47
4.2.1 PTP Validation over MPLS Tunnels 48
4.2.2 Long-Term PTP Tests over Standard Internet Connection 50
4.2.3 Conclusions and Recommendations 52

5 Network Dynamicity 53
5.1 SDN and Transport SDN Review 53
5.1.1 Developments in Transport SDN 55
5.1.2 ONF View on Transport Extensions 58
5.1.3 ABNO and Its Role as Orchestrator 59
5.1.4 ABNO-Based Integrated Demonstrator 61
5.2 The NORDUnet UNINETT Use Case 65
5.2.1 The Problem 65
5.2.2 The SDN Solution 67
5.2.3 Simulation of the NORDUnet Use Case in the ONOS/Mininet
Environment 70
5.3 Conclusions 72

6 Conclusions 74

Appendix A MOMoT Modelling Tool 77

A.1 Inputs 77
A.2 Outputs 77
A.3 Buttons 78

Appendix B Transport Extensions in the ONF OpenFlow Protocol 81

B.1 Background 81
B.2 Flow Match Fields and Actions 82
B.2.1 Examples 84
B.3 Port Description Extensions 86
B.4 Port Extensions for Adjacency Discovery 89

References 90

Glossary 96

Deliverable D13.1
Research and Study Network Technologies – iii
White Paper
Document Code: GN4-1-16-7D602
Contents

Table of Figures
Figure 2.1: PAM-2-NRZ and PAM-4: baseband signalling and eye diagrams [5] 6
Figure 2.2: Fibre optic transmission bands for standard SMF fibre 10
Figure 2.3: Suggested new technology development in 802.3bs [15] 13
Figure 2.4: Transponder evolution path 18
Figure 2.5: Optimum sub-carriers launch power for 1 Tb/s 5-SC-DP-16-QAM and
400 Gb/s 4SC-DP-QPSK super-channels for sub-carrier power pre-emphasis (SCPP)
method 19
Figure 2.6: Multi-partner pan-European testbed 21
Figure 2.7: BER performance 22
Figure 3.1: Modelling tool interface and output messages including guard band
information 28
Figure 3.2: Optical network setup 31
Figure 3.3: UNINETT Cisco–Juniper interoperability test scenario 32
Figure 3.4: Pre-FEC bit error rate on Juniper router 33
Figure 3.5: Post-FEC bit errors on Juniper router 33
Figure 3.6: Test scenario between Copenhagen and Hamburg 35
Figure 3.7: 300 km test scenario between Copenhagen and Gothenburg 36
Figure 3.8: The figure compares the pre-FEC BER curves of the field trial (lab)
measurements and three curves produced by the MOMoT tool 37
Figure 4.1: Bi-directional channel in legacy DWDM transmission system 40
Figure 4.2: Path UFE (Praha) – VUGKT (Pecny): concatenation of passive lambdas 40
Figure 4.3: Detailed scheme of a span between Reims and Nancy 41
Figure 4.4: Praha–Brno line, C- and L-band systems deployed, 306 km 41
Figure 4.5: Two uni-directional lambdas used for time transmission between Praha–
Vienna, 2 x 550 km 42
Figure 4.6: Fractional frequency instability versus averaging time of REFIMEVE+ network
branch 43
Figure 4.7: Topology of Paris–Braunschweig link with frequency comparison in
Strasbourg 43
Figure 4.8: 25-day comparison of Paris and Braunschweig atomic clocks in Strasbourg 44
Figure 4.9: Frequency instability during two phases of experimentation 45
Figure 4.10: Optical and GPS time transfer Praha–Vienna, 550 km 46
Figure 4.11: Time transfer Praha–Brno 46
Figure 4.12: Test setup 1 in AES over WAN tests 48
Figure 4.13: Test setup 2 in AES over WAN tests 49
Figure 4.14: Offset and path delay on 7 October 2015 between Erlangen and Prague 50
Figure 4.15: Offset and path delay on 2 November 2015 between Erlangen and Prague 51

Deliverable D13.1
Research and Study Network Technologies – iv
White Paper
Document Code: GN4-1-16-7D602
Contents

Figure 4.16: Offset and path delay on 4 November 2015 between Erlangen and Prague 51
Figure 5.1: Transport networks lifecycle with (a) dynamicity and (b) in-operation
planning 54
Figure 5.2: SDN architecture 55
Figure 5.3: ABNO architecture 60
Figure 5.4: ABNO-based network orchestrator 61
Figure 5.5: Optical domain demonstrator architecture 62
Figure 5.6: Example of LSP setup 65
Figure 5.7: Packet and optical layers of NORDUnet 66
Figure 5.8: A global view of the NORDUnet layerless topology 69
Figure 5.9: The primary route between R-TUG and a UNINETT router 71
Figure 5.10: The backup route between R-TUG and a UNINETT router 72
Figure A.1: Modelling tool interface 79
Figure A.2: Modelling tool interface and output messages including guard band
information 80

Table of Tables
Table 2.1: Overview of different optical bands and properties 11
Table 2.2: Performance results 23
Table 2.3: Tunable parameters in today’s vendors’ multi-rate transponder
implementations 23
Table 4.1: Slave disconnected and reconnected to reference signal 50
Table 5.1: Conventional networking and SDN comparison 56

Deliverable D13.1
Research and Study Network Technologies – v
White Paper
Document Code: GN4-1-16-7D602
Executive Summary
This white paper reports on the work carried out by GN4-1 Joint Research Activity 1 Future Network
Topologies, Task 1 Current and Future Network Technologies (JRA1 T1) to investigate the trends and
technologies in optical transport networks and how these can be controlled and managed to help
deliver the concept of zero-touch connectivity.

In the future, the National Research and Education Networks (NRENs) will face a number of
requirements from user, technology, cost and management perspectives. One way to address some
of these requirements is by introducing the concept of zero-touch connectivity (ZTC): the user
should have access to a given service with a given quality wherever he is, and whatever the
requested data is. While this vision provides simplicity to the user, behind the scenes it requires
massive technological advances, interoperability and dense integration.

In order for the zero-touch vision to become a reality, certain strict requirements are imposed on
the underlying infrastructure in terms of both high capacity and flexible use. Technologies and
approaches for increasing the capacity of the photonic layer investigated by JRA1 T1 include new
modulation schemes, such as four-level pulse-amplitude modulation (PAM-4); space-division
multiplexing (SDM) technologies, such as multi-core fibres (MCF) and few-mode fibres (FMF); and
extending the optical spectrum beyond the C band. The Task has also reviewed the work being done
by the standardisation bodies with regard to high bitrate standardisation. Importantly, from the
point of view of guiding the deployment of the next generation of NREN infrastructure, the Task has
conducted experiments to evaluate these developments in the NREN environment. Based on the
successful results of its testing, PSNC has now established and configured two operational 400G links.

With regard to flexible use, the Task has reviewed the results obtained by the GN3plus Open Call
project REACTION, which evaluated flexible optical networks (FON); investigated the latest FON
developments and experimental research, including transponder interoperability; and surveyed the
FON functionalities currently available or planned. A summary of all the results is provided here.

Combining optical waves operated by different administrative entities over the same fibre and the
same optical transport network (alien waves and spectral sharing) can lead to significant cost savings.
However, these approaches also present a number of challenges from a technical, organisational
and procedural point of view. The multi-domain optical modelling tool (MOMoT), originally
developed during a GN3plus Open Call project and further developed by JRA1 T1 during GN4-1,
helps to meet the technical challenges by predicting the impact of injecting alien waves into a host
system. The results have been validated by tests in a production NREN network. Further studies and
experiments – including by JRA1 T1 participants – have helped identify prerequisites and best

Deliverable D13.1
Research and Study Network Technologies –
White Paper 1
Document Code: GN4-1-16-7D602
Executive Summary

practice. The reliability and integrity of alien waves have been sufficiently proven for NRENs to have
transitioned AW services into production for both client and own use.

Another obvious advantage of the ZTC vision is the ability to seamlessly provide connectivity for
high-demanding services. This includes services with extremely high demands for clock stability,
which therefore need efficient and accurate synchronisation and distribution of time in several levels
and layers, and ultra-stable frequency. Joint experiments with synchronisation between peers into
different NREN domains have taken place, together with experiments that investigated by how
much it is possible to adjust the precision time protocol while still maintaining the necessary quality
in terms of jitter and delay. This work is needed to realise the ZTC vision of being able to seamlessly
provide links with given requirements for delay, jitter and synchronisation functions.

Given the above functionalities, the Task has investigated the overall control and orchestration of
transport network resources, addressing this challenge both theoretically and experimentally.
Software-defined networking (SDN) as a control mechanism for transport networks has gained
attention lately, and JRA1 T1 has reviewed the research, including demonstrations of new extensions
to the OpenFlow protocol. In practical terms, the Task has carried out tests to evaluate different
orchestration mechanisms, including application-based network operations (ABNO), and an open
network operating system (ONOS) has been investigated as part of a specific NREN use case to
provide efficient – and more optimal – resource utilisation in restoration processes.

This deliverable presents the results of comprehensive research and experimentation by JRA1 T1
into the current trends and technologies in the physical and logical domains of optical transport
networks, with the aim of supporting the ZTC vision that will help NRENs to meet the high-
demanding requirements they face from user, technology, cost and management perspectives.
While challenges and the need for further testing exist in many of the areas analysed, the work has
identified several promising solutions and, of particular importance, has assessed these in the NREN
environment. This has led to the deployment of enhanced operational services in some NRENs, and
it is hoped that the findings will similarly guide other NRENs in the development of their next-
generation infrastructure.

Deliverable D13.1
Research and Study Network Technologies – 2
White Paper
Document Code: GN4-1-16-7D602
1 Introduction
In the future, the National Research and Education Networks (NRENs) will face a number of
requirements from user, technology, cost and management perspectives. One way to address some
of these requirements is by introducing the concept of zero-touch connectivity (ZTC), which both
meets user needs and simplifies the processes for the network operator. In short, the user should
have access to a given service with a given quality wherever he is, and whatever the requested data
is. This vision provides the user with a much simpler interface to the network. However, behind the
scenes it requires massive technological advances, interoperability and dense integration.

GN4-1 Joint Research Activity 1 Future Network Topologies, Task 1 Current and Future Network
Technologies (JRA1 T1) addresses a number of these technical challenges and will provide some of
the building blocks needed to build automated services and guide the deployment of the next
generation of NREN infrastructures. This also means that network borders and domains are
becoming dispersed from the user perspective, forcing GÉANT, NREN and commercial service
providers to cooperate in a federated manner in order to provide the ZTC.

To fulfil this ZTC vision, it is important to understand and evaluate the current trends and
technologies in the physical and logical domains. This deliverable reports on the work carried out in
JRA1 Task 1, which investigates the technology trends in optical transport networks and how these
can be controlled and managed according to ZTC.

The first and – from the user’s point of view – often neglected requirement for ZTC is suitable
capacity in the underlying infrastructure. Without this hard physical footprint, not even the best
orchestration tool in the world can satisfy the users’ needs. The Task has focused significant effort
on investigating viable paths to increased capacity, including the physical functionality to utilise this
capacity optimally. In this deliverable, the focus is on the latest trends for increasing the capacity of
the photonic layer and efficient use within and outside traditional network boundaries and domains.
In particular, combining optical waves operated by different administrative entities over the same
fibre and the same optical transport network can lead to significant cost savings. However, such
alien wave approaches also present a number of challenges from a technical and political point of
view. In this deliverable, some of the most important technical challenges are considered, and a
modelling and estimation tool to predict the impact of injecting alien waves into a host system,
which Task 1 has further developed and validated during GN4-1, is described. Integration of such a
tool into the provisioning process can save hours and days of time spent on commissioning new
channels and as such can add value to the vision of zero-touch connectivity.

Another obvious advantage of the ZTC vision is the ability to seamlessly provide connectivity for
high-demanding services. This includes services with extremely high demands for clock stability,

Deliverable D13.1
Research and Study Network Technologies –
White Paper 3
Document Code: GN4-1-16-7D602
Introduction

which therefore need efficient and accurate synchronisation and distribution of time in several levels
and layers. For example, for atomic clocks, physical layer frequency synchronisation is needed. This
is addressed in Section 4 and, among others, joint experiments conducted with synchronisation
between peers into different NREN domains are described. In addition, applications such as audio
and video with strict quality requirements need a synchronised clock, which for these applications
can be achieved with functionality on layer 2 and layer 3. This deliverable reports on experiments
that investigated by how much it is possible to adjust the precision time protocol while still
maintaining the necessary quality in terms of jitter and delay. This work is needed to realise the ZTC
vision of being able to seamlessly provide links with given requirements for delay, jitter and
synchronisation functions.

Given the above functionalities it makes sense to discuss the overall control and orchestration of the
transport network resources. In its work, the Task has addressed this challenge theoretically and
experimentally. Software-defined networking (SDN) as a control mechanism for transport networks
has gained attention lately and in this deliverable the research is reviewed and demonstrations of
new extensions to the OpenFlow protocol are described. Also, tests to evaluate different
orchestration mechanisms are reported. Application-based network operations (ABNO) is
investigated as one approach to orchestrate network (in IT) resources, and an open network
operating system (ONOS) is investigated as part of a specific use case to provide efficient – and more
optimal – resource utilisation in restoration processes. This work is based on some experiments in
the NORDUnet network, which was used for the use case.

The deliverable is organised as follows:

 Section 2 discusses the technological trends for increasing the capacity of each symbol,
spectrum, core and fibre. The technologies for enabling flexible optical networks are also
addressed.
 Section 3 addresses alien waves and spectral sharing. This includes reporting on further
development of the multi-domain optical modelling tool (MOMoT) and validation of this.
Best practices with regard to alien waves from other projects and NREN partners are also
included.
 Section 4 considers time and frequency synchronisation and distribution. It is mainly divided
into synchronisation in the physical layer and using protocols in layer 2 and 3.
 Section 5 addresses control and orchestration of resources using SDN, ABNO and ONOS.
 Section 6 aims to draw together the conclusions and recommendations.

In addition, further details about the multi-domain optical modelling tool (MOMoT) are provided in
Appendix A, and details of the transport extensions in the OpenFlow protocol are provided in
Appendix B.

Deliverable D13.1
Research and Study Network Technologies – 4
White Paper
Document Code: GN4-1-16-7D602
2 Increasing Utilisation of the Photonic Layer
In order for the zero-touch vision to become a reality, certain strict requirements are imposed on
the underlying infrastructure for both the transport network and the photonic layer. These
requirements can be grouped into high capacity and flexible use. Although the end user will rarely
interact directly with these layers, the importance of the two for capacity and flexibility remains.
Without the necessary available capacity, not even the best orchestration mechanism can deliver
anything useful to the end user. Furthermore, flexibility mechanisms such as flexible optical
networks (FONs) are needed to utilise the provided capacity satisfactorily.

With these requirements in mind, this section addresses the capacity-enabling technologies, which
the Task foresees as being needed in the future. Section 2.1 presents a survey of some of the most
important trends for increasing the capacity of the fibre, which focus on the core, the spectrum and
the bit density. The study includes reports on some partner experiments, which validated the
usefulness of the techniques in an NREN context.

Enablers for providing flexibility were evaluated in detail in a previous project: the GN3plus Open
Call project Research and Experimental Assessment of Control plane archiTectures for In-Operation
flexgrid Network re-optimisation (REACTION). In Section 2.2, the ideas and latest trends are further
investigated and in particular the development and interoperability of sliceable bandwidth variable
transceivers are studied, based on research by the project partners, in addition to a survey on the
available vendor equipment related to FON.

2.1 Growing Capacity Trends

In this section the technologies that enable higher core capacity are reviewed. The emerging
modulation schemes allow more bits per symbol and four-level pulse-amplitude modulation (PAM-4)
is investigated in particular as it is simpler and thus more cost-efficient than its quadrature phase-
shift keying (QPSK) and quadrature amplitude modulation (QAM) counterparts. Another way of
increasing bandwidth is by providing more modes per core, which is why space-division multiplexing
(SDM) techniques are investigated in an NREN context, discussed in Section 2.1.1.2. The next step,
more cores per fibre and broader spectrum utilisation is also evaluated (Section 2.1.2). Section 2.1.3
addresses the standardisation trends in the major bodies for high-speed communication and, finally,
Section 2.1.4 reports on participating NREN partners’ experimental deployments, some of which
have been upgraded for the operational networks.

Deliverable D13.1
Research and Study Network Technologies – 5
White Paper
Document Code: GN4-1-16-7D602
Increasing Utilisation of the Photonic Layer

2.1.1 New Formats and Emerging Modulations

Pulse-Amplitude Modulation (PAM)

Different modulation formats have been discussed in both GN3plus Deliverable D12.3 (DJ1.1.1)
Future Network Architectures and GN3 Deliverable DJ1.2.1: State-of-the-Art Photonic Switching
Technologies [1; 2].

High-order modulation formats are today widely used on the line side of the transmissions line, such
as dual polarisation QPSK (DP-QPSK) in 100G 50 GHz dense wavelength division multiplexing (DWDM)
implementations, but non return to zero (NRZ) is still used on the client side. As a result of the
emerging higher bitrate transmissions channel and requirement for even higher bitrate client-side
interfaces, the IEEE 802.3bs 400G group, with the objective of defining a 400 Gb/s physical layer
specification, decided to use PAM-4 as the new modulation format for the 400G physical layer with
reach above 2 km. Using a higher-order modulation format reduces the requirement for higher
bandwidth devices. PAM-4 uses four-level amplitude detection to double the transmission capacity
with the same bandwidth optical devices, but at the expense of lower signal-to-noise (SNR)
performance when compared to NRZ [3; 4].

The traditional NRZ modulation consisting of 1s and 0s may be considered as PAM-2 (pulse-
amplitude modulation, two-level). Figure 2.1 shows baseband signalling and eye diagrams for PAM-
2-NRZ and PAM-4 [5]

Figure 2.1: PAM-2-NRZ and PAM-4: baseband signalling and eye diagrams [5]

The point to note here is the emerging of higher-order modulation techniques on the client side. In
the first stage, the simplest form of it is PAM-4.

Space-Division Multiplexing (SDM)

The term space-division multiplexing (SDM) is usually used with brand-new technologies, namely
multi-core fibres (MCF) and few-mode fibres (FMF). Multi-core and few-mode fibres have been a
favourite research topic for a few years now. Amplification techniques are essential for practical

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 6
Increasing Utilisation of the Photonic Layer

deployment of space-division multiplexing and such amplifiers are already available. The European
Conference on Optical Communications (ECOC) and the Optical Fibre Communication Conference
and Exposition (OFC) are the main events presenting the latest achievements in this area. It is an
open question as to whether NRENs want to participate in such SDM experiments, because it seems
that 100 Gb/s and 200 G/s systems are sufficient from the capacity point of view, and that there is a
greater current need for NRENs to address topics such as software-defined networking and zero-
touch networking.

Few-Mode Fibres
When starting to talk about few-mode fibres (FMF), reference may be made to the old and rather
well-known multi-mode fibres (MMF). Such fibres support hundreds of “modes” and have been
considered as the cheaper and not-so-excellent fellow of single-mode fibres. MMF do transmit
different modes, but all of these modes represent a single piece of information and cannot be
distinguished until some new compensating techniques, based on digital signal processing (DSP),
such as multiple input multiple output (MIMO), are deployed. MIMO techniques are known from
wireless and other systems, but their deployment in optical networking is not so common (high
frequencies of light compared to radio frequencies). DSP techniques used for modern coherent
optical transmission systems have been known for many years and have been used in modems and
wireless. It was the terahertz area (i.e. very high speeds) of optical wavelengths that was the main
obstacle when realising DSP in silicon or other semiconductors.

However, MMF do support more than 100 spatial modes at the C band – with the same 125 micron
cladding diameter, which is very important because standard fusion splicers, connectors and other
existing pieces of equipment can be used. Of course, MIMO must deal with the most detrimental
effect – crosstalk. Space-Division Multiplexing: The Future of Fiber-Optic Communications [6]
provides a detailed description of the advantages of this MMF-based SDM, including the use of
wavelength selective switches (WSSs) and erbium-doped fibre amplifiers (EDFA) and Raman
amplifiers.

The problems with MMF-SDM can be solved when just a few modes are transmitted via optical
fibres. Many useful observations can be found in Optical Amplifiers for Space Division Multiplexed
Transmission [7]. FMF usually support up to 15 modes, but this number is limited to 6 because in the
few-mode amplifiers available today, six-mode EDFAs are the highest mode-count EDFA practically
demonstrated. The first EDFA supporting three modes was demonstrated in 2011. From this it is
apparent that SDM is not a trivial area: adding just three spatial modes takes several years of intense
research, and such experiments are demonstrated in only a few excellent research centres around
the globe. The limiting factor is differential modal gain (DMG), which must be minimal. The current
situation is that 10 spatial modes can be supported with the help of optimisation of Er distribution in
the fibre. Different pumping techniques are used to achieve low DMG and both core-pumped and
cladding-pumped schemes have been proposed and investigated, with the latter considered to be
more promising.

Multi-Core Fibres
Multi-core fibres (MCF) are another promising direction of SDM, but are not without other problems.
Fibres with up to 30 cores are available, although matching amplification techniques are difficult to
find. Splicing and connecting such fibres are of course also much more challenging.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 7
Increasing Utilisation of the Photonic Layer

Nationwide research activities in Japan are summarised in PAM-4 Design Challenges and the
Implications on Test [4]. High-tech MCF are described, with attenuation of 0.181 dB/km and
crosstalk in the order of 90 dB/km has been demonstrated for 7-core fibres. Transmission distances
for 32 QAM signals are expected to reach 10,000 km. Few-mode techniques are coupled together
with multi-core fibres to achieve even higher spatial channel counts and 36-core and 3-mode fibres
have been demonstrated. Coupling of MCF into SMF are described and two methods – free-space
and fibre-bundled – are investigated. The latter method is considered to be more suited to cost-
effective production. Multi-core and few-mode amplifiers are mentioned and problems are similar
to ones described in Development of Space Division Multiplexing Technologies in Japan [8].

2.1.2 Extending the Optical Spectrum Beyond the C Band

Data transmission in L band (1570 nm – 1605 nm) represents mature technology. It is used in both
coarse wavelength division multiplexing (CWDM) and dense wavelength division multiplexing
(DWDM) multiplexes. It benefits from the same technologies as transmission in C band, especially
spectral suitability of EDFAs. It is a matter of fact that L-band EDFAs work on the tail of emission Er3+
characteristics and due to this lower absorption/emission effectivity they originally required 7–10
times longer doped fibres and high pump powers compared to C band. Long doped fibres suffered
from higher polarisation mode dispersion (PMD) and higher influence of non-linearity due to
increased four-wave mixing. This problem has been solved by introducing very heavy doped Er fibres,
reducing fibre length from 50–100 m to 12 m [9]. Slightly worse performance-to-cost ratio (worse
OSNR, more expensive doped fibre and more powerful pumps) of L band over C band transmission is
due to the fact that L-band transmission is used solely on G.653 fibres (they are not suitable for
DWDM in C band due to the presence of zero chromatic dispersion wavelength in this band) or in
combination with traditional C-band transmission. Raman amplification regains interest with the rise
of coherent transmission systems and is also available for L band [10]; due to slightly lower pump
attenuation it is even more effective than for C band. Practical experience has shown that such
necessary L-band parts as transceivers, passives or wavelength cross-connects are manufactured in
smaller volumes and therefore available with longer delivery times compared with C-band parts.

The last low-loss transmission band in standard single-mode fibre represents S band (1460 nm –
1530 nm). Unfortunately, due to the unavailability of mature amplification technology
complementary to EDFA technology, the S band is not used for DWDM transmission. The proposed
thulium-doped fibre amplifiers typically suffer from low efficiency, a high noise figure or the
necessity for environmentally sensitive fibres, e.g. fluoride based [11]. The potential of distributed
Raman amplification for S band is also limited due to high fibre attenuation at pump wavelength
around 1420 nm. Therefore this band is often used for CWDM or passive optical networks.

An alternative solution to improve the bandwidth capacity of optical transport systems is to go

further beyond the optical L-band window [12]. Using the 2000 nm range is being extensively
discussed between optical transport system vendors and research groups at universities as they
actually develop the key components such as amplifiers. Moreover, it is also being discussed
between optical transport system vendors and users – telco operators. Such a shift could result in
130 THz of total available bandwidth for the optical transmission system. The required optical
equipment, mainly the amplifiers that limit the transmission distance, is already available, namely
thulium (TDFA) and holmium (HDFA). The combined amplification window it is possible to achieve is
1650 nm – 2150 nm (TDFA – 34 THz; HDFA – 8.3 THz). The optical fibre technology is also already
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 8
Increasing Utilisation of the Photonic Layer

available. For example, the ZBLAN fibre has minimum attenuation of 0.65 dB/km at 2700 nm. Other
material combinations give minimum attenuation at around the 2000 nm range and the theoretical
loss of 0.03 dB/km is possible to achieve. High-speed silicon photodetectors for the 2000 nm range
already present the desired parameters specification [13]. It can be used in avalanche mode and
manufactured in 220 nm silicon-on-insulator (SOI) technology. Silicon photonics, 2D materials (such
as graphene and MoS2) and metamaterials could also further improve transceiver technology and
reduce the manufacturing costs, and are already available. One of the examples is low-loss all-optical
modulation in a graphene-coated fibre.

As can be seen, existing technologies can lower the manufacturing costs for the new type of
equipment and can significantly improve the available bandwidth for the optical transmission system.
The most promising aspect is to use technologies and manufacturing techniques for photonic
systems that heavily use experience from the silicon industry.

Figure 2.2 shows fibre optic transmission bands with attenuation values for typical SMF fibre. As
noted above, attenuation curves are different for each type of advanced fibre.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 9
Increasing Utilisation of the Photonic Layer

Figure 2.2: Fibre optic transmission bands for standard SMF fibre

An overview of the pros and cons of the different optical bands discussed is provided in Table 2.1.

Band Pros Cons

O, S  Inexpensive transponder technology  Short to mid range

 Best solution for FTTx systems;  Limited performance
performance/cost  Cannot be used for DWDM
C, L  Suitable for high-performance systems  Expensive transmitter technology
 Lowest attenuation for typical optical
fibres
 Can be used for DWDM

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 10
Increasing Utilisation of the Photonic Layer

Band Pros Cons

 Large potential for further
development
 Inexpensive amplifiers
U/XL and above  Potentially high bandwidth for high-  Currently not used
performance systems  Requires new fibres and amplifiers
 Expensive development

Table 2.1: Overview of different optical bands and properties

2.1.3 High-Bitrate Standardisation and Beyond 100G

Important progress in high-bitrate standardisation is delivered by three standardisation bodies. The
Institute of Electrical and Electronics Engineers Standards Association (IEEE-SA) focuses on LAN
network and Ethernet protocols development. The International Telecommunication Union
Telecommunication Standardisation Sector (ITU-T) is more involved in transport network protocols.
Finally, the Optical Internetworking Forum (OIF) builds on top of IEEE and ITU-T frameworks; OIF
involves vendors and interested parties to define details of a specific standard.

Institute of Electrical and Electronics Engineers Standards Association (IEEE-SA)

The IEEE 802.3 Working Group develops standards for Ethernet networks. IEEE 802.3 has several
efforts underway concerning specification of Ethernet operating at rates of 25 Gb/s, 50 Gb/s, (next
gen) 100 Gb/s, 200 Gb/s, and 400 Gb/s [14]. These activities are organised under different study
groups and task forces. The study groups develop the idea from the early stage and try to obtain
Project Authorisation Request (PAR) approval and define its objectives. PAR is the means by which
standards projects are started within the IEEE-SA. PARs define the scope, purpose, and contact
points for the new project. Task forces takes the idea from the approved PAR stage to when the
standard is approved. This process normally takes several years.

Currently there are a number of active study groups and task forces under IEEE 802.3 [14]. Those
related to the high bitrate standardisation include:

 IEEE P802.3bs 400 Gb/s Ethernet Task Force.

 IEEE P802.3by 25 Gb/s Ethernet Task Force.
 IEEE 802.3 50 Gb/s Ethernet Over a Single Lane Study Group.
 IEEE 802.3 Next Generation 100 Gb/s Ethernet & 200 Gb/s Ethernet Study Group.

802.3bs Task Force

The scope of the 802.3bs Task Force is to define Ethernet media access control (MAC) parameters,
physical layer specifications and management parameters for the transfer of Ethernet format frames
at 400 Gb/s. The Call for Interest was held in March 2013 and the first study group meeting was held
in May 2013. The study group set objectives to provide physical layer specifications that support link
distances of at least 100 m over MMF, at least 500 m over SMF, at least 2 km over SMF and at least

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 11
Increasing Utilisation of the Photonic Layer

10 km over SMF, and support optional 400 Gb/s attachment unit interfaces (AUIs) for chip-to-chip
and chip-to-module applications. Final approval of the standard is planned by the end of 2017.

There have been discussions regarding expanding the scope to apply to Ethernet format frames at
100 Gb/s and 200 Gb/s too.

The development of 50 Gb/s electrical I/O will enable the development of devices and systems
capable of supporting multiple Ethernet rates, which are based on integer multiples of 50 Gb/s, such
as 100 GbE, 200 GbE, and 400 GbE. The discussions have been on how the multi-lane 100G/200G
solutions might be rolled into the 802.3bs project.

50G Project
The primary goal of the 50G project is to choose solutions that provide the lowest cost, lowest
power and smallest size for 50G. It has been initiated by two study groups in order to study the two
(related) topics of an optimised single-lane project for 50 Gb/s Ethernet and the multiple-lane
variants of that.

NG 100/200 GbE Study Group, NGOATH

Progress to date for the IEEE 802.3 Next Generation 100 Gb/s Ethernet & 200 Gb/s Ethernet Study
Group [15; 16] includes:

 Some clear consensus around the various 50G single-lane objectives.

 Some clear consensus that some aspects of 200G (logic primarily, maybe AUI, maybe SMF)
are all potentially incremental work items that .3bs could take on.
 A lot of confusion around 100G objectives as people realise the challenges of the work due to
PMD choices or backwards compatibility issues.
 Some early thinking that some of the multi-lane PMDs that leverage these new 50G single-
lane work could be done with the same project as single-lane work (with 100G caveat from
above).

Changed Objectives
At the January 2016 interim meeting (a joint meeting of the 50G and NGOATH Study Groups) some
major changes to the objectives of the groups mentioned above (50G, NG 100/200 GbE and .3bs)
were suggested. The list of changes can be found on the group’s web pages at IEEE [17].

According to these changes, P802.3bs should support a MAC data rate of 200 Gb/s in addition to
400 Gb/s, and will provide physical layer specifications that support 200 Gb/s operation over:

 At least 2 km of SMF.
 At least 10 km of SMF.

P802bs support for 400 Gb/s operation over the media and distances below have not changed:

 At least 100 m of over MMF.

 At least 500 m of over SMF.
 At least 2 km of over SMF.
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 12
Increasing Utilisation of the Photonic Layer

 At least 10 km of over SMF.

As the single-mode specification of 200 Gb/s has moved to 8023.bs, the multi-mode specifications
will become part of a new task force that will also tackle 50 Gigabit Ethernet [18].

The new task force will also work with the objectives defined in [17], namely.

1. Define single-lane 50 Gb/s PHY for operation:

○ Over copper twin-axial cables.
○ Printed circuit board backplane.
○ MMF with lengths up to at least 100 m.
○ SMF with lengths up to at least 2 km.
○ SMF with lengths up to at least 10 km.
2. Define 200 Gb/s PHY for operation over:
○ Copper twin-axial cables.
○ Printed circuit board backplane.
○ MMF with lengths up to at least 100 m.
3. Define a two-lane 100 Gb/s PHY for operation over copper twin-axial cables.
4. Define a two-lane 100 Gb/s PHY for operation over a printed circuit board backplane.
5. Define a two-lane 100 Gb/s PHY for operation over MMF with lengths up to at least 100 m.

Figure 2.3: Suggested new technology development in 802.3bs [15]

International Telecommunication Union – Telecommunication Standardisation

Sector (ITU-T)
The ITU-T focuses on OTN standardisation (including G.709, G.798, G.872 and other
Recommendations). Work is underway on what is referred to as “Beyond 100G” OTN (B100G, for
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 13
Increasing Utilisation of the Photonic Layer

short), which is a more complex landscape than that of 100G where one modulation format
addressed a wide variety of applications over almost any reach (100G DP-QPSK in a 50 GHz
frequency slot was good up to about 4,000 km). There are emerging new clients above 100G
(200 GbE, 400 GbE and Flex Ethernet) that need to be carried, and, in general, the modulation
format used and the effective bitrate per wavelength vary according to reach (e.g. the same
hardware may support 100G DP-QPSK, 150G DP-8QAM, or 200G DP-16QAM, with the reach
decreasing from about 4,000 km to 1,000 km as the bitrate increases). Single wavelength 400G or
above is less seen, just because the reach tends to get too short to be interesting. Therefore, in
addition to having a variety of wavelength rates, another principle of the B100G work is that new
transport containers are defined that are distributed to multiple wavelengths or sub-carriers. What
is being worked on is a modular frame format that can be used to describe containers such as the
current ODUk in increments of 100G. This container may be mapped over a single wavelength or
distributed over several wavelengths.

While many of the signals carried over optical networks (possibly including multiple wavelengths in
aggregate) are multiples of 100G in size, some odd rates such as 150G exist. These will be addressed
from a management perspective by describing these signals in the 100G modular format but
indicating that a certain amount of the payload capacity (represented as a number of “tributary
slots”) is unavailable over a particular interface. Since these types of interfaces are single-vendor
line-side interfaces, standards will not specify a specific frame format for them, only the information
content that needs to be carried over these interfaces [19].

For client interfaces, just as an interface was defined for a client OTU4 that reused the 100 GbE
pluggable modules, B100G client interfaces are expected to be specified based on 400 GbE pluggable
modules and bonded 100 GbE pluggable modules.

Optical Internetworking Forum (OIF)

The OIF is focused on flexible Ethernet and flexible coherent DWDM frameworks. Flex Ethernet is a
method that supports non-link aggregation group (LAG) bonding of 100G Ethernet PHY to create
larger rate flows, and channelisation supporting multiple smaller rate flows (e.g. 10G, 25G) carried
over a group of bonded PHYs. One reason this is significant in the higher bitrate discussion is that it
was originally envisioned that the first client needing to be carried over OTN larger than 100G would
be 400 GbE, which is currently scheduled for completion at the end of 2017. Flex Ethernet may
provide >100G clients through the bonding of multiple 100G earlier than this [20]. Another relevant
OIF project is the Flex Coherent DWDM framework, which would ensure that the right components
are being specified to allow building of the kinds of interfaces that are emerging in OTN networks
(DSP-driven interfaces that can support different modulation formats with different effective
bitrates depending on reach) [21].

2.1.4 NREN Partner Experiments

PSNC
In June 2015 PSNC conducted 400G transmission tests in the PIONIER network. The tests were
prepared and run together with ADVA Optical Networking. The trial was conducted over the PIONIER
network, 385 km of fibre, reaching between two research and education centres in Poznań and

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 14
Increasing Utilisation of the Photonic Layer

Warsaw. The system transported the data continuously for 14 hours without any block errors. The
test system used software-defined optics to deliver flexible data rates and modulation. This means
the network can intelligently scale from 200 Gb/s to 300 Gb/s to 400 Gb/s and from QPSK to 8QAM
to 16QAM as required. All configuration and modulation options were tested and evaluated by the
PSNC team.

The 400 Gb/s trial was conducted using the operational PIONIER network resources. PIONIER is a
consortium of Polish research and education organisations and its network is operated domestically
by PSNC. For the purpose of the trial, the system was configured with two 200 Gb/s wavelengths
operating at 16QAM within an optical super-channel with spectral width of 100 GHz. The system also
featured reconfigurable optical add/drop multiplexers (ROADMs) and hybrid amplifiers. During the
14 hours of testing, the trial showed an error-free transmission with optical-signal-to-noise ratios of
approximately 20 dB.

During the 400G tests, results were presented live at the TERENA 2015 conference. Following the
tests, PSNC has established and configured an operational 400G link between the Poznań and
Warsaw nodes. PSNC has also established a second 400G link that is used for its own work and
projects.

CESNET
CESNET tested and worked with the ECI Apollo platform and Czech Light® Open DWDM together [22].
The ECI Apollo system has excellent features and is very compact, which makes testing easier. During
the trial, CESNET pushed both Apollo and Czech Light® Open DWDM to their limits by proving that
200 Gb/s DP-16QAM signals can reach further than 2,000 km over the standard single-mode fibre
G.652. CESNET and ECI experts also successfully demonstrated alien wavelengths capabilities, which
are gaining momentum, not only for NRENs but also in the global telco world.

It should be mentioned that 400G systems are almost exclusively 2x200G, with 200G transmitted
over one DWDM channel with DP-16QAM. True 400G signals using DP-64QAM were not available in
2015 (when the CESNET testing took place) and it looks as though almost all vendors went from
100G with DP-QPSK (also known as DP4QAM) to 200G with DP-16QAM, with 400G being considered
as the next step for really huge data-demanding applications.

CESNET and ECI also tested transmission over a cascade of narrow DWDM filters, with 200G DP-
16QAM signals able to be transported over 8 narrow DWDM multiplexers.

The last important point to note is the mixed scenario whereby “slow” and amplitude-modulated
signals such as 10G and 10G Ethernet were tested together with phase-modulated coherent 100G
and 200G signals. Again, results were very satisfactory and both encouraging and promising for
mutual transmission of high-speed coherent phase-modulated signals and really slow amplitude-
modulated signals (i.e. less than 1 Gb/s), which are used by new applications such as accurate time
transfer or remote sensing. As far as CESNET is aware, such experiments with real equipment are
rather rare.

The main results of the testing trial were:

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 15
Increasing Utilisation of the Photonic Layer

 The successful extension of all-optical reach to 2,000 km with single-channel 200G DP-
16QAM.
 High bitrate traffic can be easily transported as alien wavelengths over CESNET’s production
network.
 200 Gb/s signals can be transmitted over 240 km in a single hop, without the challenging,
and potentially hazardous, Raman amplification.
 New ways of compensation for chromatic dispersion in dark fibre lines with mixed coherent
and legacy amplitude-modulated signals.

2.1.5 Conclusions
A number of new formats and modulations currently offer the ability to increase the capacity of the
fibre. This has now evolved to the client side, and the simplest, and thus most cost-effective, form of
it at this stage is PAM-4. It uses four-level amplitude detection to double the transmission capacity,
but at the expense of lower signal-to-noise (SNR) performance when compared to NRZ, currently the
main client-side high-order modulation format.

With regard to increasing capacity by delivering more modes per core and more cores per fibre, the
SDM technologies of FMF and MCF are promising, but not without problems. FMF usually support up
to 15 modes, but this is limited to 6 because of constraints imposed by the available few-mode
amplifiers. MCF with up to 30 cores are available, but again, matching amplification techniques are
difficult to find, and splicing and connecting such fibres are much more challenging. Few-mode
techniques can be combined with multi-core fibres to achieve even higher spatial channel counts.
However, it is debateable whether SDM is currently relevant to NRENs, whose needs today can be
met by traditional 100 Gb/s and 200 Gb/s systems.

In respect of broader spectrum utilisation, existing technologies can lower the manufacturing costs
for the new type of equipment and can significantly improve the available bandwidth for the optical
transmission system. The most promising aspect is to use technologies and manufacturing
techniques for photonic systems that heavily use experience from the silicon industry.

Important progress in high-bitrate standardisation is being delivered by three standardisation bodies.

Within the IEEE-SA, which focuses on LAN and Ethernet protocols development, the IEEE 802.3
Working Group has several efforts underway concerning specification of Ethernet operating at rates
of 25 Gb/s, 50 Gb/s, (next gen) 100 Gb/s, 200 Gb/s, and 400 Gb/s. ITU-T is more involved in OTN
network protocols, and work is underway on B100G, where, in general, the modulation format used
and the effective bitrate per wavelength vary according to reach. Another principle of the B100G
work is that new transport containers are defined that are distributed to multiple wavelengths or
sub-carriers. A modular frame format is being worked on that can be used to describe containers
such as the current ODUk in increments of 100G.

GÉANT partners PSNC and CESNET have conducted successful 400G transmission experiments. The
PSNC test system used software-defined optics to deliver flexible data rates and modulation. This
means the network can intelligently scale from 200 Gb/s to 300 Gb/s to 400 Gb/s and from QPSK to
8QAM to 16QAM as required. All configuration and modulation options were tested. Following the
tests, PSNC has established and configured two operational 400G links. The CESNET test
demonstrated the successful extension of all-optical reach to 2,000 km; that high-bitrate traffic can
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 16
Increasing Utilisation of the Photonic Layer

be easily transported as alien wavelengths over CESNET’s production network; that 200 Gb/s signals
can be transmitted over 240 km in a single hop, without Raman amplification; and new ways of
compensation for chromatic dispersion in dark fibre lines with mixed coherent and legacy
amplitude-modulated signals.

2.2 Flexible Optical Networks

While Section 2.1 addressed technologies and concepts for increasing the available bandwidth, in
this section the focus is on the flexible use of the available capacity using flexible optical networks
(FON). First, Section 2.2.1 presents a review of the results obtained by the GN3plus Open Call project
REACTION, which evaluated FON. This is continued by a survey of the latest FON developments in
Section 2.2.2 and experimental research in Section 2.2.3. Finally, Section 2.2.4 reports on some
available implementations at commercial vendors.

2.2.1 Results of GN3plus REACTION

The GN3plus Open Call project Research and Experimental Assessment of Control plane
archiTectures for In-Operation flexgrid Network re-optimisation (REACTION) [23] designed a flexible
optical network scenario enabling software-controlled super-channel transmission. Innovations were
introduced in the context of data plane, control plane, and routing and spectrum allocation
strategies.

Bandwidth Variable Transponders

Before reviewing the results of the REACTION project, it is useful to discuss one of the key building
blocks in flexible optical networks, namely, bandwidth variable transponders (BVT). BVT have been
recently deployed in vendor multi-rate transponders. BVT make it possible to tune a certain set of
transponder functions in a point-to-point connection (one flow and one destination). A further
development of BVT has led to sliceable bandwidth variable transponders (S-BVT), which are not
commercially available yet. S-BVT provide the ability to allocate a certain amount of capacity on one
or several independent flows and transfer the flows to multiple destinations. The S-BVT also make it
possible to select from among different modulation formats, forward error correction (FEC) types,
baud rates and optical spectrum occupancy (12.5 GHz granularity, flexi-grid). Enabling flexi-grid and
super-channel features in S-BVT requires that all nodes along the link path support flexi-grid
functionalities, but the other variables in S-BVT could be implemented in a fixed-grid network
environment.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 17
Increasing Utilisation of the Photonic Layer

Fixed rates but

Fixed rate & tunable
BVT S-BVT
Fixed parameters parameters (FEC,
Modulation ...)

Figure 2.4: Transponder evolution path

Results

Data Plane
From a data plane architectural perspective, the REACTION project enhanced a bandwidth variable
transponder supporting 1 Tb/s multi-carrier transmission to support, in addition to dynamic
adaptation of transmission parameters, the sliceable functionality. Specifically, such sliceable BVT
are capable of creating multiple optical flow units (i.e. sub-carriers) that can be aggregated or
independently routed according to the traffic requirements.

The use of sliceability during provisioning and restoration in flexible grid optical networks was
addressed. Specifically, a scheme was proposed to exploit the possibility of establishing/recovering
an optical connection as a single super-channel or as a number of independent sub-carriers. Both
centralised and distributed implementations of the proposed schemes were evaluated through
simulations in a GMPLS-based scenario.

Results showed that, despite the introduced spectrum overbuild, the utilisation of sliceability
permits the amount of established/recovered traffic to increase.

Control Plane
From the control plane perspective, the REACTION project developed a solution relying on a GMPLS-
based distributed control plane with a path computation element (PCE) architecture. Specifically, a
novel PCE architecture was investigated. The architecture relies on an active stateful front-end PCE,
in charge of routing and spectrum allocation (RSA) computations and a back-end PCE in charge of
performing complex network re-optimisation solutions. The PCE architecture also relies on the
northbound distribution of link-state (LS) and traffic-engineering (TE) information through border
gateway protocol (BGP) (i.e. BGP-LS), utilised to provide the PCE architecture (also in the context of a
hierarchical implementation for multi-domain scenarios) with adequate networking information.

Routing and Spectrum Allocation

Novel routing and spectrum allocation (RSA) algorithms were designed and evaluated in the context
of flexible optical networks and, specifically, to be encompassed within the proposed PCE
architecture.

S-BVT capabilities were evaluated in the context of restoration and for in-operation network
planning operations:

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 18
Increasing Utilisation of the Photonic Layer

 Restoration: multi-path recovery and bitrate squeezing were applied to maximise the
amount of restored bitrate, also exploiting limited portions of spectrum resources along
multiple routes. A software-defined network (SDN) architecture was introduced to
adequately support the S-BVT configuration. The SDN architecture was applied to
experimentally assess that the overall re-configuration time upon failure detection was
concluded within two seconds, largely dominated by the proprietary control of optical nodes.
 In-operation network planning operations were considered. To increase traffic restorability
in flexi-grid networks, a multi-path after-failure repair optimisation (MP-AFRO) algorithm was
applied to reduce the sub-connections count by aggregating those belonging to the same
original connection and rerouting the resulting connection to release spectral resources. A
heuristic algorithm was deployed inside an in-operation planning tool in the form of back-
end PCE (bPCE) inside the application-based network operations (ABNO) architecture
controlling a network. The bPCE was connected to the centralised active stateful PCE.

Additionally, the benefits of flexi-grid were evaluated in the context of the UNINETT NREN network.
The expected evolution of the NREN traffic matrix was assumed to evaluate the benefits provided by
the adoption of high-rate transmission systems with and without the flexi-grid technology. Results
showed that fibre exhaustion will occur after around seven years from now, further postponed in
the case of flexi-grid networks [1].

2.2.2 Recent Developments on FON

Flexible optical networks maximise spectral efficiency through use of super-channels on a flexible
channel grid. Super-channels receive a negative penalty through non-linear interactions of closely
spaced sub-carriers and pass-band narrowing in network ROADMs. Numerical analyses have been
carried out to evaluate super-channel optimisation to maximise product of spectral efficiency and
optical reach [24]. Non-linear interactions are minimised by sub-carrier power optimisation where
edge sub-carriers have smaller launch power (see Figure 2.4). The pass-band narrowing penalty is
optimised by introducing a guard band around every super-channel. The best performance in terms
of product of spectral efficiency and optical reach is achieved for a super-channel guard band of
6 GHz (although 12.5 GHz tends to be the minimal granularity). Sub-channels within a super-channel
may be Nyquist-shaped by an optical inverse-Gaussian filter of order 2 to improve the sub-channels’
bandwidth utilisation. Such optimised super-channels will benefit from the minimal frequency
granularity of filtering elements of 3.125 GHz that are already available on the market [25].

Figure 2.5: Optimum sub-carriers launch power for 1 Tb/s 5-SC-DP-16-QAM and 400 Gb/s 4SC-DP-QPSK super-
channels for sub-carrier power pre-emphasis (SCPP) method

Although many vendors offer turn-key network solutions for backbone networks, there is increasing
need for vendor interoperability for network subsystems, or even elements, in order to reduce

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 19
Increasing Utilisation of the Photonic Layer

overall cost. For the future flexible optical network, an important requirement is a common network
management platform with an identical access to all network elements, independently of their origin.
On top of it, flexible planning tools are proposed to fully optimise resource use in the network [26].
New network management systems are therefore needed sooner than new optical fibres. Available
capacity of the current fibre footprint is expected to last till the end of the next decade [27].

2.2.3 Transponder Interoperability

Currently, most system integrators are about to develop transceivers beyond 100 Gb/s. Although
“black link” (BL) standardisation at 100 Gb/s is not yet fully accomplished, now is the right time to
begin thinking about and investigating how to realise BL interoperability at even higher channel
speeds.

NREN networks often consist of equipment purchased from different system vendors.
Interconnection of terminals (e.g. routers and switches) with a dense-wavelength-division-
multiplexing (DWDM)-based optical network has traditionally been realised through the deployment
of transponders with an intrinsic demarcation between the transport network and the client layer.
This interconnection can also be achieved by removing one grey short-reach transmitter (TX) and
receiver (RX) pair per channel (eliminating the transponders and substituting them with so-called
coloured interfaces in the packet machines) with associated cost savings. This configuration is known
as the BL approach, and related multi-vendor interoperable specifications are already part of the
ITU-T’s Recommendations G.698.1 and G.698.2 for DWDM links with and without optical line
amplifiers, respectively. This configuration is often referred to as an “alien wavelength” concept
when the DWDM TX and the DWDM RX are provided by the same vendor A. The alien wave concept
is discussed in more detail in Section 3.

The DWDM link consists of fibres and optical network elements – e.g. optical (de)multiplexers,
amplifiers, and (reconfigurable) optical add/drop multiplexers [(R)OADMs] – and is considered as
“black” if the design details are proprietary to the BL designer. However, the overall worst-case BL
transfer characteristics must be specified, e.g. minimum/maximum accumulated dispersion,
maximum differential group delay, and maximum optical signal-to-noise ratio (OSNR) path penalty.
In the in-force version of G.698.2 [28], application codes for non-return-to-zero modulated signals
with channel data rates up to a nominal 10.71 Gb/s have been standardised. Currently, ITU-T’s
G.698.2 BL work focuses on 100 Gb/s polarisation-multiplexed quadrature phase shift keying (PM-
QPSK) modulated signals. This modulation format requires clear definitions and specification
methodologies for parameters not present in existing optical interface standards, such as error
vector magnitude (EVM) or spectral excursion.

In the case of a single carrier communication, the following main transmission parameters have to
be considered (see [29]):

 Modulation format. Multiple types of modulation formats may be supported, enabled by the
use of a digital to analog converter at the transmitter. Possible formats include binary PSK
(BPSK), QPSK, m-quadrature amplitude modulation (QAM), etc.
 Sampling rate and analog bandwidth. To achieve interoperability, minimum hardware
requirements in terms of sampling rate, ADC resolution, and analog bandwidth have to be
guaranteed.
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 20
Increasing Utilisation of the Photonic Layer

 DSP algorithms. DSP may require standardisation in the case of data-aided algorithms
because the training sequences need to be agreed upon between the TX and RX. On the
other hand, if blind DSP algorithms are employed, less standardisation effort is expected.
 Optical layer parameters. To achieve interoperability, optical parameters have to be defined
within a strict range of working conditions. The range of output power values at the
transmitter (i.e. launch power) and receiver (i.e. maximum values of the overall and per-
channel received optical power and input power sensitivity) require dedicated
standardisation.
 Frequency slot. The supported wavelength tunability range, slot-width granularity, and the
actual nominal central frequency have to be specified, together with the possible required
optical cross connections and their filter characteristics. In addition to TX and RX, this
information has to be provided to intermediate switching nodes.

In addition to these per-carrier parameters, the following information has to be considered in super-
channel connections consisting of multiple sub-carriers:

 Number of sub-carriers. The number of supported sub-carriers has to be specified (e.g. eight
or four sub-carriers in the case of a 1 Tb/s channel).
 Sub-carrier spacing. Sub-carrier spacing should be compliant with flexible grid specifications
(e.g. the granularity of 6.25 GHz).

The interoperability of two S-BVT implementations has been recently demonstrated on a pan-
European multi-domain multi-vendor elastic optical network (EON) testbed [30] within the IDEALIST
project [31]. Two S-BVT implementations from the Inter-University National Consortium for
Telecommunications (CNIT)/Ericsson and Coriant, with hard-decision (HD) and soft-decision (SD)
FECs, were evaluated. The control architecture was able to configure the S-BVTs so the multi-vendor
transmission reach is failure-free up to 300 km with current FEC standards. The testbed, illustrated in
Figure 2.6, consisted of data and control planes.

Controller Controller

Flex-grid WSS
LASER
(even)
DSPRX
50GSa/s
Optical
Sampling
64 GSa/s DWDM link Front-
scope
DSPTX (13 GHz) end
(20GHz)

LASER
DSPRX
DAC Board
(odd)
Pisa (Italy)
[dB]

Munich (Germany)

Wavelength [nm]

Figure 2.6: Multi-partner pan-European testbed

The data plane setup consists of flexi-grid nodes, based on configurable spectrum selective switches
(SSS), a CNIT/Ericsson DSP unit at the Tx and two different ones (CNIT/Ericsson and Coriant) at the Rx
as part of an optical coherent testbed. The Tx is able to provide a super-channel with different
configurable numbers of carriers and capacity (i.e. 1 carrier for 100 Gb/s, 3 carriers for 400 Gb/s, and
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 21
Increasing Utilisation of the Photonic Layer

7 carriers for 1 Tb/s). At 1 Tb/s, PM-16QAM Nyquist-shaped signals shaped by a roll-off = 0.05 and
symbol rate = 23 GBd were adopted.

The control plane is performed via controllers located at each physical node of the network.
Controllers are connected with the data plane testbed (i.e., SSS, Tx and Rx) by means of USB, serial
and GPIB interfaces. Controllers are able to automatically configure SSS (i.e. filter shape as the
reserved frequency slot), transponders parameters (i.e. symbol rate, number of carriers, sub-carrier
central frequencies) and DSP parameters (i.e. modulation format, FEC). Multi-vendor interoperability
between CNIT/Ericsson and Coriant DSPs was achieved by the exchange of novel application code
(AC) and transponder class (TC) attributes, in line with ITU recommendations [28].

Figure 2.7 reports the transmission performance of the single-vendor (CNITTX  CNITRX - SV) and
cross-vendor (CNITTX  CORRX - CV), where COR stands for Coriant. In both cases, blind-DSP
algorithms [32] were used because an interoperability scenario would not allow the usage of
algorithms that require knowledge of the link [33] or of training sequences [34].

Figure 2.7: BER performance

If the cross-vendor (CV) solution is selected, only already standardised FEC can be used. For example,
the ITU standard G975.1 hard-decision super-FECstd (HD-FECstd) with 20% overhead could use a pre-
FEC BER threshold = 1×10-2. This value could increase, if new SD-FEC become standard, up to a
reasonable pre-FEC BER threshold = 2×10-2. Based on the pre-FEC thresholds, the following
conclusions can be drawn concerning the experiment carried out by employing the testbed depicted
in Figure 2.6. If the usage of already standardised HD-FECstd is assumed, transmission up to ~750 km
in the case of SV transmission is guaranteed. This halves in cross-vendor mode. Although the system
performance has been significantly reduced, such a scenario would still guarantee error-free
transmission, with CV transmission, over the majority of European links. Moreover, if a standardised
FEC (SD-FECstd) could be adopted, the reach would approach 1,100 km for SV transmission, and ~600
km for CV transmission. Finally, in the case of proprietary FEC, the transmission distances would be
~1,800 km [35], and once again about half for CV transmission. These values are summarised in
Table 2.2.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 22
Increasing Utilisation of the Photonic Layer

Reach [km]

Case HD-FECstd SD-FECstd SD-FECth

SV 750 1100 1800

CV 300 600 900

Table 2.2: Performance results

In addition to the results of the conducted experiment, Figure 2.7 also displays the results obtained
within a second experiment that has been reported in [29; 32]. In this previous analysis, the vendor
configuration was mirrored by having (CORTX  CORRX - SV) and cross-vendor (CORTX  CNITRX - CV).
In [29; 35] the channel configuration was slightly different (32 GBd and 38 GHz channel spacing). The
performance of this experiment is reported by the curves with filled markers in Figure 2.7 and in this
comparison, it is clearly visible that the two single-vendor scenarios and two cross-vendor-ones,
between them, achieve similar performance. From a DSP perspective, the degraded performance of
CV could be associated to the lack of knowledge of the system, such as for example Tx / Rx I/Q skew
compensation. The CV transmission, with standard HD-FEC, can reach ~300 km, and therefore if
longer distances are needed, a solution concerning standardisation on SD-FEC must be agreed within
the ITU panels.

2.2.4 FON Vendor Available Implementations

In order to meet the future capacity demand on optical networks and to gain more capacity and
network efficiency, introducing flexibility into the network is the most cost-efficient strategy to
choose [1; 31]. Taking a longer perspective, to enable 1 Tb/s client signals in transport networks the
only cost-efficient option is to migrate the fixed network to a flexible optical network.

In today’s commercially available flexi-rate transponders (BVT), a subset of the functionalities

mentioned in Section 2.2.1 are available. JRA1 T1 conducted a vendor survey during the GN4-1
project to identify the functionalities currently available and those coming in a two-year term. There
are a variety of solutions and implementations, but a subset of common functionalities could be
found in more or less all vendor implementations. Table 2.3 shows these subsets.

FEC type Modulation Spectral occupancy Line rate Number of Baud rate
format carriers

G-FEC BP-BPSK 37.5 GHz up to 200 GHz 50G 1–2 25 Gbaud –

with 12.5G Hz granularity 32 Gbaud
HG-FEC BP-QPSK 100G
SD-FEC BP-8QAM 200G
E-FEC BP-16QAM 400G
U-FEC

Table 2.3: Tunable parameters in today’s vendors’ multi-rate transponder implementations

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 23
Increasing Utilisation of the Photonic Layer

Not all of these parameters are supported by all vendors. There are some minor differences
between them. Modulation format and FEC are the most basic common building blocks in already-
existing multi-rate transponders. Spectral occupancy and Baud rate, on the other hand, have been
implemented slightly differently from vendor to vendor. Regarding spectral occupancy, both
transponders with limited spectral tuning flexibility and transponders that give the ability to tune the
granularity by 0.5 GHz can be seen.

The advantage of flexi-rate transponders is that they give the network operators the ability to tune
the transponder parameters in order to support a certain service with the highest network
utilisation rate. For example, if operators want to deliver a 400G service over a short distance, the
most optimised way to deliver the service is to choose the highest possible modulation format, with
SD-FEC in order to minimise spectral occupancy usage.

Forward Error Correction

Different FEC types add some overhead to the transmission channel. The amount of overhead can
vary from 7% to more than 20%. More overhead means more coding gain. The FEC types used in
transponders are usually vendor proprietary and this means that it is not possible to run
transponders from different vendors with each other. The exceptions are G-FEC and high-gain (HG)
multi-vendor FEC.

2.2.5 Conclusions
With regard to enabling effective, optimal use of the availability capacity through flexible optical
networks (FON), the GN3plus Open Call project REACTION introduced significant innovations in the
context of data plane (support for sliceability), control plane (novel PCE architecture), and routing
and spectrum allocation strategies (novel RSA algorithms). In each case, results confirmed the
capacity-utilisation benefits of a flexible network over a fixed network, including postponing fibre
exhaustion. Developments since REACTION have further enhanced FON’s ability to maximise
spectral efficiency and reach by minimising the negative penalties to which super-channels are
susceptible, namely interactions of closely spaced sub-carriers and pass-band narrowing in network
ROADMs. For the future, there is increasing need for vendor interoperability for network subsystems,
or even elements, and hence for a common network management platform that permits identical
type of access to all network elements, independently of their origin, together with flexible planning
tools to fully optimise resource use in the network.

The interoperability of two S-BVT implementations has been recently demonstrated on a pan-
European multi-domain multi-vendor elastic optical network (EON) testbed within the IDEALIST
project. Experiments proved the need for a new soft-decision FEC standard which would significantly
increase reach in single- and cross-vendor transmission.

A survey of BVT vendors showed a subset of common functionalities, though with implementations
– particularly of spectral occupancy and Baud rate – varying from vendor to vendor.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 24
3 Spectral Sharing and Alien Waves
In Section 2 the physical media and the enabling technologies for increasing the capacity were
addressed. A special type of spectrum utilisation is the alien wave (or alien wavelength) and alien
spectrum, where the spectrum is opened and waves from other equipment and/or entities are
injected into a system. This section deals with these types of networks and the focus is on the
further development of a modelling tool for estimating the viability of alien waves as well as
reporting on experiments with alien waves for, among other purposes, validating the modelling tool.

Hence, Section 3.1 provides an overview of the advantages of alien waves and shared spectrum, and
it forms the basis for the rest of the section. In Section 3.2 the multi-domain optical modelling tool
(MOMoT) is described, with the latest extensions, followed in Section 3.3 by a report on best
practices with alien waves and, in particular, the deployment of an alien wave-based optical
transport layer. Section 3.4 describes experimental scenarios and results to validate the estimations
provided by the MOMoT modelling tool.

3.1 Advantages of Alien Waves and Shared Spectrum

Alien waves (AW) are wavelengths that are unknown to the network domain and that share the
spectrum with native wavelengths. Spectral sharing is another term used for alien wavelengths. The
topic has been discussed within the NREN community for a number of years and is now finally
gaining momentum as AWs are deployed more and more throughout the NREN networks.

As research is increasingly becoming a global effort, research networks equally need to become
more dynamic and flexible by extending their resources and capabilities beyond single countries,
regions or domains. To accommodate future research projects and global partnerships, research
networks can be developed in a more open and federated way. Instead of creating new multi-
domain services in the traditional “cross border” type of manner, where information is typically
exchanged at the bare minimum information level (e.g. back-to-back information exchange on a
black-and-white interface), research networks can tear down the boundaries by looking at what the
available resources are, and by opening up access to these resources (e.g. by the exchange of
information on OTN level or access to spectrum in another network).

One way of doing this for the individual research networks could be by opening up access to the
DWDM spectrum and thereby offering a means of utilising foreign infrastructure. An example is alien
wavelengths, in which resources from a carrier network are being utilised to transport optical
channels, not controllable by the carrier.

Deliverable D13.1
Research and Study Network Technologies – 25
White Paper
Document Code: GN4-1-16-7D602
Spectral Sharing and Alien Waves

In the past there have been some studies and experiments on alien wavelength communication in
the research communities. In particular, JRA1 T1 participants SURFnet and NORDUnet have
partnered up to turn these experiments into real-life production services that are now being used
for stabilising and expanding their networks.

As SURFnet and NORDUnet’s partnership on developing alien wavelength services was established
some time ago, a lot of testing has been carried out in order to establish the reliability and integrity
of the optical signal interaction. Furthermore, planning and development effort was put into
defining the correct responsibilities, suitable procedures, and proper monitoring tools for putting the
services into production.

Although vendors traditionally take a more conservative view on sharing their network capabilities
(as it typically diminishes their revenues), there has been a tendency for the industry to open up to
the concept after all. Ciena’s DWDM platform (6500) has a setting for adding “Foreign” wavelength
channels and can even control its power along the system. Other vendors, such as Alcatel Lucent
(with their 1626LM platform) and ADVA (with their FSP 2000 / FSP 3000 systems) can also take in
alien wavelengths. Infinera also supports AW networking, but in a less scalable configuration.

The latest trend is that some of the vendors seem to be taking the concept even further, as will be
discussed in Sections 3.3.2 and 3.3.3.

3.2 Modelling Tool Development

In GN3plus, the Open Call project MOMoT [36] developed a multi-domain optical modelling tool
(MOMoT) to estimate and predict AW performance and assess the implications for existing traffic in
the network path. During GN4-1, the modelling tool has been further developed by JRA1 T1 and new
functions have been added. The following sub-sections introduce the modelling tool and the
extensions developed, provide guidelines on how to use the tool, and outline the basic principles,
capabilities and assumptions of its adopted models. The main advances compared with the version
of the MOMoT tool delivered by the GN3plus MOMoT Open Call project are also summarised.

3.2.1 Introduction to the Tool

The MOMoT tool enables users to input certain characteristic parameters, such as length from A to B,
wavelength, input power, bitrate, etc., and based on these the tool will estimate whether or not the
connection is likely to be successful and what the impact on existing services may be. The tool will
serve as a “back-of-the-envelope” calculation and evaluation of the feasibility to deploy an AW in a
given network scenario. Development of the tool has been done with speed and effectiveness in
mind, so instead of doing fully detailed and time-consuming multi-channel simulations, a safe-zone
approach has been applied. The tool makes a quick assessment of multi-channel effects, without a
deep simulation, and warns the user of any likely implications.

The modelling tool estimates the bit error rate (BER) of alien wavelengths in circuit-switched optical
networks with reference to the following modulation formats:

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 26
Spectral Sharing and Alien Waves

 On-off keying (OOK) with direct detection.

 Polarisation multiplexing quadrature phase shift keying (PM-QPSK) with coherent detection.

Thus, typical bitrates to be considered can be 10 Gb/s and 100 Gb/s.

3.2.2 Guidelines on How to Use the Tool

The modelling tool is software (.exe) providing a user-friendly interface where link, native and alien
wavelength parameters can be set. Based on the provided input parameters, the software outputs
bit error rate estimation.

Operation of the Tool

The user can input various parameters, among others the bitrate of the alien wavelength, the
channel spacing, the modulation format and power levels. In combination with information on the
location of dispersion compensating fibre (DCF), the tool provides the following output:

 Evaluated BER pre-FEC: in this field the estimated BER of the (alien) wavelength is reported.
 Warning if the power of the alien wavelength exceeds the threshold of 2 dBm. This has been
introduced since the level of power may be too high and it may induce excessive cross-phase
modulation (XPM) on active wavelengths.
 In the case of injecting a PM-QPSK alien wave into an OOK native network, output messages
will report the BER of the alien wave at varying guard band between the alien wave and a
native OOK.
Indeed, the BER of a PM-QPSK is strongly affected by the presence of OOK neighbours
because of XPM. Thus, by spacing out PM-QPSK and OOK, the performance of the PM-QPSK
improves. Guard band is defined as the number of free channels between a PM-QPSK and an
OOK. Another kind of output message is reported if the performance is strongly affected by
the PMD.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 27
Spectral Sharing and Alien Waves

Figure 3.1: Modelling tool interface and output messages including guard band information

3.2.3 Basic Principles, Capabilities and Assumptions of the Tool

The modelling tool is based on an analytical model for BER estimation detailed in Modeling and
Distributed Provisioning in 10–40–100-Gb/s Multirate Wavelength Switched Optical Networks [37].
Two models are adopted for the two different transmission techniques considered:

1. OOK with direct detection: e.g. suitable for 10 Gb/s.

2. PM-QPSK with coherent detection: e.g. suitable for 100 Gb/s.

The model aims at providing the estimated value of BER before forward error correction (pre-FEC
BER) for both the alien wavelength and the native lightpaths active in the network. The model and
the modelling tool are built based on assumptions described in [37], and summarised here:

 Amplifier noise figure is assumed to be the same for all spans.

 The following fibre parameters are the same for all spans and are not visible to the user:
effective area of 80 μm2, PMD parameter 0.1 ps/sqrt(km).
 The digital signal processing (DSP) at the receiver, for PM-QPSK signals, can completely
compensate for linear impairments (i.e., both polarisation mode dispersion (PMD) and
chromatic dispersion (CD)).

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 28
Spectral Sharing and Alien Waves

 Four-wave mixing (FWM), stimulated Raman scattering (SRS) and stimulated Brillouin
scattering (SBS) are not explicitly included in the model. These can be considered as worst-
case margins (e.g. lower than 1 dB as optical signal-to-noise ratio (OSNR) penalty [37]). In any
case, the power levels typically used in telecom networks are low enough to consider these
effects as negligible.
 Other effects such as ageing, filtering effects, etc. are considered as margins to the OSNR in
the computations that can be configured through the interface. A typical value for the
margins can be 3 dB

In general, the adopted models account for the following physical impairments:

 Amplified spontaneous emission (ASE).

 PMD.
 CD.
 Self-phase modulation (SPM).
 Cross-phase modulation (XPM). XPM is particularly relevant when a PM-QPSK alien
wavelength is surrounded by OOK wavelengths.
 As stated earlier, other physical impairments are considered through the configurable worst-
case margins.

For more details on the adopted models, please refer to [37]. Also, a more detailed description of
the inputs, outputs and the operation can be found in Appendix A.

3.2.4 Extensions to the Tool

In this sub-section, the main advances with respect to the version of the tool delivered by the
GN3plus Open Call MOMoT project are summarised:

 Enabled the 10G OOK BER computation.

 Included margins as an input parameter of the system.
 Unlocked the bitrate in case of 10G OOK.
 Modified the horizontal header of the table.
 Solved a bug when loading some configuration files: introduced a check validating the
structure of the configuration file (avoiding the tool crash at start-up).
 Solved a problem related to both “warning: VOA attenuation” and “Error 2” error messages
(avoiding problem with specific configuration files with high power): such warning and error
messages are avoided.

The modelling tool developed in the GN3plus Open Call MOMoT project estimates, for a subset of
available modulations, the impact and viability of injecting alien waves into host optical transport
networks. The tool has been further developed in the context of GN4-1 and, among others, 10G OOK
BER computation has been added, which was specifically requested in GN3plus by NRENs. The tool
provides a simple graphical user interface and accounts for a wide range of physical impairments,
which are relevant when transporting alien waves.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 29
Spectral Sharing and Alien Waves

3.3 Best Practices from Other Projects

In order to gain practical experience and develop guidelines on how to design, commission and
maintain alien wave services, different trials and studies have been performed in the NREN
communities. This section outlines general best practice, and the lessons learned from AW trials by
JRA1 T1 NREN third-party participants SUNET and UNINETT, and from NREN operators’ experiences.

3.3.1 General Best Practice

Setting up alien waves between two research networks essentially comes down to the technicalities
of how well the actual platforms support adopting and managing alien wavelengths. Despite this,
there is an equal challenge in simply partnering up with other research networks. It is crucial that
service deployment aspects, such as agreeing responsibilities or defining the operational procedures,
are handled collaboratively, and at an early stage of the partnering. Further essential factors include:
definition of standards and agreements on the technical details for the service deployment;
documenting all the design parameters and build values (e.g. launch powers, receive inputs, etc.);
sufficient monitoring of the alien wave services; and reporting of alarms and problems in a clear
fashion to each partner.

The concept of sharing network resources and opening up for non-native signals does not have to be
restricted to single-signal adoption. Instead of merely assigning a designated input point on a
dedicated channel, a slice of the native spectrum (or even the full spectrum) can be shared with the
partner, thus giving them better possibilities for expansion. This concept is typically referred to as
spectrum sharing.

3.3.2 SUNET Alien Wave-Based Network

During the design phase of SUNET’s (the Swedish research network) new network, it was decided to
take a new approach in terms of establishing router connectivity between the universities.

Instead of relying on the traditional way of building networks, in which expensive transponders are
utilised for the connectivity, SUNET are planning to use coherent coloured interfaces in the routers.
Specifically, these coherent interfaces are 100G Juniper line cards with CFP2 client interfaces, which
will be connected directly towards the next university site (the red line in Figure 3.2 below) as alien
waves via optical amplifiers from ADVA. Furthermore, backbone routers are likewise connected via
alien waves (the blue lines in Figure 3.2).

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 30
Spectral Sharing and Alien Waves

Figure 3.2: Optical network setup

The effective channels (originating in the Juniper line cards) will therefore be “alien” to the ADVA
platform. The plan is to control and manage the spectrum in a collaborative manner, by utilising a
GMPLS control plane interworking on the ADVA and Juniper systems.

The 100G coherent line cards from Juniper are beta version and SUNET is one of the first customers
in the world to try them out. The process of building this network has started and the first link has
been established; finalisation of the complete network is expected to continue until October 2016.

3.3.3 UNINETT 100G Alien Wavelength Cisco–Juniper Interoperability Test

The purpose of the test was to verify support of 100G alien wavelengths in UNINETT’s (Norway’s
NREN) optical network. In addition, UNINETT wanted to gain some experience with new 100G
OTN/DWDM cards from Juniper and Cisco, and prove the interoperability between them. The
following tests have been performed:

 100G single-vendor alien wavelength (AW) test over Coriant hiT 7300 platform.
 100G multi-vendor AW test over Coriant hiT 7300 platform.

Test Layout and Equipment

The DWDM path of the link was established over two point-to-point DWDM systems based on a
Coriant hiT 7300 platform between Trondheim and Oslo. As shown in Figure 3.3 below, the DWDM
systems on the Oslo side are connected to each other by using dedicated dark fibre between mux
and demux.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 31
Spectral Sharing and Alien Waves

Both routers are located in Trondheim but in different locations. A Spirent test generator with 10G
interface and logical loop was used on the routers in order to create 40 Gb/s of traffic over the 100G
AW link.

Figure 3.3: UNINETT Cisco–Juniper interoperability test scenario

The total length of the test DWDM path is 1,235 km, and it consists of nine “optical line repeater”
NEs, six OADMs and two terminals. One of the DWDM systems is based on 100 GHz spacing and the
other on 50 GHz spacing. The system utilisation on both systems is over 50%, running a mix of 10G
and 100G operational wavelengths. 100G and 10G wavelengths are grouped in separate optical
bands with 200 GHz of guard band between them. The total OSNR is about 16.2 dB.

The 100G DWDM MX MIC is based on the 100G CFP2-ACO coherent pluggable optics technology.
Both routers were configured with HG-FEC (multi-vendor) on 193 THz frequency channel. 40 Gb/s of
load was generated on the 100G AW link and the test ran over several days. Figure 3.4 and Figure
3.5 shows the results on the Juniper interfaces.

As shown in the figures, neither packet loss nor post-FEC errors were registered. The minimum and
maximum pre-FEC BER measured during a 1-second interval over a 15-minute period are shown in
Figure 3.4.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 32
Spectral Sharing and Alien Waves

Figure 3.4: Pre-FEC bit error rate on Juniper router

Figure 3.5: Post-FEC bit errors on Juniper router

Interoperability Test Conclusion

The test result was better than expected. Configuration of the routers and DWDM was done in
accordance with the procedure and without any obstacles. Even if the DWDM system had an OSNR
value close to the limit of HG-FEC operation, there was no pre-FEC bit error and packet loss.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 33
Spectral Sharing and Alien Waves

3.3.4 Operator Experiences with Alien Waves

There are definitely many possibilities in alien waves and spectrum sharing and it has a lot of
potential in the research communities. As mentioned in Section 3.1, SURFnet and NORDUnet have
already implemented some of their services as alien waves and the stability of these services has
been proven in their production environment for a couple of years now.

The willingness to implement wavelengths from other vendors (operated by other systems and
other operating centres) is founded on past simulations, experiments and trials by NRENs and within
the GÉANT context that have given the confidence to commence real implementation.

During the experiments and simulations mentioned above, it was investigated whether or not it was
possible to find an optimum power area for the native and alien signals’ interaction. Clearly, the
non-linear effects become evident when the signal intensities are relatively increased, and the
results of the experiments showed both cross-phase and self-phase modulation problems when
using too high signal powers or having too little guard band.

Apart from these findings, it was concluded that the operational challenges in partnering up for
sharing waves or spectrum are just as important as solving technical problems at the commissioning
stage.

3.4 Experimental Scenarios and Key Results

Several experimental deployments have been conducted with and without the JRA1 T1 partners’
participation. The results of these are useful for gaining confidence and for providing some
engineering rules of thumb for optimal implementation.

Another matter is how to determine the actual design parameters and the specific physical values
that are needed when the alien waves are adopted.

It is here that the MOMoT tool can be very useful and, conveniently, can calculate the effect of the
alien/native signal interaction. It is easy to model the production environment, and then calculate
concrete values to be used on commissioning of the alien waves.

However, in order to have confidence in these values, verification testing has been performed to
check the integrity of the tool and, if necessary, make corrections and improvements. The initial
verification testing was performed in GN3plus through a trial with SURFnet and Dante, which was
compared with the estimations from the MOMoT tool.

In order to gain more verification assurance, it is desirable to have multiple trials to confirm the
integrity of the tool and thus more trials was performed in NORDUnet’s live production network
with BER vs. launch power as a performance criterion.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 34
Spectral Sharing and Alien Waves

3.4.1 Experiments in NORDUnet

1st Trial Copenhagen–Hamburg and Back

In the first trial NORDUnet connected two transponders (at the same site in Copenhagen 1), and
then set up up a channel going uni-directionally from Copenhagen to Hamburg and back to
Copenhagen again. The link from Copenhagen 2 to Hamburg 1 goes via Fyen and down towards
Germany. The other link, from Hamburg 2 back to Copenhagen 1, goes via Sealand and thus the total
coverage is approximately 1,000 km.

transponder

transponder
Unidirec onal
setup. RX from the
TX to the line line side
side

Copenhagen 2 Copenhagen 1

App.
1000
km

Hamburg 1 Hamburg 2

Figure 3.6: Test scenario between Copenhagen and Hamburg

Unfortunately the reach of the transponders that were available for the trial was not long enough,
and it was not possible to successfully compare the results with the MOMoT tool. Due to this reach
limitation, it was soon decided to create another trial in the same network but covering a shorter
distance.

2nd Trial Copenhagen–Gothenburg

The second trial was approximately 300 km long, going from Copenhagen, Denmark to Gothenburg,
Sweden. As in the first trial, two transponders were connected in a traditional point-to-point uni-
directional setup (see Figure 3.7). Again, a channel was added along with the normal production
services with a guard band of 500 GHz. The channel was manually power-adjusted to fit the rest of
the spectrum throughout the three nodes, and kept static after the links were balanced. RX pre-FEC
values were measured in the Gothenburg end point B, as a function of launch-power variation in
Copenhagen 1 end point A.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 35
Spectral Sharing and Alien Waves

Gothenburg
transponder
Point B
RX from the
line side

App. 300 km

transponder
Point A
TX to the line
side

Copenhagen 2 Copenhagen 1

Figure 3.7: 300 km test scenario between Copenhagen and Gothenburg

In order to verify the predictions of the MOMoT tool, the Gothenburg link values, such as span
lengths, launch power, bitrate and modulation format, were used as input.

Figure 3.8 shows the comparison of the actual link measurements and three graphs that represent 3
different scenarios calculated by the tool. The margin M and the noise figure NF were varied in pairs:
(M = 3, NF = 3), (M = 3, NF = 7) and (M = 7, NF = 7). The margin is a parameter that takes fibre ageing
and other non-modelled factors into account. Typically, a value of around 3 is a good representation
for an average optical link. The noise figure of the amplifiers is often known for optical networks.
However, in this case, its value was not verified so two variations were chosen.

In Figure 3.8 the tool resemblance with the actual measurements looks fairly reasonable when
compared with the (M = 3, NF = 7) tool curve. At input powers above 1 dB, however, the lab curve is
bending off. This can be explained by the power limitation that takes place in the link. Due to
amplifier gain management in the link, the effect of further increases in the power is reduced. Also,
it should be mentioned that the lab results curve was parallel-shifted 5 dB to the left in this graph,
which accounts for losses occurring in the initial phase of the transmission and underway in a
wavelength selective switch. In terms of an AW link evaluation where these losses are unknown, the
link quality estimation would not overestimate the quality, but rather be pessimistic about the
success of the link. Minimising the insertion losses or being aware of these so they can be taken into
account is therefore always preferred.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 36
Spectral Sharing and Alien Waves

Figure 3.8: The figure compares the pre-FEC BER curves of the field trial (lab) measurements and three curves
produced by the MOMoT tool

3.5 Conclusions
Alien waves and spectral sharing are gaining momentum among NRENs as one way of becoming
more dynamic, flexible and open in order to meet R&E needs and potentially provide savings on
fibre deployment. Studies and experiments – including by JRA1 T1 participants SURFnet and
NORDUnet – have helped identify prerequisites and best practice not simply in terms of technology
but also in terms of definition of responsibilities, standards, procedures, documenting design
parameters and build values, monitoring and reporting. The reliability and integrity of alien waves
have been sufficiently proven for SURFnet and NORDUnet to have transitioned AW services into
production for both client and own use.

Valuable results have been obtained on defining an optimum power area for the native and alien
signals’ interaction: the non-linear effects become evident when the signal intensities are relatively
increased, and both cross-phase and self-phase modulation problems are encountered when using
too high signal powers or having too little guard band.

The multi-domain optical modelling tool (MOMoT), originally developed during a GN3plus Open Call
project and further developed during GN4-1, helps determine these and other design parameters
and specific physical values needed for AW adoption. Verification tests in NORDUnet’s live
production network, with BER vs. launch power as a criterion, showed a reasonable correlation
between MOMoT’s predictions and actual measurements, and indicated that the tool would
underestimate link quality rather than overestimate; minimising the insertion losses or being aware
of these so they can be taken into account is always preferred.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 37
4 Frequency and Time Distribution
In order to provide zero-touch connectivity and to facilitate a number of high-demanding
applications, it is necessary to provide mechanisms to synchronise and distribute time and frequency
information among laboratories and users. Solutions to the challenge can be sought in different
layers, and in this section the issue is addressed in both the physical layer and in layer 2/3.

In Section 4.1, the concepts are described for distribution of accurate time and stable frequency in
the physical layer. Different conceptual architectures are explained and several joint inter-NREN
experiments described, which were carried out over links ranging up to 1,500 km. The results
indicate the viability of the concepts, and an overview of the future challenges is also provided.

While the physical time transfers are the costly and accurate highways, the precise transfer of time
information can also be supported in the higher layers. This is investigated in Section 4.2 with audio
and video as the candidate applications. In particular, precision time protocol (PTP) is investigated,
and how it can work for connections running on top of MPLS tunnels. Also, a joint experiment
evaluating PTP over a standard Internet connection on a 500 km link between Erlangen and Prague is
discussed.

4.1 Distribution on the Lowest Layer

Accurate time or stable frequency is required by many applications in navigation, metrology,
telecommunications, geodesy, astronomy and other scientific and industrial areas. One source of
accurate time is often a Global Navigation Satellite System (GNSS) receiver providing the one pulse
per second (1PPS) signal. However, such a solution is scalable only with difficulty and is sometimes
not possible due to problems with antenna installation (e.g. large buildings, tunnels). Furthermore,
time and frequency distribution over optical links allows significant improvements of uncertainty and
stability compared with GNSS methods [38; 39].

Precise time and ultra-stable frequency transfer opens the possibility to set up experiments with a
high level of precision that has not been attainable to date. Without being exhaustive, examples
include:

 Antenna networking in astrophysics.

 Fundamental constants measurements (e.g. Boltzmann constant).

Deliverable D13.1
Research and Study Network Technologies – 38
White Paper
Document Code: GN4-1-16-7D602
Frequency and Time Distribution

 Satellite link tests (e.g. Metrological Fibre Network with European Vocation + (REFIMEVE+)
signal could be used by the European Space Agency’s (ESA) spatial atomic clock program
PHARAO/ACES within the next few years).
 Fundamental physics tests [40; 41] (e.g. spectroscopy of hydrogen and its derivatives, high-
resolution spectroscopy in the physics of atoms, ions and molecules, the search for parity
violation effect in molecules).
 Geodesic applications (tidal measurements, seismology).

4.1.1 Physical Layer

Standard transmission over fibre pairs is used in telecom applications, but due to many effects,
transmission delay is changing, e.g. thermal dependence of the refractive index, thermal material
expansion, polarisation mode dispersion, etc. This is not an issue for data transmission. However, for
extremely precise time or ultra-stable frequency transmission, the propagation delay must be stable.
It can be achieved using the same physical path (fibre, amplifiers, muxes, etc.) for both directions of
propagation and further improved via additional propagation time stabilisation.

From this point of view, distribution can be either over a pair of uni-directional channels or over one
bi-directional channel. Further, the channel can use dedicated fibre or lambda as follows:

 Metrological institutes can lease their own dark fibre infrastructure to transmit metrological
signals, independently of any carrier or R&E network. For instance, the PTB in Germany
acquired fibre IRUs from Braunschweig to Munich (920 km) and to Strasbourg (700 km) and
built a communication network from them. Also, project OPTIME in Poland is based on the
usage of dedicated fibres. This solution gives great autonomy to the experiment and to use
non-standard DWDM transmission systems, such as research and development materials or,
for instance, Brillouin amplifiers instead of erbium amplifiers. But for the metrology
laboratory, the main drawback of this approach is the need for specific competences to
launch RFQs, to operate the fibre network and, above all, the need for additional funding for
IRUs.

 The second approach consists of spectrum sharing between a partner (e.g. an NREN), which
leases dark fibres, and a metrological laboratory, to which is given the opportunity of using
NREN fibres to transmit its metrological signal. In this WDM infrastructure, the metrological
signal is considered as “alien” and can be a WDM wavelength or a broader part of the
spectrum. This fibre mutualisation and therefore the cost sharing between partners is a main
advantage of this solution. Moreover, the metrological laboratory relies on NREN
competences to operate the network and can concentrate solely on its core “business”.
However, spectrum sharing between different kinds of optical signals can lead to
architecture and performance compromises.

Obviously, establishing a bi-directional channel over dedicated fibre requires some specialised
equipment, e.g. bi-directional amplifiers. Similarly, when fibre is shared with legacy DWDM, all
legacy uni-directional components must be bypassed (see Figure 4.1).

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 39
Frequency and Time Distribution

Figure 4.1: Bi-directional channel in legacy DWDM transmission system

Different kinds of mutualised architecture can be built. In the following sub-sections, bi-directional
alien lambda and uni-directional alien lambdas are considered in detail.

Bi-Directional Alien Lambda

From a practical point of view, a bi-directional alien lambda/lambdas can be established in CWDM or
passive DWDM systems only; all telecommunications using optical amplifiers contain isolators
preventing bi-directional operation. This fact limits reach. However, multiple passive spans can be
concatenated together and amplified outside. Such concatenated lambda for precise time
transmission, established in the CESNET network in 2013, was reported in All optical two-way time
transfer in strongly heterogeneous networks [42]. It consists of CWDM and DWDM parts showing
total attenuation of 22+21 dB, as shown in Figure 4.2.

Figure 4.2: Path UFE (Praha) – VUGKT (Pecny): concatenation of passive lambdas

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 40
Frequency and Time Distribution

Bi-Directional Alien Lambda Bypassing DWDM Devices

This architecture is mainly used within the REFIMEVE+ project in France, aiming to deploy a national
infrastructure on RENATER DWDM backbone to disseminate an ultra-stable frequency towards
scientific laboratories. The metrological signal is transmitted bi-directionally through each dark fibre
to compensate it precisely, so this solution is not compliant with usual DWDM technology, which
requires isolators.

Figure 4.3 depicts the Nancy–Reims link on RENATER, equipped for REFIMEVE+ transmission.

Mono-directional Multiplexer
Optical Amplifiers (OADM)
Node 54km 82km 69km 72km
Node
Switches -16dB -18dB -16dB -16dB Switches
& &
Routers Routers

Station Station
1 La Veuve Revigny Troussey 2
Reims Nancy
Bi-directional Ultrastable signal
Optical Amplifiers regeneration

Figure 4.3: Detailed scheme of a span between Reims and Nancy

Dedicated OADMs are installed on each PoP and in-line amplifier (ILA) to make the metrological
signal (in red) bypass DWDM equipment. Once extracted in ILAs, the signal is reamplified separately
from the NREN signal. This insertion of OADM causes an extra attenuation of 1.6 dB on each span.

CESNET reported in 2014 on precise time transmission over the 306 km line Praha–Brno [42]. This
line features both C- and L-band transmission systems. The excessive attenuation of 27 dB in the
span Praha–Potehy was solved by creation of a bi-directional ILA site only.

Figure 4.4: Praha–Brno line, C- and L-band systems deployed, 306 km

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 41
Frequency and Time Distribution

Uni-Directional Alien Lambda

The uni-directional alien lambda solution, less accurate than bi-directional solutions (10-16 instead of
10-20 for 1,000 km), avoids the problem of coping with transmission devices uncompliant with
DWDM equipment. The metrological signal is transmitted through the whole photonic layer
provided by the NREN/carrier, including through uni-directional EDFA.

Precise time transmission between Praha and Vienna over uni-directional lambdas has operated
successfully since 2011 [42]. Lambdas pass through two different transmission systems, allowing
comparison of national approximation of UTC: UTC(TP) and UTC (BEV).

Figure 4.5: Two uni-directional lambdas used for time transmission between Praha–Vienna, 2 x 550 km

4.1.2 Experimental Results

Paris–Strasbourg: Frequency Transfer

In 2015, a performance evaluation was realised on the first branch of the REFIMEVE+ network, from
Paris to Strasbourg [43]:

 Paris–Nancy–Paris (1,100 km).

 Paris–Strasbourg–Paris (1,480 km): extension of Paris–Nancy link.

On Paris–Nancy–Paris, a frequency stability of 4×10-16 at 1 s integration time was demonstrated,

which averages down to 5×10-20 at 60,000 s integration time. This stability performance allows the
best optical clocks to be compared after only 100 s.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 42
Frequency and Time Distribution

Figure 4.6 shows fractional frequency instability versus averaging time of the Villetaneuse–Nancy–
Villetaneuse 1,100 km compensated optical link (red squares) and of its extension to Strasbourg
(blue circles). The stabilities are calculated from -type data using modified Allan1 deviation.

-15
10

Modified Allan deviation y() -16

10

-17
10

-18
10

-19
10

-20
10 0 1 2 3 4 5
10 10 10 10 10 10
Integration time, , s

Figure 4.6: Fractional frequency instability versus averaging time of REFIMEVE+ network branch

On Paris–Strasbourg–Nancy, long-term stable operation of the link was more difficult than with the
Nancy link. The REFIMEVE+ team observed instabilities of EDFA gains caused by the room-
temperature variations. As the input optical signal can be very weak, the link stabilisation becomes
more difficult to implement. At present this restricts long-term operation of the optical link,
requiring unavoidable readjustment sessions.

Paris–Strasbourg–Braunschweig: Sr Atomic Clock Comparison

The REFIMEVE+ branch Paris–Strasbourg and the PTB link Braunschweig–Strasbourg join at the
RENATER PoP in Strasbourg, where frequency comparison equipment is housed. In 2015 this setup
made it possible to accomplish the first comparison of two strontium atomic clocks by long-haul
fibre links, between two National Metrology Institutes (NMIs).

Figure 4.7: Topology of Paris–Braunschweig link with frequency comparison in Strasbourg

1Allan deviation is a measure of frequency stability in clocks, oscillators and amplifiers. For details see [44].
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 43
Frequency and Time Distribution

The French and German links, described in Figure 4.7, are built upon different transmission
technologies. On the French side, the signal is periodically reamplified by repeater laser stations
(RLS) and EDFA amplifiers, whereas the PTB use fibre Brillouin amplification (FBA).

Two independent frequencies are emitted from the French and German atomic clocks and those are
compared in Strasbourg, where a GPS makes it possible to evaluate the frequency instability SrPTB-
SrSYRTE as represented in Figure 4.8 below [45]:

Figure 4.8: 25-day comparison of Paris and Braunschweig atomic clocks in Strasbourg

The frequency instability can also be expressed with the total Allan deviation, as in Figure 4.9, where
one can see a statistical uncertainty lower than 3x10-17 during measurement over one day, and of
2x10-17 from 5,000 to 50,000 seconds.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 44
Frequency and Time Distribution

Figure 4.9: Frequency instability during two phases of experimentation

The stability of time transfer is usually expressed in terms of Time Deviation (TDEV), introduced for
the first time in All optical two-way time transfer in strongly heterogeneous networks [42]. A
comparison of uncertainty between optical time transfer and GPS-based methods at the link Praha–
Vienna (as in Figure 4.5) is shown in Figure 4.10. The link connects Czech and Austrian laboratories of
the National Time and Frequency Standard. In both of them GPS calibration receivers are installed,
allowing comparison of national representation of timescales: UTC(TP) in Praha and UTC(BEV) in
Vienna. This way optical time transfer (red line) can be directly compared with GPS-based time
transfer. The green line is TDEV of the Common View (CV GPS) method, when the signal of the same
GPS satellite, visible from both sites, is compared. The blue line shows the results of the precise
point positioning (PPP) technique, which uses GPS phase and code observations for data post-
processing.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 45
Frequency and Time Distribution

Figure 4.10: Optical and GPS time transfer Praha–Vienna, 550 km

Time transfer stability measured between two atomic clocks (a caesium clock in Praha and a
hydrogen maser in Brno, with the link as in Figure 4.4) is shown in Figure 4.11. The descending part
of the graph is typical for white phase noise, which is reduced as the averaging interval increases.
For intervals longer than 10 s, the white frequency clock (ascending part of graph) of the used clocks
dominates and therefore the influence of noise produced by the time transfer system is hidden.
According to Figure 4.11, the smallest uncertainty attained for Praha–Brno time transfer is 17 ps
over an averaging interval of 8 s.

Figure 4.11: Time transfer Praha–Brno

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 46
Frequency and Time Distribution

4.1.3 Conclusions: Management of Waves for Frequency Distribution

Transmission using uni-directional waves can be handled with mature technology, using common
alien waves at high performance, which still out-performs GNSS methods.

The precise time and ultra-stable frequency transmission as a bi-directional alien wavelength is in its
infancy. Most deployed links are point-to-point on a fixed wavelength. It is difficult to regenerate a
signal to a different wavelength because of the price of the precise lasers and transceiver stations,
and because most amplifiers and transmission equipment are prototypes and need to be
industrialised.

Consequently, several challenges need to be solved to pave the way to a remotely controlled,
automated and flexible metrological network:

 To be able to “multicast” the ultra-stable frequency to different PoPs (a first experimentation

was performed in 2015 within the REFIMEVE+ project [46]).
 To be able to monitor these signals at the optical layer, given that they are narrow and bi-
directional.
 To create and to test SNMP MIBs and/or an alternative solution to remotely monitor
metrological equipment.
 To find a reliable and cost-effective way to monitor third-party equipment (i.e. metrological
in-line amplifiers) in NREN ILAs.
 To define best practices for metrological network configuration (amplifier gains, optical
powers to be chosen to achieve the best performance without disturbing NREN traffic) in
order to model metrological alien wavelengths on NREN DWDM infrastructures.
 To understand the kinds of incidents that can occur on a metrological network, to train
efficiently the NREN and GÉANT NOCs.

4.2 Time Distribution in Layer 3

For the synchronisation of clocks throughout a network, the precision time protocol (PTP) (IEEE 1588
protocol) was proposed to obtain clock accuracy in LANs in the sub-microsecond range. With such
accuracy, PTP is suitable for measurement and control systems [47] and provides precision beyond
network time protocol (NTP). PTP also has the advantage that costs for GPS receivers can be reduced,
as not all nodes in the network need to have access to GPS signals: PTP slaves (for example, at
locations where GPS signals are inaccessible) can be used, which obtain their clock synchronisation
by using the reference of a GPS-based PTP grandmaster clock at another location within the network.

Since PTP was initially designed for LANs, several experiments were conducted during GN4-1 to
determine the level of clock synchronisation with PTP over wide area networks where longer
distances with higher delays are involved, which will affect PTP’s delay-based algorithm. The
following sections describe experiments over MPLS tunnels, as well as long-term tests of PTP
synchronisation over the Internet.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 47
Frequency and Time Distribution

4.2.1 PTP Validation over MPLS Tunnels

IEEE 1588 / PTP was developed to distribute high-precision time and frequency over packet-switched
networks. Such high precision can only be achieved if all network elements (in particular, all switches
and routers) support PTP on the hardware layer. Currently, most of the installed switches and
routers in a network do not support PTP.

MPLS is a common technology for network providers (e.g. NRENs or commercial providers) to offer
IP or Ethernet services to different customers or separate networks logically, in particular for wide
area networks (WANs). The following measurements investigate the potential and properties of a
typical MPLS service to transport time-sensitive data and the corresponding time synchronisation via
PTP.

Network Environment of the Test and Results

The WAN consists of Cisco routers connected via 10 GE links. An MPLS tunnel was established
between Munich and Frankfurt and between Frankfurt and Berlin. The bandwidth of these tunnels
was not limited. L2 measurements were taken to evaluate the characteristics of these tunnels;
results showed the packet jitter was within a range of 50 – 60 us.

Figure 4.12: Test setup 1 in AES over WAN tests

A linear audio stream (based on AES67-Standard) was used as a time-sensitive data application.

In test setup 1 (shown in Figure 4.12), the PTP grandmaster (GPS synchronised) was located in
Munich. PTP traffic was transmitted in-band, and transmitted at the same tunnel as the audio traffic.
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 48
Frequency and Time Distribution

Every audio source and sink was synchronised via PTP. However, it was not possible to set up an
error-free audio transmission and audio click noises could be heard permanently. As the quality of
the audio signal was distorted to such a high extent, it was not even necessary to use any audio
measurements to evaluate the audio quality.

Figure 4.13: Test setup 2 in AES over WAN tests

In test setup 2 (shown in Figure 4.13), a PTP grandmaster (GPS synchronised) was located at every
location. The time and frequency synchronisation was done out-band via GPS and not via the MPLS
tunnel. In contrast to test setup 1, an error-free audio transmission was possible. Audio clicks were
not heard and measurements of audio quality levels confirmed the subjective impression.

There are two possibilities to explain such different behaviour: first, the PTP synchronisation via an
MPLS tunnel was not precise enough. Second, it could also be the case that PTP implementation of
the audio (AES67) equipment was limited in some way.

In a LAN environment, the AES67 devices worked without error. A back-to-back test of the AES67
devices with series-connected WAN emulator indicated that a PTP packet jitter of 50 us caused
permanent audio clicks as described above.

Currently, the tests described here are being repeated using different AES67 devices. To reduce the
PTP packet jitter, the network design will also be changed.

Existing standards and recommendations do not describe the necessary accuracy of PTP for
provisional audio and video applications.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 49
Frequency and Time Distribution

4.2.2 Long-Term PTP Tests over Standard Internet Connection

Over the course of several weeks, PTP tests were also conducted over the Internet between
Erlangen, Germany and Prague, Czech Republic. A (GPS-synchronised) PTP grandmaster was placed
in Prague and a PTP slave 500 km away in Erlangen used this Czech grandmaster as a reference
signal. When the tests were first started on 21 September 2015, the slave detected the reference
signal within 30 seconds and confirmed a sync. Until the end of the tests on 9 November 2015, the
slave had lost its reference signal four times and changed its state to uncalibrated (Table 4.1).

Reference disconnected Reference (re)detected Time duration Days between glitches

2015-10-07 18:27:29 UTC 2015-10-07 18:30:47 UTC 0:03:18 22 days

2015-11-02 18:50:55 UTC 2015-11-02 18:52:46 UTC 0:01:51 26 days

2015-11-02 18:59:47 UTC 2015-11-02 19:01:29 UTC 0:01:42 0 days

2015-11-04 17:16:33 UTC 2015-11-04 17:30:51 UTC 0:14:18 2 days

Table 4.1: Slave disconnected and reconnected to reference signal

Path delay values and clock offset values were investigated for three periods: 21–25 September, 28
September – 9 October, and 31 October – 9 November 2015. Path delay values between Erlangen
and Prague in these intervals ranged from a minimum of 4.883 ms to a maximum of 26.432 ms. This
maximum path delay occurred on 7 October, when the slave lost its reference to the grandmaster
(Figure 4.14):

Figure 4.14: Offset and path delay on 7 October 2015 between Erlangen and Prague

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 50
Frequency and Time Distribution

Such a jump in delay is indicative of packets suddenly being rerouted over a different path. The
reference disconnect happened as a consequence at the same time (18:50:55).

On 2 November 2015 the slave got disconnected from its reference in Prague twice between
18:50:55 and 19:01:29 UTC. The path delay measurements during that time span show an increased
elevation with higher variation compared to the hours before and after the reference signal was lost
(Figure 4.15):

Figure 4.15: Offset and path delay on 2 November 2015 between Erlangen and Prague

On 4 November 2015 the measurements show that at the time of the sync disconnect of the slave
from the grandmaster no values were registered, neither for clock offset nor for path delay (Figure
4.16):

Figure 4.16: Offset and path delay on 4 November 2015 between Erlangen and Prague

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 51
Frequency and Time Distribution

At this time it is unclear what caused the interruption of measurements and the loss of the reference
signal.

4.2.3 Conclusions and Recommendations

These long-term measurements show that adverse network conditions can lead to a loss of the
reference signal that the PTP slave obtains from the grandmaster clock. Clock offsets in a range of
milliseconds and even seconds are also quite large. The tests should be repeated, placing
grandmaster and slave in closer proximity to each other to see how favourable shorter path delays
may be. More investigations need to be done over varying distances, and also longer distances and
longer timeframes, in order to be able to determine how far away slaves can be placed from PTP
grandmasters and how exactly network conditions such as jitter affect slave calibration to the
grandmaster.

In the audio tests AES67 over WAN described above, there was jitter in the MPLS tunnel as low as 50
– 60 us; nevertheless, it was not possible in the tests to have a clear audio without distorting noise.
More experiments have to follow to determine if such network jitter values are to be blamed for
audio distortions or if different equipment could lead to better results. As existing standards and
recommendations do not describe the level of accuracy of PTP needed for provisional audio and
video applications, many additional tests are needed in various scenarios to be able to narrow down
network jitter or loss requirements.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 52
5 Network Dynamicity
Network dynamicity is the ability of a network to respond and adapt rapidly and optimally to
changing demands and/or conditions in terms of resources, traffic, service usage and failures among
other aspects. It is key to enabling national resource and education networks to meet new and
changing service requirements.

To fully utilise the increased capacity as described in Sections 2 and 3, it should be possible to
orchestrate the network resources, optionally combined with orchestration of compute and storage
resources. Software-defined networks (SDN) have gained momentum, mainly in the packet layer, as
the new paradigm for controlling network resources and to provide programmability of network
functions. The concept of SDN is also moving into the transport layer, which enables integrated,
layerless approaches for restoration, taking both the transport and the packet layer into
consideration.

This section addresses the control and orchestration of transport network resources. First, in
Section 5.1, the basics of traditional SDN and how it applies to transport SDN are reviewed and the
main trends are reported. The focus is then placed on the extensions to the OpenFlow protocol to
support transport networks (Section 5.1.2), before moving up to the orchestration layer where
application-based network operations (ABNO) as an orchestration tool is described (Section 5.1.3)
and a deployed partner-developed demonstrator is discussed (Section 5.1.4). Second, a special use
case to optimise a failure scenario in the NORDUnet network is used to conceptually evaluate the
potential benefits of using transport SDN in combination with an open network operating system
(ONOS) to provide layerless restoration.

5.1 SDN and Transport SDN Review

The availability of the flexible grid technology facilitates finer spectrum utilisation and the evolution
towards elastic optical networks (EON). Deploying elastic optical networks based on the flexible grid
technology, different services can be provided by adapting optical connection frequency slot width
to the requested bitrate. A provisioning algorithm has to compute a route and find a frequency slot
that must be contiguous in the spectrum and the same along the links in its route [48], in the
absence of expensive spectrum converters. In addition, the algorithm has to decide from among
different modulation formats so as to efficiently use optical resources as a function of the length of
the computed route.

Deliverable D13.1
Research and Study Network Technologies – 53
White Paper
Document Code: GN4-1-16-7D602
Network Dynamicity

Notwithstanding those advances, dynamicity at the optical layer has been kept rather limited so far
as a result of the large traffic aggregation performed in the upper network layers. Hence, optical
transport networks are currently statically configured and managed. In fact, long planning cycles
upgrade and prepare optical transport networks for the next planning period, where spare capacity
is usually installed to ensure that traffic forecast and failure scenarios can be supported.

Nevertheless, due to the introduction of new services, such as datacentre interconnection, that
NRENs must provide, huge changes in the traffic, not only in volume but also in its distribution and
dynamicity, can be anticipated.

Forecast
Periodical
Forecast
Periodical Periodical Network
Design
Periodical Network
Design

Network Operation

Network Operation Monitor and Reconfigure / Operation

Measure Re-optimise (Provisioning and Recovery)
Monitor and Operation
Measure In-operation planning
(Provisioning and Recovery)

Figure 5.1: Transport networks lifecycle with (a) dynamicity and (b) in-operation planning

The network lifecycle therefore consists of several steps that are performed sequentially. The initial
step receives inputs from the service layer and from the state of the resources in the already
deployed network and configures the network to be capable of dealing with the forecast traffic, for a
period of time. Once the planning phase produces recommendations, the next step is to design,
verify and manually implement the network changes. While in operation, the network capacity is
continuously monitored and that data is used as input for the next planning cycle (Figure 5.1(a)).

Dealing with traffic dynamicity requires connection provisioning to be automated, which explains
the development of centralised architectures based on the software-defined networking (SDN)
concept.

SDN decouples the network control and forwarding functions, enabling the network control to
become directly programmable and the underlying infrastructure to be abstracted for applications
and network services. SDN is dynamic, manageable, cost-effective and adaptable, making it ideal for
the high-bandwidth, dynamic nature of today’s applications.

Operating the network dynamically might bring cost savings, but it also might cause non-optimal
network resource utilisation. To solve that, network resources can be made available by applying in-
operation network planning [49], i.e. by reconfiguring and/or re-optimising the network on-demand,
which leads to an extended network lifecycle (Figure 5.1(b)).

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 54
Network Dynamicity

5.1.1 Developments in Transport SDN

The evolution of SDN and its further advancement have played a most significant role in the
development of next-generation packet networks to meet the upcoming social needs of the growing
population. SDN as a programmable network having “service provisioning on the fly” has induced a
keen interest both in the academic world and in industry.

The concept of SDN derives its basis from the computer engineering area, where an abstraction-
based architecture has simplified the programming problems of writing and maintaining software.
Abstraction is the process by which data and programs are defined with a representation similar in
form to its meaning, while hiding away the implementation details. Different levels of abstraction
can be defined, providing different amounts of detail (high and low levels) and, in turn, creating
different interfaces (instances of abstraction). The objective of the SDN paradigm is realised by
redesigning the architecture of networks in order that they operate in a similar way to that of
computing architectures.

Figure 5.2 shows a logical view of the SDN architecture. Three layers can be identified in this
architecture: an application layer, a control layer, and a data layer. Hence, the SDN paradigm
envisions a network architecture where the network devices (routers, switches, optical nodes, etc.)
become programmable. This objective is realised by introducing appropriate levels of abstraction
that are accessible by applications through the use of control interfaces, specifically, application
programming interfaces (APIs). The interface between the application layer and control layer goes
under the name of northbound interface (NBI) and is specified by the API, while the interface
between control layer and data layer is the southbound interface (SBI) defined by various protocol
specifications, OpenFlow being the most popular.

Application Application Layer

Application
Application

NBI

Control Layer
SDN Controller

SBI (OpenFlow)

Network device
Data Layer
Network device Network device

Figure 5.2: SDN architecture

Network intelligence is (logically) centralised in SDN controllers, which maintain a global view of the
network. As a result, the network appears to the applications and policy engines as a single, logical
switch. With SDN, enterprises and carriers gain vendor-independent control over the entire network
from a single logical point, which greatly simplifies the network design and operation. SDN also
greatly simplifies the network devices themselves, since they no longer need to understand and

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 55
Network Dynamicity

process thousands of protocol standards but merely accept instructions from the SDN controllers. A
comparison between conventional networking and SDN is shown in Table 5.1.

Characteristics Conventional Networking Software Defined Networking

Features Data and control planes are customised in Segregates the data plane from the control
each node. For each problem a new plane with a centralised programmable
protocol is proposed with complex controller that makes the network control
network control. simple.

Configuration When new equipment is added into the Unification of the control plane over all
existing network, the heterogeneity of the kinds of network devices, including routers,
network devices’ manufacturers and switches and load balancers, permits
configuration interfaces requires a certain automated configuration with centralised
level of manual configuration procedures, validation via software controlling. As such,
which is tedious and error prone. an entire network can be programmatically
configured and dynamically optimised based
on network status.

Performance Due to heterogeneity among networking Provides an opportunity to improve network

devices and coexistence of various performance globally with centralised
technologies, optimising the performance control and by having a feedback
of the network as a whole is difficult. mechanism to exchange information among
different tiers of the networking
architecture.

Innovation Difficulties are encountered while Comparatively, SDN encourages

implementing new ideas and design due implementation of new ideas, applications
to widely used proprietary hardware, and new revenue-earning services
which prevents modification for conveniently and flexibly through the
experimentation and adoption. programmable network platform.

Cost Both data and control planes are Data plane and control planes are
embedded on the switches, thereby decoupled from each other which makes
making the switch more complex and the structure of the switches simpler and
costly. easier to manufacture, which in turn leads
to a lower-cost solution.

Table 5.1: Conventional networking and SDN comparison

The SDN approach has become very natural for store and forward packet networks, especially in the
core-metro segments and, more recently, also in the access segment. The extension of the SDN
architecture for optical networks poses several challenges. One of the key challenges in designing
and operating optical networks is the interaction, control and management between the optical and
packet layers.

Multiple standards bodies are currently working on developing SDN standards for the optical
network. The two main standards/protocol efforts related to SDN in the optical network are
generalised multi-protocol label switching (GMPLS) and OpenFlow:

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 56
Network Dynamicity

 GMPLS is often used as the control plane for optical switching; it is a proposed IETF standard
to simplify the creation and management of IP services over optical networks, and it is often
used as the control plane for optical switching.
 SDN/OpenFlow was designed specifically for packet transport and needs modification to
provide SDN-like control to the optical layer.

The Optical Transport working group within the Open Networking Foundation (ONF) is addressing
SDN/OpenFlow control capabilities for optical transport networks [50]. The work includes identifying
use cases, defining a target reference architecture for controlling optical transport networks
incorporating OpenFlow, and creating OpenFlow protocol extensions. The identified use cases are:

 Direct control of optical components in enterprise networks.

 Carrier Ethernet network virtualisation.
 Service provider datacentre interconnection.
 Packet-optical integration.

Nevertheless, services such as content distribution, distributed databases, or inter-datacentre

connectivity place a set of new requirements on the operation of networks. They need on-demand
and application-specific reservation of network connectivity, reliability and resources (such as
bandwidth) in a variety of network applications (such as point-to-point connectivity, network
virtualisation or mobile back-haul) and in a range of network technologies from packet down to
optical. An environment that operates to meet these types of requirements is said to have
application-based network operations (ABNO) [51]. ABNO brings together many existing
technologies and may be seen as the use of a toolbox of existing components enhanced with a few
new elements. Some identified use cases are:

 Inter-AS connectivity.
 Pseudo-wire operations and management.
 Multi-layer networking.
 Network re-optimisation and adaptive network management.
 Cross-stratum optimisation.
 Virtual network topologies operations and management.

ABNO includes a provisioning manager responsible for the establishment of connections either by
giving instructions to a GMPLS control plane running in the networks or by programming individual
network devices. In the latter case, the provisioning manager would act as an OpenFlow Controller.

In addition, the current network architecture will need to evolve to include a functional block
between the service layer and the network elements to support multi-service provisioning in multi-
vendor and multi-technology scenarios. Two standard interfaces are required. First, the northbound
interface (NBI), which, among other tasks, gives an abstracted view of the network, enabling a
common entry point to provision multiple services and to provision the planned configuration for
the network. Moreover, this interface allows coordination of the network and service layer
according to service requirements. Second, the southbound interface (SBI), covering provisioning,
monitoring and information retrieval.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 57
Network Dynamicity

5.1.2 ONF View on Transport Extensions

The potential of SDN, owing to its separation of control and data layer, is already visible in the
packet world, with major vendor and service providers stepping forward to support products and
services based on SDN technologies. The advantages of SDN for optical transport over existing
control plane solutions, namely GMPLS, have been shown to be substantial with regard to several
specific aspects. An SDN-based architecture can simplify the complexities of handling traffic among
various networking technologies. It allows the underlying infrastructure to be abstracted and used
by applications and network services as a virtual entity. This allows network operators to define and
manipulate logical maps of the network, creating multiple co-existing network slices (virtual
networks) independent of underlying transport technology and network protocols. Furthermore, the
separation of the control plane and data plane makes SDN a suitable candidate for an integrated
control plane supporting multiple network domains and multiple transport technologies. The
architecture is not only well suited to address the present optical networking problems related to
supporting different administrative and technology segments, bridging the gap between packet and
optical layers, but it also supports new operations such as virtualisation, cross-layer orchestration,
bandwidth on demand (BoD), load balancing and many more.

Due to the unique analogue features of the optical layer, various static and dynamic attributes and
properties (e.g. modulation format, capacity, power, impairments) need to be considered in the
process of optical resource abstraction. Depending on the type of the optical transmission and
networking elements involved and the requirements of the upper layer operations, different levels
of abstraction (i.e. the amount of information to be exposed to the upper layer) need to be defined.

One of the key challenges of optimally using the optical infrastructure is to create an abstracted
optical resource model and describe it in a way that is simple enough for the higher control layer to
utilise it while capturing the true properties of the distributed non-linear, noisy and dispersive
analogue optical channel. Existing models are often conservative or over-simplified, and targeted to
direct-detection systems, now superseded by digital coherent transmission systems. A key challenge
here is to develop simplified models for the optical physical layer, starting from an understanding of
the underlying physics of the optical channel and the optical layer analogue characteristics.

Within this technological context, the target of a unified, centralised control plane involves, on the
one hand, extending the OpenFlow protocol to support circuit switching (that is, where a dedicated
communications channel or circuit is established between end points) and, on the other hand,
ensuring that operator use cases and workflows are fully covered. This is of importance since the
operation of an optical transport network is significantly different to the operation of a packet
switched network. For example, the former usually involves operator intervention before
provisioning a new optical connection, and the lifetimes of services are on different timescales.

From a standardisation perspective, extending OpenFlow for optical networks is perceived to be a

difficult task, given the intrinsic complexity of the optical technology. One of the reasons is that
there is no common, comprehensive, abstract model of optical devices, such as reconfigurable
optical cross-connects, nor the associated information and data models. That said, initial efforts are
being made by the ONF Optical Transport working group and other standards-developing
organisations, as well as in related research initiatives. In fact, the ONF Optical Transport working
group has been working internally on preliminary OpenFlow extensions for multi-technology
switches with GMPLS label encodings, complementing and adapted to OpenFlow 1.1. It is expected
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 58
Network Dynamicity

that newer versions will improve the support of optical networks, in order to address, in a more
comprehensive way, identified requirements both for fixed- and flexi-grid, notably filter
configuration and configuration of cross-connections of media channels in media matrices.

The northbound abstraction is crucial not only for defining application-based services and policies
but also for integrating different technology and administrative domains, which are typical of
optical-based core networks. The two main aspects to consider for the NBI are the set of
functionalities that are exposed to external applications, together with the different interfaces that
determine the different mechanisms used to interact with them.

The application abstraction is a common-map abstraction, based on a data abstraction of a network-

wide common map manipulated by a network API. The common map has full visibility into both
packet and circuit switched networks, allowing creation of network applications that work across
multiple layers and multiple optical technologies. Full visibility allows applications to jointly and
globally optimise network functions and services across multiple layers and technologies.
Implementing network functions as centralised applications is simple and extensible, as the
common-map abstraction hides the details of state distribution (including east-west API for multiple
controllers) from the applications, allowing unified operation over multiple layers.

One way to represent information to applications is as graphs. A few implementations of transport

network controllers, such as the open network operating system (ONOS) [52] and packet and circuit
network convergence (PAC.C), use network graphs as the method to define northbound abstraction.
These graph abstractions help optical network control and management functions to utilise the
wealth of graph computation algorithms that are widely used in the optical world. Another added
benefit is to use existing and emerging PCE orchestration architectures to allow applications to
seamlessly define network requirements over multiple administrative and technology domains.

See also Appendix B for the latest document about OpenFlow transport extensions from ONF.

5.1.3 ABNO and Its Role as Orchestrator

Standardisation bodies, especially the IETF, have been working to address operators’ and NRENs’
requirements and, as a result, the application-based network operations (ABNO) architecture is now
being proposed as a candidate solution. The ABNO architecture consists of a number of standard
components and interfaces which, when combined together, provide a method for controlling and
operating the network. A simplified view of the ABNO architecture is represented in Figure 5.3. It
includes:

 The ABNO controller as the entrance point to the network for NMS/OSS and the service
layer for provisioning and advanced network coordination. It acts as a system orchestrator,
invoking its inner components according to a specific workflow.
 The path computation element (PCE) defined as an entity to serve path computation
requests. The PCE protocol (PCEP) might be used to carry path computation requests and
PCE responses.
 The virtual network topology manager (VNTM) coordinates virtual network topology (VNT)
configuration by setting up or tearing down lower-layer connections, and advertising the
changes to higher-layer network entities.
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 59
Network Dynamicity

 The provisioning manager is responsible for the establishment of connections. This can be
done by interfacing the control plane using PCEP or by directly programming the data path
on individual network nodes using the network configuration protocol (NetConf) or acting as
an OpenFlow controller.
 The operations, administration and maintenance (OAM) handler is responsible for detecting
faults and taking actions to react to problems in the network. It interacts with the nodes to
initiate OAM actions such as monitoring and testing new links and services.

OSS/NMS / Application Service Orchestrator

Policy ABNO Controller

Agent OAM
Handler

ALTO
VNTM
Server Back-end
PCE I2RS
Client
Front-
Topology end PCE
Module

Provisioning Manager

Client Network Layer

Server Network Layer

Figure 5.3: ABNO architecture

A multi-domain optical transport network composed of heterogeneous optical transport

technologies (e.g. flexi-/fixed-grid optical circuit switching and optical packet switching) and control
plane technologies (e.g. centralised OpenFlow or distributed GMPLS) does not naturally interoperate,
and a network orchestration mechanism is required. A network orchestrator allows the composition
of end-to-end network service provisioning across multi-domain optical networks comprising
different transport and control plane technologies. SDN is a key technology to address this
requirement, since the separation of control and data planes makes the SDN a suitable candidate for
end-to-end provisioning service orchestration across multiple domains with heterogeneous control
and transport technologies.

The ABNO architecture is the ideal element for such orchestration. Figure 5.4 presents the building
blocks of the ABNO architecture that are required to support the multi-domain and multi-layer
network orchestration. The orchestration controller runs the different workflows and can interwork
with the different blocks. The topology module gathers the domain topology of each SDN/OpenFlow
domain controller exposed by the NBI and/or the GMPLS/PCE domain through new protocols such as
BGP-LS to obtain the traffic engineering database (TED) by BGP peering. The TED stores the global
network topology, which is composed of the intra-domain topologies with the inter-domain links
connecting them. The PCE handles the path computation across the network graph provided by the
topology module and needs to be extended to deal with OpenFlow datapath identifiers. The
provisioning manager is responsible for the actual flow-establishment requests to the OpenFlow
domain controllers through each specific controller’s NBI, and to the AS-PCE of the GMPLS domains.
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 60
Network Dynamicity

The flow server is responsible for storing the state of the provisioned flows in the label switch path
database (LSPDB). Finally, the VNTM is responsible for coordinating the path provisioning in multi-
layer networks by performing the layered establishment of connections in the server layer (e.g.
optical connections) and its promotion as a logical link in the client layer (e.g. IP). It is worth
mentioning that the NBI of the SDN/OpenFlow or GMPLS/PCE controllers is typically technology and
vendor dependent. Thus, the network orchestrator shall implement different plugins for each of the
controllers’ NBIs.

ABNO-based orchestrator

Orchestration Controller

VNTM Topology Flow

PCE
Module Module
TED LSPDB

Provisioning Manager OAM Handler

REST/API REST/API
PCEP
BGP-LS GMPLS
NBI API Controller NBI API

SDN Controller PCE SDN Controller

GMPLS GMPLS
OpenFlow Controller Controller OpenFlow

Figure 5.4: ABNO-based network orchestrator

5.1.4 ABNO-Based Integrated Demonstrator

An emulated environment is being developed within JRA1 Task 1. Figure 5.5 shows the optical
domain architecture where an SDN controller has been deployed and is being studied. The SDN
controller has a database storing the network topology, the state of the network resources, and the
already established connections (label switched paths (LSPs)).

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 61
Network Dynamicity

Network Orchestrator

REST/API

NBI API

SDN Controller

OpenFlow

OFP

Optical Network

Figure 5.5: Optical domain demonstrator architecture

To control the network elements from the SDN controller, the OpenFlow protocol (OFP) has been
used. Since OFP v1.4, new properties in support of optical ports have been added, allowing
configuration and monitoring of the laser in the transmitter, and of frequency and power in the
receiver. These properties support optical ports in optical cross-connects (OXC). However, they do
not include modulation format support for the transmitters. In view of that, the Task has
implemented extensions to OFP v1.4 to support modulation formats in the form of EXPERIMENTER
properties. The following sub-sections define the extensions implemented in OFP v1.4.

Port Description
Regarding the port description, the standard ofp_port_desc_prop_optical currently retrieves
the port features (RX tunable, TX tunable, TX power adjustable, values specified in frequency or
wavelength units), minimum and maximum TX and RX frequency/wavelength, TX and RX grid spacing
in frequency or wavelength units, and minimum and maximum TX power. To extend those, Task 1
has implemented a new experimental property to obtain the modulation formats supported by the
port. The property is named ofp_port_desc_prop_experimenter_modulation_formats and
its structure is as follows:

struct ofp_port_desc_prop_experimenter_modulation_formats {
uint16_t type; /* OFPPDPT_EXPERIMENTER. */
uint16_t length; /* Length in bytes of this property. */
uint32_t experimenter; /* Experimenter ID. */
uint32_t exp_type; /* Type=1 Modulation Formats Description. */
uint8_t num_mod_formats; /* Number of modulation formats supported. */
uint32_t mod_formats[0]; /* List of modulation formats supported. Each following
structure
ofp_port_desc_prop_experimenter_modulation_format. */
uint32_t pad[0]; /* 64-bit alignment */
};

struct ofp_port_desc_prop_experimenter_modulation_format {
uint16_t id; /* Identifier of the modulation format. */
uint16_t bits_per_symbol; /* Spectral efficiency in bits per baud. */
};

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 62
Network Dynamicity

Port Configuration
Port configuration messages currently support the definition of configuration flags, central
frequency, offset, slot width and transmission power of the port, but not the desired modulation
format configuration. The Task has implemented a new experimental property to set the desired
modulation format of the port. The property is named
ofp_port_mod_prop_experimenter_modulation_format and its structure is as follows:

struct ofp_port_mod_prop_experimenter_modulation_format {
uint16_t type; /* OFPPMPT_EXPERIMENTER. */
uint16_t length; /* Length in bytes of this property. */
uint32_t experimenter; /* Experimenter ID. */
uint32_t exp_type; /* Type=1 ModulationFormatConfiguration. */
uint16_t mod_format_id; /* Modulation format id. One from Modulation Formats
Description. */
float_t symbol_rate; /* Symbol rate in bauds/s formatted as an IEEE 32 bit
float */
uint16_t num_subcarriers; /* Number of subcarriers used. */
uint16_t bits_per_symbol; /* Bits per symbol as reported in Modulation Formats
Description. */
uint32_t pad[0]; /* 64-bit alignment */
};

Optical Frequency Slot Matching

The optical label for WDM specifies the grid identifier, the channel spacing and the nominal central
frequency index, as defined in IETF RFC6205 [53]. In the case of the flexible grid technology, an extra
field defining the slot width has to be included, as defined in [54].

In the demonstrator, the Task has used a 20-bit MPLS-like label, the structure of which actually
depends on the technology used.

0 1 0 1

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

|GR | CS | n | t | RES | |GR | n | m | t |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

WDM label format Flexible grid label format

The fields in the label are as follows:

 Grid (GR, 2 bit): Supported values are 01 for ITU-T DWDM, 10 for ITU-T CWDM, 11 for flexible
grid and 00 is a reserved value.
 Channel Spacing (CS, 3 bits): Supported values are 001 for 100 GHz, 010 for 50 GHz, 011 for
25 GHz, 100 for 12.5 GHz, 101 for 6.25 GHz, and 000 is a reserved value. For flexible grid
technology, this field is assumed to be 6.25 GHz.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 63
Network Dynamicity

 Nominal Central Frequency index (n, 9 bits): The maximum spectrum width is 4 THz; when
using 6.25 GHz slices, 640 possible indexes need to be representable. The reference central
frequency, e.g. 193.1 THz, corresponds to n=0; n is a signed integer in two-complement
notation, thus requiring 9 bits for its representation. The nominal central frequency is
computed as 193.1 THz + n x CS / 1000.
 Slot Width (m, 6 bits): This field is used only in flexible grid technology. The field size for m
has been computed to allow encoding a 1 Tb/s connection that, using the worst modulation
format, e.g. QPSK, requires 20 frequency slices of 12.5 GHz.
 Ethernet Type (t, 3 bits): This field is not required by the IETF label format, but it is required
in Task 1’s integrated demonstrator since, when enveloping an Ethernet frame in an MPLS
frame, its Ethernet type, e.g. ARP, IP, IPv6, IGMP, MPLS, etc. field is substituted by MPLS,
thus preventing the reconstruction of the original frame at the receptor. For this reason, a
field used to encode the original frame protocol was added. Supported values are: 0000 for
ARP, 0001 for IP, 0010 for IPv6, 0011 for IGMP and 0100 for MPLS. Remaining values are
reserved for future use.

End-to-End Management
In the case of a multi-domain/-layer network, a network orchestrator needs to be configured on top
of the set of single-layer/-domain SDN controllers. To that end, the SDN controllers must export the
underlying network topology and resources to the controller, so that an end-to-end view of the
network can be created. The IETF is currently working on defining data models to specify optical
networks based on YANG. Since YANG model definitions are still not defined, the Task decided to
implement a REST/API in the demonstrator, so that the SDN controllers can export their topology
and resources.

A different REST/API needs to be developed to set up, update or tear down an LSP. For the set-up
operation an LSP identifier (symbolic path name) the end-points of the LSP and the explicit route
object (ERO) needs to be provided. For the update operation, the LSP’s symbolic path name and the
new ERO are enough. For the tear-down operation, the LSP’s symbolic path name is enough.

Demonstrator Deployment
The demonstrator to test the single-layer/-domain functionality described above has been
experimentally implemented using Ryu SDN controller v3.25, since it supports all available OFP
versions as well as the standardised optical port extensions. Controller applications developed for
the Ryu controller use the cookie field in each flow configured in the network elements to identify
the LSP for which that rule is being configured. The controller application has been implemented in
Python and was configured with a northbound interface in the form of a REST API used to receive
requests from an orchestrator. In addition, a southbound interface to control the underlying
network elements has been configured to use OFP v1.4 with extensions to support WDM and
flexible grid technologies.

Mininet v2.1.0 has been used to orchestrate the deployment and configuration of each network
element in the topology. Each network element has been deployed using an OpenVSwitch v2.4.0
bridge with the optical extensions for WDM and flexible grid technology emulation. The Task
emulated the transmission of an optical signal by encapsulating each Ethernet frame being

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 64
Network Dynamicity

converted from electrical to optical into an MPLS-like frame where the MPLS label encoded the
optical spectrum label. Intermediate OXCs switched the frame according to that MPLS label and, at
the receiver, the MPLS label is removed to restore the original Ethernet frame.

Network Orchestrator LSP:

id = “X1-X3-1”
src = X1/TP1
REST/API
dst = X3/TP2
NBI API ero = X1/P1 – X2/P2 – X3
label = n, m
L0 Ryu SDN Ctrl

OpenFlow
OFP

TP1 P2
P1 TP2
P2 X2
P1
X1 X3
Optical Network

Figure 5.6: Example of LSP setup

When the network orchestrator needs to set up an LSP, it computes the LSP’s explicit route object
(ERO), containing the resource assignment for that LSP along a route. For instance, in Figure 5.6, the
LSP to be set up is between X1/TP1 and X3/TP2. Upon the reception of an ERO in the SDN controller,
the ERO is used to configure the transmitter and receiver transponders and contains the source and
destination end points specifying the node and its port. In the example in Figure 5.6, the source port
is transponder TP1 in X1 and the destination port is transponder TP2 in X3. The ERO defines the set
of individual connections to be set up in every OXC along the route and includes a label defining the
frequency slot that must be used for the LSP along the route.

5.2 The NORDUnet UNINETT Use Case

This use case was selected for investigation because it represents a typical situation in a packet-
optical network, where routing might not be optimal because of lack of coordination between the
path calculation elements of different network layers.

5.2.1 The Problem

NORDUnet use three routers (R-USI, R-OSC, and R-TUG) for providing IP connectivity to UNINETT
(Figure 5.7)2.

2The case description was provided by Alberto Colmenero and Rasmus Lund from NORDUnet.
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 65
Network Dynamicity

Figure 5.7: Packet and optical layers of NORDUnet

IP-traffic from NORDUnet to UNINETT is aggregated by R-TUG and sent to a UNINETT destination
either through the R-TUG–R-USI primary connection (cost 10) or the R-TUG–R-OSC primary
connection (cost 100). Both connections use the OTN/DWDM infrastructure circuits and have 10G
capacity. The R-TUG–R-USI connection uses the direct TUG–USI circuit while the R-TUG–R-OSC
connection uses the compound circuit TUG–FRE–ORE–UNI–OSC. The cost of the connections reflects
the characteristics of the circuits they use, such as distance, number of spans, delays; that is why the
cost of the R-TUG–R-OSC connection is 10 while the cost of the R-TUG–R-OSC connection is 100.

Optical boxes (TUG, USI, OSC and others) have 10GE tributary ports to provide connections with IP
routers and 100G line ports with ODU4 framing for inter-connections. Within optical boxes, IP traffic
maps onto ODU2e frames that are then multiplexed into the respective time slots of the ODU4
signal.

Under normal conditions, router R-TUG sends IP packets to UNINETT through router R-USI, as the
cost of this route (10) is less than the cost of the alternative route through router R-OSC (100).

The problem with a non-optimal routing arises when an outage in the fibre between USI and TUG
takes place. The OTN layer of the NORDUnet optical network has a GMPLS control plane, which is
used for rerouting OTN circuits in the event of fibre and wavelength faults. So, when the OSPF
protocol detects a loss of connectivity along the TUG–OSC OTN circuit, it recalculates a route and
sets up a backup one. In this example case, the backup circuit goes around the fibre ring along the
TUG–FRE–ORE–UNI–OSC–USI route.

For the IP routers, the rerouting happening at the OTN layer remains invisible and router R-TUG
keeps sending IP packets towards UNINETT through router R-USI, as this route still has a cost of 10.
In fact, after the rererouting, router R-TUG should have sent traffic to UNINETT through router R-
OSC, as this route at the OTN layer has better characteristics (hop number, distance, delay) than the
route through R-USI. However, to do so, the routers would have to have known the corrected costs
of the OTN circuits, but such an information exchange in automatic mode between the IP and OTN
layers is not supported by NORDUnet network equipment (Juniper routers and Ciena OTN/DWDM
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 66
Network Dynamicity

boxes at the moment. As a result the route from R-TUG towards UNINETT destinations remains
non-optimal, one optical hop longer than an alternative route through R-OSC and hence resulting
in longer delays of signal propagation along a route. Such a situation takes place until the manual
intervention of an IP administrator who corrects the route costs according to the information passed
to him or her by an ONP/DWDM network administrator – an informal procedure, which might take
too much time and involve mistakes.

5.2.2 The SDN Solution

The problem described above is not specific to NORDUnet, as historically the IP control plane
worked on its own, without any interaction with the optical layers. The following sub-sections
summarise the current situation and developments with regard to layer interaction to establish the
context for then considering SDN as a natural way to effective routing in multi-layer packet-optical
networks.

Manual Interaction of Layers

Usually, the packet and optical networks of a network provider (if it owns both) are managed by two
different teams of network administrators (sometimes by two different Network Operations Centres
(NOCs)). The optical network administrators establish circuit-based connections for IP routers
according to requests from the packet network administrators, who specify the end points and
bandwidth of a connection. If the packet network administrator who makes such a request takes
into account the topology of the optical network, the resulting route might be optimal or close to
optimal. However, if such a request is made in “blind” mode, i.e. without knowledge of the optical
network topology, the result could be quite far from optimal.

GMPLS-Based Layer Interaction

The introduction of GMPLS brought an automatic interaction between the control planes of packet
and optical networks. However, the main model of the IP-GMPLS interaction functionality, called the
overlay model, is very close to the manual blind mode described above. In the overlay model, the IP
control plane has a limited view of the optical network topology, seeing only the entry points of the
optical network, to which routers are connected, and makes a request to the optical GMPLS control
plane to set up a circuit between two of those entry points. The effectiveness of the resulting
combined route going through IP-OTN-DWDM layers as in the manual blind mode depends to some
extent on luck.

Another GMPLS model – the peer model – assumes that all the nodes of both networks have full
knowledge about the layerless topology of the combined packet-optical network. In such a case the
routing might be optimal; it depends on the routing protocol used but, in principle, having the full
topology information available at each network node potentially allows effective routing to be
achieved. The implementation of the peer GMPLS model faces several challenges:

 The IP and underlying optical network can be under different administrative control – such a
situation was very typical in the past; now more and more providers own and control both
types of networks.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 67
Network Dynamicity

 All network nodes – IP routers and OTN/DWDM switches – have to support GMPLS protocols;
for routers this is not common, as GMPLS protocols differ from the standard IP control plane
protocols.
 The complexity of the path calculation task grows significantly with the addition of extra
nodes and links to the combined network topology graph; this could be too heavy a task for
network nodes.

Because of these challenges, development of the GMPLS technology and its applications mostly
stuck to the overlay model and hence cannot make use of the potential benefits of routing based on
a global view of the packet-optical network topology.

SDN-Based Layer Interaction

The SDN paradigm assumes the logically centralised control of a network. One of the consequences
of this paradigm is that it provides a natural way to control all network layers from a single point (the
SDN controller) that has a global view of the network as a whole and optimises its behaviour
accordingly.

The global view of a packet-optical network can be achieved through two kinds of generalisation [55]:

 A common-flow abstraction that generalises packet flows and circuit flows.

 A common-map abstraction that allows a packet-optical topology to be presented as a flat
layerless map.

The benefits of a global view of a packet-optical network are multi-faceted as such a view allows
different aspects of network behaviour to be optimised, including establishing optimal routes for
compound (packet-circuit-packet) flows on an end-to-end basis.

SDN control of a packet-optical network based on a generalised layerless topology is similar to the
GMPLS peer model approach but it copes better with the challenges described above:

 Network nodes do not need to run complex GMPLS protocols. Instead, they need only
comply with an SDN switch specification (e.g. from ONF) and support an SDN controller-
switch protocol (e.g. OF).
 The increased complexity of the path calculation due to the increased dimension of the
combined network graph is not a big problem for the centralised computational power of an
SDN controller: to deploy one or several very powerful computers running a path
computation element (PCE) application is much simpler than increasing the computational
power of each network node.

Of course, as with a GMPLS peer model, an SDN layerless global approach cannot help if the packet
and optical networks are under different administrative control.

SDN-based control of a packet-optical network can use an overlay approach too. In such a case there
are two separate topology databases, one for the packet network and one for the optical network
[51]. Two different SDN controllers control their respective networks separately but coordinate their
work if necessary. For example, when an IP SDN controller needs to set up a new link between

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 68
Network Dynamicity

routers, it makes a request to an optical SDN controller specifying the end points and bandwidth of a
connection.

Taking an SDN Layerless Approach to the NORDUnet Use Case

Taking an SDN layerless approach to the NORDUnet use case results in the topology shown in Figure
5.8. The topology includes both physical and virtual elements of the combined network. The physical
elements are represented by physical ports and links – fibres between routers and tributary 10GE
ports of the optical nodes. Virtual elements are wavelengths of the DWDM and time-slots of the
OTN layers and virtual ports.

A virtual port maps data between layers. For example, virtual port 1 of the TUG optical node maps
incoming packets of a packet flow onto ODU2e frames. Virtual port 2 of TUG maps ODU2e data onto
time-slots of an ODU4 frame of one of the wavelengths outgoing from one of the TUG 100G ports.

Figure 5.8: A global view of the NORDUnet layerless topology

An example of several first hops of a path of flow 3 going through R-TUG packet port 2 and then
through TUG and FRE towards R-USI might look like this:

R-TUG {Flow3, Port2} -> TUG {Port2->VP1} -> TUG{VP1-mapping->ODU2e}

-> TUG{ VP1->VP2} ->TUG{VP2-multiplexing-> (ODU4, time-slots k, …,
j)} -> TUG{VP2-mapping ->(3,Port2)} -> FRE{(Port1, l3)-> (Port2,
l5) -> …

This example shows that the dimension of the combined layerless graph of a network topology
grows not only because the combined topology includes more physical nodes and links than each
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 69
Network Dynamicity

separate topology (packet or optical) on its own. This growth is also the result of multiple choices,
which produce some mapping and multiplexing operations. For example, an ODU2e frame can be
mapped onto different wavelengths of output port 2 and the number of choices depends on the
availability of eighty time-slots in an ODU4 frame of each wavelength.

The global view of a combined packet-optical NORDUnet topology gives the PCE element of an SDN
controller the possibility to find an optimal route from R-TUG towards UNINETT in accordance with a
metric, which can be either quite traditional, such as the number of physical hops, or more
sophisticated, taking into account signal delays, BER, etc.

If the number of hops is used as a metric, then it is not difficult to see that the fibre outage in
question (i.e. between USI and TUG) should cause a rerouting of flows going from R-TUG towards
UNINETT through R-OSC and not through R-USI as happened in reality, because the former route has
one hop fewer.

5.2.3 Simulation of the NORDUnet Use Case in the ONOS/Mininet

Environment
The purpose of this simulation was to find out how effectively existing SDN controllers can make
routing decisions for packet-optical networks using the NORDUnet use case as a reference topology.

The open network operating system (ONOS) [52] was chosen for this simulation because:

 It is a very popular open SDN controller from the Open Networking Lab (ON.Lab) – a non-
profit organisation founded by SDN inventors and leaders from Stanford University and the
University of California, Berkeley.
 The ONOS distribution includes a packet-optical tutorial application that supports layerless
routing.
 It is integrated with Mininet [56] and LINC Open Extension (LINC-OE) switch [57], which
simulates a DWDM multiplexor and OF v.1.3 protocol.

Unfortunately, Task 1 failed to find a software implementation that simulates a switch with OTN
capabilities as is required for the NORDUnet use case. LINC-OE software turned out to be the most
suitable to our purpose as it supports multiplexing and cross-connecting wavelengths in the fixed
grid environment (80 wavelengths per port). LINC-OE software was written before the ONF
published its Optical Transport Protocol Extensions v.1.0, so it uses its own version of the extensions
10 OF 1.3 that are supported by the ONOS SDN controller.

However, despite the lack of OTN functionality, the two-layer topology (IP and DWDM) allows (to
some extent) simulation of the routing decisions taking place in the three-layer NORDUnet network.

If it is assumed that ODU4 frames of all 80 wavelengths of each optical 100G port of the NORDUnet
network always have 8 spare time-slots required for multiplexing a tributary ODU2e frame then the
selection of an optimal route will not depend on the OTN layer details and will depend only on the
wavelengths routes. The NORDUnet OTN and DWDM infrastructures were not heavily used at the
time of this simulation, so the assumption is very likely to be reasonable and the simulation of the
two-layer network should have given some plausible results.
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 70
Network Dynamicity

Figure 5.9 below shows the ONOS topology view of the simulated NORDUnet case. The topology
includes three routers (R-TUG, R-USI, and R-UNI) that are simulated by the standard Mininet switch
software. Six NORDUnet optical nodes (TUG, FRE, ORE, UNI, OSC and USC) are simulated by LINC-OE
software. Router R-UNI was added to the topology to represent an IP destination in UNINETT so that
the ONOS PCE could make end-to-end routing decisions. R-UNI is connected by two packet links to
the NORDUnet routers R-USI and R-OSC to provide diverse routes from NORDUnet to UNINETT.
Router R-UNI does not correspond to any real UNINETT router (in contrast to the other topology
nodes); it was introduced just to have some sink point in UNINETT.

Several Mininet hosts were also added to the topology to simulate end-to-end flows, which trigger
the ONOS intent mechanism which in its turn evokes the ONOS PCE.

The topology view reflects the fact that ONOS treats a packet-optical network as layerless, putting
the nodes of different types at the same level.

Figure 5.9: The primary route between R-TUG and a UNINETT router

The route found by the ONOS PCE for a situation where there is no outage in the network is shown
in Figure 5.9 by the yellow-purple line. The yellow hops correspond to packet flows while purple
ones are optical flows. The route has a minimal number of hops between end hosts among all
possible routes between R-TUG and R-UNI.

The result of rerouting flows between R-TUG and R-UNI after the outage is shown in Figure 5.10. The
outage was simulated by changing the state of the TUG 100G port facing USC to “down” in the
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 71
Network Dynamicity

system config files of the respective LINC-OE switch. The ONOS PCE found the new route that goes
through R-OSC and not through R-USI as happened in reality. In other words, the ONOS PCE found
(taking into account the combined layerless topology) the route that was one hop better than the
route that was found by the isolated OSPF protocol, which saw only the OTN layer topology.

Figure 5.10: The backup route between R-TUG and a UNINETT router

The results of the simulation look quite encouraging, as they show that the modern SDN controller –
ONOS in this case – can work effectively in a packet-optical environment using a global view of the
combined network. At the same time it is quite obvious that further developments in this direction
are needed, such as the ability to work with OTN switches, and the ability to specify more
sophisticated metrics than just hop numbers and some others.

5.3 Conclusions
Software-defined networking, now that it has moved to the transport layer as well as the packet
layer, is a promising candidate concept for providing the network dynamicity required to fully utilise
the increased capacity delivered by the approaches discussed in Sections 2 and 3. It can simplify the
complexities of handling traffic among various networking technologies, offering a central,
integrated multi-layer view and allowing the underlying infrastructure to be abstracted (in the forms
of maps and graphs) and used by applications and network services as a virtual entity. The
architecture is not only well suited to address the present optical networking problems related to
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 72
Network Dynamicity

supporting different administrative and technology segments, bridging the gap between packet and
optical layers, but it also supports new operations such as virtualisation, cross-layer orchestration,
bandwidth on demand (BoD), load balancing and many more. The OpenFlow protocol extensions
developed by the ONF are further increasing SDN’s potential within optical networks.

Similarly, an application-based network operations (ABNO) environment is being increasingly

favoured as an orchestration mechanism to meet new requirements (such as on-demand and
application-specific reservation of network connectivity, reliability and resources across multiple
network applications and technologies) of new services. JRA1 T1 has built an ABNO-based integrated
demonstrator, which provides an emulated environment for evaluating the ability of SDN to control
network resources and provide programmability of network function. It uses the OpenFlow Protocol
extensions, in the form of EXPERIMENTER properties, to provide modulation format support for the
transmitters.

A use case to optimise a failure scenario has demonstrated the ability of transport SDN, combined
with an open network operating system (ONOS), to work effectively in a packet-optical environment
using a global view of the combined network, making optimal routing decisions to provide layerless
restoration. At the same time it is clear that further developments in this direction are needed, such
as the ability to work with OTN switches, and the ability to specify more sophisticated metrics than
just hop numbers and some others.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 73
6 Conclusions
In this deliverable the concept of zero-touch connectivity is cited as a way to satisfy the
requirements from the NRENs’ user base, whether these are university students or high-demanding
research projects. The vision is to provide the needed bandwidth to the user wherever the user is
located and whenever the user requests it. The deliverable addresses the underlying processes and
requirements needed in order to fulfil or partly fulfil this vision. These include a high-capacity flexible
network infrastructure and key functionalities. The deliverable therefore addresses the flexibility and
capacity development of the photonic layer, the flexible deployment of infrastructures using alien
wavelengths, functionalities for time and frequency synchronisation, and overall resource
management and orchestration.

Growing Capacity Trends

A number of new formats and modulations currently offer the ability to increase the capacity of the
fibre, though none is completely problem free. Client-side higher-order modulation techniques have
started to emerge, of which the simplest, and thus most cost-effective, form at this stage is PAM-4.
The SDM technologies of FMF and MCF are promising, but not without problems, including
constraints imposed by the available amplification techniques.

In parallel with the important progress in high-bitrate standardisation being delivered by three
standardisation bodies, GÉANT partners PSNC and CESNET have conducted successful 400G
transmission experiments, including the use of software-defined optics to deliver flexible data rates
and modulation. As a result, PSNC has established and configured two operational 400G links and
the CESNET tests demonstrated the successful extension of all-optical reach to 2,000 km and with
high-bitrate traffic that can easily be transported as alien wavelengths over CESNET’s production
network. These deployments are direct results of the experiments conducted within GN4-1 and
show the viability of high bitrate transmission for NRENs.

With regard to enabling effective, optimal use of the availability capacity through flexible optical
networks (FON), the GN3plus Open Call project REACTION introduced significant innovations in the
context of data plane (support for sliceability), control plane (novel PCE architecture), and routing
and spectrum allocation strategies (novel RSA algorithms). Developments since REACTION have
further enhanced FON’s ability to maximise spectral efficiency and reach by minimising the negative
penalties. For the future, there is increasing need for vendor interoperability for network
subsystems, or even elements, and hence for a common network management platform.

A survey of BVT vendors showed a subset of common functionalities, though with implementations
– particularly of spectral occupancy and Baud rate – varying from vendor to vendor. Further, the
interoperability of two S-BVT implementations has been recently demonstrated on a pan-European
Deliverable D13.1
Research and Study Network Technologies – 74
White Paper
Document Code: GN4-1-16-7D602
Conclusions

multi-domain multi-vendor elastic optical network (EON) testbed within the IDEALIST project.
Experiments proved the need for a new soft-decision FEC standard which would significantly
increase reach in single- and cross-vendor transmission.

Spectral Sharing and Alien Waves

Alien waves and spectral sharing are gaining momentum among NRENs as one way of providing a
more dynamic, flexible and cost efficient infrastructure. The reliability and integrity of alien waves
have been sufficiently proven for SURFnet and NORDUnet to have transitioned AW services into
production for both client and own use. Valuable results have been obtained on defining important
physical transmission parameters for the native and alien signals’ interaction. The multi-domain
optical modelling tool (MOMoT), originally developed during a GN3plus Open Call project and
further developed during GN4-1, helps determine these and other design parameters needed for
AW adoption. Verification tests in NORDUnet’s live production network, with BER vs. launch power
as a criterion, showed a reasonable correlation between MOMoT’s predictions and actual
measurements. This is very useful for future AW deployments, as NRENs using the extended
MOMoT tool can safely estimate the impact on the alien and the host waves. This potentially
improves the setup time of alien waves.

Frequency and Time Distribution

The transmission of accurate time and stable frequency over alien lambdas is possible and is already
in operation or has been tested.

When higher time or frequency stability is required, bi-directional transmission using single bi-
directional all-optical channel needs to be established. The use of non-standard telecom devices
(mainly amplifiers) opens up some issues to be solved, especially monitoring of bi-directional signals
at the optical layer and identifying best engineering practices (amplifier gains, optical powers to
achieve the best performance without disturbing NREN traffic).

The techniques investigated are very promising, because they provide sufficient accuracy for very
high-demanding applications such as remotely compared atomic clocks.

The suitability of precision time protocol (PTP) for providing clock synchronisation over layer 3
networks was investigated. In particular, MPLS tunnels were used and an audio stream selected as a
time-sensitive data application. The transfer was measured and compared to LAN environments.
These long-term measurements show that adverse network conditions can lead to a loss of the
reference signal that the PTP slave obtains from the grandmaster clock. More investigations need to
be done over varying distances, and also longer distances and longer timeframes, in order to be able
to determine how far away slaves can be placed from PTP grandmasters and how exactly network
conditions such as jitter affect slave calibration to the grandmaster. In the audio tests, there was
jitter in the MPLS tunnel as low as 50 – 60 us; nevertheless, it was not possible in the tests to have a
clear audio without distorting noise. As existing standards and recommendations do not describe the
level of accuracy of PTP needed for provisional audio and video applications, many additional tests
are needed in various scenarios to be able to narrow down network jitter or loss requirements.

Network Dynamicity
Software-defined networking, now that it has moved to the transport layer as well as the packet
layer, is a promising candidate concept for providing the network dynamicity required to fully utilise
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 75
Conclusions

the increased capacity and flexibility of optical networks. The approach is not only well suited to
address the present optical networking problems related to supporting different administrative and
technology segments, bridging the gap between packet and optical layers, but it also supports new
operations such as virtualisation, cross-layer orchestration, bandwidth on demand (BoD), load
balancing and many more. The OpenFlow protocol extensions developed by the ONF are further
increasing SDN’s potential within optical networks.

The application-based network operations (ABNO) environment is being increasingly favoured as an

SDN-aware orchestration mechanism to meet new requirements (such as on-demand and
application-specific reservation of network connectivity, reliability and resources across multiple
network applications and technologies) of new services. JRA1 T1 has built an ABNO-based integrated
demonstrator, which provides an emulated environment for evaluating the ability of SDN to control
network resources and provide programmability of network function. It uses the OpenFlow Protocol
extensions, in the form of EXPERIMENTER properties, to provide modulation format support for the
transmitters.

A use case to optimise a failure scenario has demonstrated the ability of transport SDN, combined
with an open network operating system (ONOS), to work effectively in a packet-optical environment
using a global view of the combined network, making optimal routing decisions to provide layerless
restoration. At the same time it is clear that further developments in this direction are needed, such
as the ability to work with OTN switches, and the ability to specify more sophisticated metrics than
just hop numbers and some others.

Summary Evaluation
This deliverable presents the results of comprehensive research and experimentation by JRA1 T1
into the current trends and technologies in the physical and logical domains of optical transport
networks, with the aim of supporting the ZTC vision that will help NRENs to meet the high-
demanding requirements they face from user, technology, cost and management perspectives.
While challenges and the need for further testing exist in many of the areas analysed, the work has
identified several promising solutions and, of particular importance, has assessed these in the NREN
environment. This has led to the deployment of enhanced operational services in some NRENs, and
it is hoped that the findings will similarly guide other NRENs in the development of their next-
generation infrastructure.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 76
Appendix A MOMoT Modelling Tool
The multi-domain optical modelling tool is software (.exe) providing a user-friendly interface where
link, native and alien wavelength parameters can be set. Based on the provided input parameters,
the software outputs bit error rate (BER) estimation. Figure A.1 below shows the interface of the
modelling tool, where input parameters can be set and outputs are visualised.

A.1 Inputs
The user can set the following parameters through the interface:

 The bitrate of the native or alien wavelength in Gb/s (Native).

 The defined channel spacing in the transmission link (Native).
 The bitrate of the alien wavelength in Gb/s (Bitrate).
 The modulation format of the alien wavelength in Gb/s (Modulation format).
 The noise figure of the amplifiers in dB (Noise Figure).
 The launch power of the native and alien wavelengths at the transmitter, in dBm (TX power).
 The number of spans (integer number #).
 The residual chromatic dispersion of each span when dispersion compensating fibres (DCFs)
are installed in the network.
 Whether DCFs are installed or not.
 The worst-case margins (in dB) to account for non-modelled impairments (such as fibre
ageing).
 The parameters related to each span:
○ Length in km (Length).
○ Attenuation in dB (Attenuation).
○ Chromatic dispersion in ps/nm (chromatic dispersion).

A.2 Outputs
When the button “Compute” is pressed, the software will run and the following output will be
presented:

Deliverable D13.1
Research and Study Network Technologies – 77
White Paper
Document Code: GN4-1-16-7D602
Appendix A MOMoT Modelling Tool

 Evaluated BER pre-FEC: in this field the estimated BER of the (alien) wavelength will be
reported.
 Output messages: in this field, possible messages will be displayed. For example, if the power
of the alien wavelength exceeds the threshold of 2 dBm, the following message is displayed:
“Warning: new channel may be disruptive for adjacent channels”. This has been introduced
since the level of power may be too high and it may induce excessive cross-phase modulation
(XPM) on active wavelengths. If there is no warning, nothing is visualised in this field. In the
case of injecting a PM-QPSK alien wave into an OOK native network, output messages will
report the BER of the alien wave at varying guard band between the alien and a native OOK,
as shown in Figure A.2. Indeed, the BER of a PM-QPSK is strongly affected by the presence of
OOK neighbours because of XPM. Thus, by spacing out PM-QPSK and OOK, the performance
of the PM-QPSK improves. Guard band is defined as the number of free channels between a
PM-QPSK and an OOK. Another kind of output message is reported if the performance is
strongly affected by the PMD.

A.3 Buttons
The following buttons are present in the user interface:

 Compute: when this button is pressed, the model is run to evaluate the BER pre-FEC.
 Reset: when this button is pressed, the output only is reset, while the last input parameters
are kept in memory and can be changed one by one if necessary. This button is particularly
useful when varying just one or a few parameters (e.g. TX power). Thus, only one or a few
parameters have to be changed manually.
 Reset all: when this button is pressed, all inputs and outputs are reset.
 Save file: a file including the configuration of all the input parameters can be saved
(“configuration file”).
 Browse file: once a “configuration file” has been saved, this button enables the identification
of the directory where the file is placed.
 Load file: when this button is pressed, the configuration parameters saved are loaded into
the “configuration file”.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 78
Appendix A MOMoT Modelling Tool

Figure A.1: Modelling tool interface

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 79
Appendix A MOMoT Modelling Tool

Figure A.2: Modelling tool interface and output messages including guard band information

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 80
Appendix B Transport Extensions in the ONF
OpenFlow Protocol

B.1 Background
The initial version 1.0 of the OpenFlow protocol was designed to control packet switches and routers;
accordingly, the main elements of OF v1.0 (such as flow and port descriptions) were packet-oriented
only. However, it became clear that optical transport equipment and networks (based on OTN, SDH
and DWDM technologies) that had been an integral part of modern telecommunication networks for
years also have to be included in the SDN realm in a consistent way, allowing smooth control of
combined packet-optical networks.

Since standardisation of OF v1.0 in 2009, the SDN community has been actively working on
extending the OF protocol with an ability to operate with switched units of optical transport
networks – time-slots, wavelengths and fibres. Until recently, these extensions were mostly
proprietary, suggested both by researchers and vendors. One of the most popular documents
describing such extensions was Extensions to the OpenFlow Protocol in support of Circuit Switching.
Addendum to OpenFlow Protocol Specification (v1.0) – Circuit Switch Addendum v0.3 [58], developed
by the OpenFlow Switch Consortium in 2010.

The ONF took over from the OpenFlow Switch Consortium in the development of the OF protocol in
2011 and started introducing more flexibility in the protocol functionality by adding so-called
EXPERIMENTER elements to its constructions. Initially this was done to the protocol messages only
(since OF v1.1), and then extended to flow match and action fields, queue and statistics properties
(since of v1.2) and port description properties (since OF v1.4).

The EXPERIMENTER type of OF protocol element allows a researcher or vendor to define a new
structure for those elements, for example, a new structure for a protocol message, and implement it
in the network equipment and SDN controllers to achieve some specific functionality. The
EXPERIMENTER type of protocol element can be either proprietary or standard, i.e. defined by a
standards body. The OF optical transport extensions described in Sections 5.1.4.1 and 5.1.4.2 of this
document are examples of proprietary extensions.

A specification Optical Transport Protocol Extensions v.1.0 [59] developed by the ONF describes the
OpenFlow standard extensions based on the EXPERIMENTER protocol elements. The specification is
seen by the ONF as an initial document that systematises the OF optical transport extensions in
three areas:
Deliverable D13.1
Research and Study Network Technologies – 81
White Paper
Document Code: GN4-1-16-7D602
Appendix B Transport Extensions in the ONF OpenFlow Protocol

 Flow match fields and actions.

 Port description extensions for OTN ports.
 Port extensions for adjacency discovery.

The ONF assumes that the extensions described in the separate specification will gradually become
parts of the main OF protocol specification.

The ONF is also going to develop optical transport extensions for other areas such as:

 OAM/monitoring of optical network links.

 Support of protection switching mechanisms.
 Support of multilayer connections (reflecting the adaptation of a client signal into a server
one).
 Support of OF protocol use for vertical controller-to-controller communications.

The following sections consider the OF transport extensions in the three initial areas.

B.2 Flow Match Fields and Actions

The proposed extensions allow the selection of circuit-switched flows based on signals of two types:

 Wavelength signals (optical channel (OCh)).

 OTN signals (ODUj/k frames).

In both cases the match fields are the OF Extensible Match EXPERIMENTER type (0xFFFF) and include
EXPERIMENTER_ID 0xFF000007 assigned to the Optical Transport Working Group of ONF.

Two match fields are used to select a particular signal:

 OFPXMT_EXP_OCH_SIGTYPE and OFPXMT_EXP_OCH_SIGID for wavelength signals.

 OFPXMT_EXP_ODU_SIGTYPE and OFPXMT_EXP_ODU_SIGID for OTU signals.

A xxx_SIGTYPE field is used for a broad description of the signal type, while a xxx_SIGID field
specifies the signal more precisely.

Wavelength Signals

In the case of wavelength signals, the OFPXMT_EXP_OCH_SIGTYPE field can have one of two values:

 OFPOCHT_FIX_GRID if a signal uses a fixed grid.

 OFPOCHT_FLEX_GRID if a signal uses a flex grid.

The second field OFPXMT_EXP_OCH_SIGID describes the precise position of the signal on the grid
using the following structure:

/* OFPXMT_EXP_OCH_SIGID Payload format */

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 82
Appendix B Transport Extensions in the ONF OpenFlow Protocol

struct ofp_oxm_exp_OCH_sigid {
uint32_t oxm_header; /* oxm_class = OFPXMC_EXPERIMENTER */
uint32_t experimenter; /* Experimenter ID = 0xFF000007 */
uint8_t grid_type; /* Grid Type */
uint8_t chl_spacing; /* Channel spacing */
uint16_t n; /* n is used to calculate the frequency as in [ITU
G.694.1]
* Frequency(THz)= 193.1 THz + n*chl_spacing
(THz)*/
uint16_t m; /* m is used to identify the slot width as defined
in [ITU G.694.1],
* Slot Width (GHz) = m*12.5 (GHz)
* For fix grid networks, m=1 */ }

In this structure the grid_type parameter gives more information about the grid type than the
OFPXMT_EXP_OCH_SIGTYPE field as it can differentiate between DWDM, CWDM and flex grids;
however, the flex grid case is left for further discussion in the current version 1.0 of the specification.

OTU Signals

For OTU signals the OFPXMT_EXP_ODU_SIGTYPE field gives information about the high order (HO)
ODU frame type of the signal, including ODU flex frames:

enum ofp_odu_signal_type {
OFPODUT_ODU1 = 1,
OFPODUT_ODU2 = 2,
OFPODUT_ODU3 = 3,
OFPODUT_ODU4 = 4,
OFPODUT_ODU0 = 10,
OFPODUT_ODU2E = 11,
OFPODUT_ODUfCBR = 20,
OFPODUT_ODUfGFPfHAO = 21,
OFPODUT_ODUfGFPf = 22,
};

If an HO OTN switching takes place, a signal is fully defined by the OFPXMT_EXP_ODU_SIGTYPE field
and together with an input port number (described by the OFPXMT_OFB_IN_PORT field) a circuit-
switched flow is fully defined too.

However, for a low order (LO) OTN switching that operates with an ODU frame multiplexed into a
higher order ODU frame, the second flow match field OFPXMT_EXP_ODU_SIGID that describes the
positions of the LO frame time-slots within a multiplexed HO ODU structure:

/* OFPXMT_EXP_ODU_SIGID Payload format */

struct ofp_oxm_exp_ODU_sigid {
uint32_t oxm_header; /* oxm_class = OFPXMC_EXPERIMENTER */
uint32_t experimenter; /* Experimenter ID ID = 0xFF000007 */
uint16_t tpn; /* Tributary Port Number */

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 83
Appendix B Transport Extensions in the ONF OpenFlow Protocol

uint16_t tslen; /* Number of Tributary Slots included in tsmap,

based on the Server ODU type and TS
* Granularity. */
uint8_t tsmap[0]; /* Tributary slot bitmap */
};

The ONF approach to defining LO ODU data within a multiplexed HO ODU frame is similar to the one
taken by the IETF in defining GMPLS labels for OTN data [60]. In both cases the bitmap bits indicate
which tributary slots in the HO ODUk the LO ODUj will be multiplexed into. Each bit in the bitmap
represents the corresponding tributary slot in the HO ODUk, with a value of 1 or 0 indicating
whether the tributary slot will be used by the LO ODUj or not.

Specific actions extensions for the optical transport are not defined by the specification. Instead the
specification suggests using the standard OF SET_FIELD action with the new flow match field types,
described above.

B.2.1 Examples
This section presents three examples of how operations with circuit flows can be defined by the use
of the new flow match extensions:

 ODU2 HO – ODU2 HO flow.

 ODU0 LO (ODU2) to ODU0 LO (ODU2) flow.
 1GE to ODU0 LO-ODU flow.

B.2.1.1 Example 1: ODU2 HO – ODU2 HO flow

This is a simple case, as a switch just needs to transmit ODU2 frames from one port 2 to port 7
without looking inside the frame header fields. Only the OFPXMT_EXP_ODU_SIGTYPE match flow
field is needed for this:

Match:

ofp_match_type = OFPMT_OXM
ofp_oxm_class = OFPXMC_OPENFLOW_BASIC
ofp_oxm_field = OFPXMT_OFB_IN_PORT
data = 2 (input port number)
ofp_oxm_class = OFPXMC_EXPERIMENTER
ofp-oxm_field = OFPXMT_EXP_ODU_SIGTYPE
experimenter = 0xFF000007
sigtype = OFPODUT_ODU2

Instruction:

type = OFPIT_APPLY_ACTIONS

Actions:

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 84
Appendix B Transport Extensions in the ONF OpenFlow Protocol

type = OFPAT_OUTPUT
port = 7 (output port number)

B.2.1.2 Example 2: ODU0 LO (ODU2) to ODU0 LO (ODU2) flow

This example shows the creation of a flow of a signal contained in ODU0 frames multiplexed into
ODU2 frames. To specify a signal both SIGTYPE and SIGID match fields are needed. The flow input
port is 3, the output port is 8, and both are ODU2 type.

Match:

ofp_match_type = OFPMT_OXM
ofp_oxm_class = OFPXMC_OPENFLOW_BASIC
ofp_oxm_field = OFPXMT_OFB_IN_PORT
data = 3 (input port number)
ofp_oxm_class = OFPXMC_EXPERIMENTER
ofp-oxm_field = OFPXMT_EXP_ODU_SIGTYPE
experimenter = 0xFF000007
sigtype = OFPODUT_ODU0
ofp_oxm_class = OFPXMC_EXPERIMENTER
ofp-oxm_field = OFPXMT_EXP_ODU_SIGID
experimenter = 0xFF000007
tpn = 1 (tributary port number)
tslen = 8 (ODU2 frame has 8 time-slots with 1.25 Gbps
granularity)
tsmap = 00100000 (tributary slot bitmap)

Instruction:

type = OFPIT_APPLY_ACTIONS

Actions:

type = OFPAT_SET_FIELD
ofp_oxm_class = OFPXMC_EXPERIMENTER
ofp-oxm_field = OFPXMT_EXP_ODU_SIGID
experimenter = 0xFF000007
tpn = 1 (tributary port number)
tslen = 8
tsmap = 00001000 (tributary slot bitmap)
type = OFPAT_OUTPUT
port = 8 (output port number)

The tributary slot bitmap field values define the cross-connect action required from a switch, taking
a signal from the 3rd time-slot of the input ODU2 frame and placing it into the 5th time-slot of the
output ODU2 frame.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 85
Appendix B Transport Extensions in the ONF OpenFlow Protocol

B.2.1.3 Example 3: 1GE to ODU0 LO-ODU flow

This example shows that hybrid packet-optical flows can also be defined by using the new flow
match fields. The flow in this example is between packets of input 1GE port number 2 and the ODU0
time-slot of the ODU2 output port 5.

Match:

ofp_match_type = OFPMT_OXM
ofp_oxm_class = OFPXMC_OPENFLOW_BASIC
ofp_oxm_field = OFPXMT_OFB_IN_PORT
data = 2 (input 1GE port number )
ofp_oxm_class = OFPXMC_EXPERIMENTER
ofp-oxm_field = OFPXMT_EXP_ODU_SIGTYPE
experimenter = 0xFF000007
sigtype = OFPODUT_ODU0

Instruction:

type = OFPIT_APPLY_ACTIONS

Actions:

type = OFPAT_SET_FIELD
ofp_oxm_class = OFPXMC_EXPERIMENTER
ofp-oxm_field = OFPXMT_EXP_ODU_SIGID
experimenter = 0xFF000007
tpn = 1 (tributary port number)
tslen = 8
tsmap = 01000000 (tributary slot bitmap)
type = OFPAT_OUTPUT
port = 5 (output ODU2 port number )

B.3 Port Description Extensions

The extension specification defines properties of the OTN port only, as those of the optical (xWDM)
port are defined in the main OF specification since version 1.4. The main OF specification v.1.4 (and
higher) introduces three types of physical ports properties:

/* Port description property types. */

enum ofp_port_desc_prop_type {
OFPPDPT_ETHERNET = 0, /* Ethernet property. */
OFPPDPT_OPTICAL = 1, /* Optical property. */
OFPPDPT_EXPERIMENTER = 0xFFFF, /* Experimenter property. */
};

(Note that OF versions lower than 1.4 define only the OFPPDPT_ETHERNET type of port properties).

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 86
Appendix B Transport Extensions in the ONF OpenFlow Protocol

For convenience, the definition of the optical port properties from the OF v.1.4 specification are
given here:

/* Optical port description property. */

struct ofp_port_desc_prop_optical {
uint16_t type; /* OFPPDPT_3OPTICAL. */
uint16_t length; /* Length in bytes of this property. */
uint8_t pad[4]; /* Align to 64 bits. */
uint32_t supported; /* Features supported by the port. */
uint32_t tx_min_freq_lmda; /* Minimum TX Frequency/Wavelength *
uint32_t tx_max_freq_lmda; /* Maximum TX Frequency/Wavelength */
uint32_t tx_grid_freq_lmda; /* TX Grid Spacing
Frequency/Wavelength */
uint32_t rx_min_freq_lmda; /* Minimum RX Frequency/Wavelength */
uint32_t rx_max_freq_lmda; /* Maximum RX Frequency/Wavelength */
uint32_t rx_grid_freq_lmda; /* RX Grid Spacing
Frequency/Wavelength */
uint16_t tx_pwr_min; /* Minimum TX power */
uint16_t tx_pwr_max; /* Maximum TX power */
};
OFP_ASSERT(sizeof(struct ofp_port_desc_prop_optical) == 40);

The features of the optical port are defined by the supported parameter:

/* Features of optical ports available in switch. */

enum ofp_optical_port_features {
OFPOPF_RX_TUNE = 1 << 0, /* Receiver is tunable */
OFPOPF_TX_TUNE = 1 << 1, /* Transmit is tunable */
OFPOPF_TX_PWR = 1 << 2, /* Power is configurable */
OFPOPF_USE_FREQ = 1 << 3, /* Use Frequency, not wavelength */
};

The transport extensions specification uses the OFPPDPT_EXPERIMENTER properties type with the
EXPERIMENTER ID assigned to OTWG (0xFF000007 ) to define the OTN port properties.

The most informative data structure describing the OTN port properties (among others) is the
following:

/* OTN Layer Stack Entry Encoding */

struct ofp_port_optical_transport_layer_entry {
uint8_t layer_class; /* OFPOTPL_* */
uint8_t signal_type; /* OFP(OTP/OCH/ODU/ODUCL)T_*/
uint8_t adaptation; /* OFPADAPT_*/

The three enum variables of this structure allow the major parameters of the OTN port to be
described:

 The variable layer_class defines the type of port, either ODU or ODUCLT (a client port).

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 87
Appendix B Transport Extensions in the ONF OpenFlow Protocol

 The variable signal_port specifies the type of the port signal type:

○ For the ODU port it can be one of the following:

OFPODUT_ODU1 = 1,
OFPODUT_ODU2 = 2,
OFPODUT_ODU3 = 3,
OFPODUT_ODU4 = 4,
OFPODUT_ODU0 = 10,
OFPODUT_ODU2E = 11,
OFPODUT_ODUfCBR = 20,
OFPODUT_ODUfGFPfHAO = 21,
OFPODUT_ODUfGFPf = 22

○ For the ODUCLT port it can be one of the following:

OFPODUCLT_STM16 = 1,
OFPODUCLT_STM64 = 2,
OFPODUCLT_STM256 = 3,
OFPODUCLT_STM1 = 4,
OFPODUCLT_STM4 = 5,
OFPODUCLT_1GBE = 6,
OFPODUCLT_10GBE = 7,
OFPODUCLT_40GBE = 8,
OFPODUCLT_100GBE = 9,
OFPODUCLT_FC100 = 10,
OFPODUCLT_FC200 = 11,
OFPODUCLT_FC400 = 12,
OFPODUCLT_FC800 = 13,
OFPODUCLT_FC1200 = 14,
OFPODUCLT_GPON = 15,
OFPODUCLT_XGPON = 16,
OFPODUCLT_IB_SDR = 17,
OFPODUCLT_IB_DDR = 18,
OFPODUCLT_IB_QDR = 19,
OFPODUCLT_SBCON_ESCON = 20,
OFPODUCLT_DVB-ASI = 21,
OFPODUCLT_SDI = 22,
OFPODUCLT_SDI1G5 = 23,
OFPODUCLT_SDI3G = 24,
OFPODUCLT_ATM = 25,
OFPODUCLT_ETH = 26,
OFPODUCLT_MPLS = 27,
OFPODUCLT_IP = 28

 Finally, the variable adaptation shows what type of adaptation the port supports:

OFPADAPT_OTS_OMS = 1,
OFPADAPT_OMS_OCH = 2,

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 88
Appendix B Transport Extensions in the ONF OpenFlow Protocol

OFPADAPT_OPS_OCHr = 3,
OFPADAPT_OPSM_OTUk = 4,
OFPADAPT_OCH_OTUk = 5,
OFPADAPT_ODUk_ODUij = 6,
OFPADAPT_ODUk_ODUj21 = 7,
OFPADAPT_ODUkh_ODUj21 = 8,
OFPADAPT_ODU0_CBRx = 9,
OFPADAPT_ODUk_CBRx = 10,
OFPADAPT_ODUk_CBRxg = 11,
OFPADAPT_ODUk_RSn = 12,
OFPADAPT_ODUk_ATM = 13,
OFPADAPT_ODUk_ETH = 14,
OFPADAPT_ODUkh_ETH = 15,
OFPADAPT_ODUk_ETHPPOS = 16

In OF v.1.3 protocol the OTN port information is passed to a controller by using a multi-part message;
the structures that are used in this message for describing port properties are the same as for OF 1.4.

B.4 Port Extensions for Adjacency Discovery

The transport extensions specification also introduces port extensions for adjacency discovery of
OTN boxes. These extensions suggest that the OTN boxes use Protocol for automatic discovery in
transport networks (Recommendation ITU-T G.7714.1/Y.1705.1) [61]. According to this protocol, the
OTN boxes exchange identity information (such as an E.164 name or an arbitrary number), which
allows neighbours to be identified and a network topology to be built. The exchange happens in-
service as the box ID is placed into the Trial Termination Identifier (TTI) field of an ODU header. By
default, the ODU tandem connection monitoring (TCM) sub-layer 6 (TCM6) is used for discovery.

The port extensions for adjacency discovery define the new port properties structures, which can
store three TTI values:

 To be sent by the port.

 Expected to be received on the port.
 Actually received on the port.

A controller can obtain the TTI values stored by the port and modify them using the OF messages.
Having knowledge of these values from all the ports, a controller is able to build a network topology
map.

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 89
References
[1] H. Wessing (DTU), K. Bozorgebrahimi (UNINETT), A. Tzanakaki (Bristol), B. Belter
(PSNC), S. Naegele-Jackson (FAU), A. Metz (IRT), P. Skoda (CESNET), J. Vojtech
(CESNET), V. Olifer (Janet), Deliverable D12.3 (DJ1.1.1) Future Network
Architectures, 2015
http://geant3plus.archive.geant.net/Resources/Deliverables/Documents/D12-
3_DJ1-1-1_Future-Network-Architectures.pdf
[2] L. Lange Bjørn (NORDUnet), K. Bozorgebrahimi (UNINETT), E. Camisard (RENATER),
P. Gasner (RoEduNet), M. Hůla (CESNET), M. Karásek (CESNET), R. Lund
(NORDUnet), R. Nuijts (SURFNET), R. Octavian (RoEduNet), P. Škoda (CESNET), S.
Šíma (CESNET), P. Turowicz (PSNC), K. Turza (PSNC), S. Tyley (DANTE), J. Vojtěch
(CESNET), V. Vraciu (RoEduNet), G. Zervas (University of Essex), Deliverable
DJ1.2.1: State-of-the-Art Photonic Switching Technologies, 2010
[archive not currently available]
[3] S. Dey, PAM4: The Solution For 400G Client Side Links, NeoPhotonics blog
http://blog.neophotonics.com/pam4-the-solution-for-400g-client-side-links/
[4] Keysight Technologies White Paper, PAM-4 Design Challenges and the
Implications on Test
http://literature.cdn.keysight.com/litweb/pdf/5992-0527EN.pdf
[5] Tektronix, PAM4 Signaling in High Speed Serial Technology: Test, Analysis, and
Debug
http://www.tek.com/document/application-note/pam4-signaling-high-speed-
serial-technology-test-analysis-and-debug
[6] R. Ryf and P. J. Winzer, Space-Division Multiplexing: The Future of Fiber-Optic
Communications, IEEE Photonic Soc. News, vol. 29, no. 5, 2015
http://www.ieee.org/ns/periodicals/Photo/oct2015/html5/index.html
[7] Y. Jung, Q. Kang, J. Sabu, S. Alam, and D. J. Richardson, Optical Amplifiers for
Space Division Multiplexed Transmission, IEEE Photonic Soc. News, vol. 29, no. 5,
2015
http://www.ieee.org/ns/periodicals/Photo/oct2015/html5/index.html
[8] J. J.Sakaguchi, W. Klaus, B. J. Puttnam, and Y. Awaji, Development of Space
Division Multiplexing Technologies in Japan, IEEE Photonic Soc. News, vol. 29, no.
5, 2015
http://www.ieee.org/ns/periodicals/Photo/oct2015/html5/index.html
[9] S. Tammela, Fiber Amplifiers, Directly Modulated Transmitters and a Ring
Network Structure for Optical Communications, Helsinki University of Technology,
2004
https://www.researchgate.net/publication/36145031_Fiber_amplifiers_directly_
Deliverable D13.1
Research and Study Network Technologies – 90
White Paper
Document Code: GN4-1-16-7D602
References

modulated_transmitters_and_a_ring_network_structure_for_optical_communic
ations
[10] A. Puc et al., L-Band unrepeated WDM experiment over 451 km using all-Raman
amplification and ROPA, Xtera Technical Paper
http://www.xtera.com/wp-
content/uploads/2015/10/lbandUnrepeatered_2008.pdf
[11] T. Sakamoto, S. I. Aozasa, M. Yamada, and M. Shimizu, Hybrid fiber amplifiers
consisting of cascaded TDFA and EDFA for WDM signals, J. Light. Technol., vol. 24,
no. 6, pp. 2287–2295, 2006
https://www.researchgate.net/publication/3243513_Hybrid_fiber_amplifiers_co
nsisting_of_cascaded_TDFA_and_EDFA_for_WDM_signals
[12] A. D. Ellis, N. Mac Suibhne, D. Saad, and D. N. Payne, Communication networks
beyond the capacity crunch, Philos. Trans. A. Math. Phys. Eng. Sci., vol. 374, no.
2062, Mar. 2016.
[13] J. J. Ackert, D. J. Thomson, L. Shen, A. C. Peacock, P. E. Jessop, G. T. Reed, G. Z.
Mashanovich, and A. P. Knights, High-speed detection at two micrometres with
monolithic silicon photodiodes, Nat. Photonics, vol. 9, no. 6, pp. 393–396, May
2015
http://www.nature.com/nphoton/journal/v9/n6/full/nphoton.2015.81.html
[14] www.ieee802.org/3
[15] J. D’Ambrosia, 802.3bs PROJECT DOCUMENTATION CONSIDERATION, 2015
http://www.ieee802.org/3/50G/public/adhoc/archive/dambrosia_120215_50GE
_NGOATH_adhoc_v2.pdf
[16] P. Anslow (Ciena), M. Brown (APM), 50G, 100G, 200G work partitioning, 2015
http://www.ieee802.org/3/50G/public/adhoc/archive/anslow_120915_50GE_NG
OATH_adhoc.pdf
[17] IEEE, 50 Gb/s Ethernet over a Single Lane and Next Generation 100 Gb/s and 200
Gb/s Ethernet Study Groups Adopted Objectives
http://www.ieee802.org/3/50G/public/objectives_50G_NGOATH_01a_0316.pdf
[18] S. Hardy, IEEE sets objectives for 200 Gigabit Ethernet, 10 March 2016
http://www.lightwaveonline.com/articles/2016/03/ieee-sets-objectives-for-200-
gigabit-
ethernet.html?cmpid=EnlENABLINGMarch102016&eid=288553906&bid=1335905
[19] S. Trowbridge and P. Anslow, OTN Support for 25GbE
http://www.ieee802.org/3/25GSG/public/Sept14/trowbridge_25GE_01_0914.pdf
[20] OIF Aims to Enable More Flexible Ethernet
http://www.lightreading.com/ethernet-ip/100g-ethernet/oif-aims-to-enable-
more-flexible-ethernet/d/d-id/713717
[21] D. Porchivina, OIF LAUNCHES FLEX COHERENT DWDM FRAMEWORK EFFORT,
ELECTS NEW OFFICERS, 19 November 2015
http://www.oiforum.com/oif-launches-flex-coherent-dwdm-framework-effort-
elects-new-officers/
[22] ECI® and CESNET Announce Successful Trial of 400G Flex-Grid Blade Over Live
CESNET Network in the Czech Republic, 9 December 2015
http://www.ecitele.com/media/1504/successful-400g-trial-with-cesnet-
120915.pdf
[23] http://geant3plus.archive.geant.net/opencall/Optical/Pages/REACTION.aspx

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 91
References

[24] O. Vassilieva, T. Yamauchi, S. Oda, I. Kim, T. Hoshida, Y. Aoki, J. C. Rasmussen, and

M. Sekiya, Flexible grid network optimization for maximum spectral efficiency and
reach, in 2015 European Conference on Optical Communication (ECOC), 2015, pp.
1–3
[25] P. S. Khodashenas, Evaluating the optimum filter resolution and sub-channel
spectrum granularity for flexible super-channels, Conf. Opt. Fiber Commun. Tech.
Dig. Ser., vol. 2015-, 2015
https://www.researchgate.net/publication/273662612_Evaluating_the_Optimum
_Filter_Resolution_and_Sub-Channel_Spectrum_Granularity_for_Flexible_Super-
Channels
[26] V. Gkamas, A comparison study of joint and sequential multi-layer planning for IP
over flexible optical networks, Opt. Fiber Commun. Conf. Ofc 2015, 2015
[27] A. Ehrhardt, W. Weiershausen, M. Gunkel, and A. Mattheus, High-speed networks
in field operation: An operator’s view, in 2015 European Conference on Optical
Communication (ECOC), 2015, pp. 1–3.
[28] R. G.698.2, Amplified multichannel dense division multiplexing applications with
single channel optical interfaces, ITU-T, 2009
https://www.itu.int/rec/T-REC-G.698.2/en
[29] M. Gunkel et al., Vendor-interoperable elastic optical interfaces: Standards,
experiments, and challenges, IEEE/OSA J. Opt. Commun. Netw., vol. 7, no. 12,
2015
https://www.osapublishing.org/jocn/abstract.cfm?uri=jocn-7-12-B184
[30] O. Gonzalez de Dios, R. Casellas, F. Paolucci, A. Napoli, L. Gifre, A. Dupas, E.
Hugues-Salas, R. Morro, S. Belotti, G. Meloni, T. Rahman, V. Lopez, R. Martinez, F.
Fresi, M. Bohn, S. Yan, L. Velasco, P. Layec, and J. Fernandez-Palacios,
Experimental Demonstration of Multi-vendor and Multi-domain Elastic Optical
Network with data and control interoperability over a Pan-European Test-bed, J.
Light. Technol., vol. 34, issue 7, pp. 1610–1617, 15 February 2016
http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=7407291&url=http%3A%
2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D7407291
[31] A. Napoli, M. Bohn, D. Rafique, A. Stavdas, N. Sambo, L. Poti, M. Nolle, J. K.
Fischer, E. Riccardi, A. Pagano, A. Di Giglio, M. S. Moreolo, J. M. Fabrega, E.
Hugues-Salas, G. Zervas, D. Simeonidou, P. Layec, A. D’Errico, T. Rahman, and J. P.
F.-P. Gimenez, Next generation elastic optical networks: The vision of the
European research project IDEALIST, IEEE Commun. Mag., vol. 53, no. 2, pp. 152–
162, Feb. 2015
http://cpham.perso.univ-pau.fr/ENSEIGNEMENT/PAU-
UPPA/RHD/PAPER/EON.pdf
[32] M. Kuschnerov, F. N. Hauske, K. Piyawanno, B. Spinnler, M. S. Alfiad, A. Napoli,
and B. Lankl, DSP for Coherent Single-Carrier Receivers, J. Light. Technol., vol. 27,
no. 16, pp. 3614–3622, Aug. 2009
http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=5071205&url=http%3A%
2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5071205
[33] A. Napoli, Z. Maalej, V. A. J. M. Sleiffer, M. Kuschnerov, D. Rafique, E. Timmers, B.
Spinnler, T. Rahman, L. D. Coelho, and N. Hanik, Reduced Complexity Digital Back-
Propagation Methods for Optical Communication Systems, J. Light. Technol., vol.
32, no. 7, pp. 1351–1362, Apr. 2014

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 92
References

http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6717017&url=http%3A%
2F%2Fieeexplore.ieee.org%2Fiel7%2F50%2F4357488%2F06717017.pdf%3Farnum
ber%3D6717017
[34] M. Kuschnerov, M. Chouayakh, K. Piyawanno, B. Spinnler, E. de Man, P.
Kainzmaier, M. S. Alfiad, A. Napoli, and B. Lankl, Data-Aided Versus Blind Single-
Carrier Coherent Receivers, IEEE Photonics J., vol. 2, no. 3, pp. 387–403, Jun. 2010
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5447711
[35] M. Gunkel et al., Elastic Black Link for future vendor independent optical networks,
Conf. Opt. Fiber Commun. Tech. Dig. Ser., pp. 1–3, 22-26 March 2015
http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=7121612&url=http%3A%
2F%2Fieeexplore.ieee.org%2Fiel7%2F7111754%2F7121453%2F07121612.pdf%3F
arnumber%3D7121612
[36] http://geant3plus.archive.geant.net/opencall/Optical/Pages/MOMoT.aspx
[37] N. Sambo, M. Secondini, F. Cugini, G. Bottari, P. Iovanna, F. Cavaliere, and P.
Castoldi, Modeling and Distributed Provisioning in 10–40–100-Gb/s Multirate
Wavelength Switched Optical Networks, J. Light. Technol., vol. 29, no. 9, pp.
1248–1257, May 2011
http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=5722966&url=http%3A%
2F%2Fieeexplore.ieee.org%2Fiel5%2F50%2F4357488%2F05722966.pdf%3Farnum
ber%3D5722966
[38] M. Rost, D. Piester, W. Yang, T. Feldmann, T. Wübbena, and A. Bauch, Time
transfer through optical fibres over a distance of 73 km with an uncertainty below
100 ps, Metrologia, vol. 49, no. 6, pp. 772–778, Dec. 2012
https://arxiv.org/ftp/arxiv/papers/1209/1209.4467.pdf
[39] Ł. Śliwczyński, P. Krehlik, and M. Lipiński, Optical fibers in time and frequency
transfer, Meas. Sci. Technol., vol. 21, no. 7, p. 075302, Jul. 2010
https://www.researchgate.net/publication/258294827_Optical_fibers_in_time_a
nd_frequency_transfer
[40] A. Matveev, C. G. Parthey, K. Predehl, J. Alnis, A. Beyer, R. Holzwarth, T. Udem, T.
Wilken, N. Kolachevsky, M. Abgrall, D. Rovera, C. Salomon, P. Laurent, G. Grosche,
O. Terra, T. Legero, H. Schnatz, S. Weyers, B. Altschul, and T. W. Hänsch, Precision
measurement of the hydrogen 1S-2S frequency via a 920-km fiber link, Phys. Rev.
Lett., vol. 110, no. 23, p. 230801, Jun. 2013
http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.110.230801
[41] B. Argence, Quantum cascade laser frequency stabilization at the sub-Hz level,
Nat. Photonics, vol. 9, no. 7, pp. 456 – 460, 2015
http://www.nature.com/nphoton/journal/v9/n7/full/nphoton.2015.93.html
[42] J. Vojtĕch, V. Smotlacha, and J. Radil, All optical two-way time transfer in strongly
heterogeneous networks, in Proceedings of Spie--the International Society for
Optical Engineering, 2014, vol. 9202, p. 92020S
http://spie.org/Publications/Proceedings/Paper/10.1117/12.2061346
[43] N. Chiodo, N. Quintin, F. Stefani, F. Wiotte, E. Camisard, C. Chardonnet, G.
Santarelli, A. Amy-Klein, P.-E. Pottie, and O. Lopez, Cascaded optical fiber link
using the internet network for remote clocks comparison, Opt. Express, vol. 23, no.
26, pp. 33927–37, Dec. 2015
https://arxiv.org/ftp/arxiv/papers/1509/1509.05885.pdf
[44] https://en.wikipedia.org/wiki/Allan_variance

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 93
References

[45] P.-E. Pottie, Remote comparisons Sr-Sr OLC with a long haul optical fiber link, in
Nano-k conference, 2015
http://gdr-af2015.sciencesconf.org/conference/gdr-
af2015/program/pottie_1.pdf
[46] A. Bercy, Ultra-stable optical fiber link: multiple-users dissemination, study of
fundamental and technological limits and new applications, University Paris, 2015.
[47] T. Committee, IEEE Std 1588-2008, IEEE Standard for a Precision Clock
Synchronization Protocol for Networked Measurement and Control Systems, vol.
2008, no. July. 2008
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4579760&filter=AND(
p_Publication_Number:4579757)
[48] L. Velasco, A. Castro, M. Ruiz, and G. Junyent, Solving Routing and Spectrum
Allocation Related Optimization Problems: From Off-Line to In-Operation Flexgrid
Network Planning, J. Light. Technol., vol. 32, no. 16, pp. 2780–2795, Aug. 2014
http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6781635&url=http%3A%
2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D6781635
[49] L. Velasco, A. Castro, D. King, O. Gerstel, R. Casellas, and V. Lopez, In-operation
network planning, IEEE Commun. Mag., vol. 52, no. 1, pp. 52–60, Jan. 2014
http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6710064&url=http%3A%
2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D6710064
[50] ONF, TR 509 – Optical Transport Use Cases, Open Networking Foundation, 2014
https://www.opennetworking.org/images/stories/downloads/sdn-
resources/technical-reports/optical-transport-use-cases.pdf
[51] D. King and A. Farrel, A PCE-based Architecture for Application-based Network
Operations, IETF – RFC 7491, 2015
https://tools.ietf.org/html/rfc7491
[52] http://onosproject.org
[53] T. Otani, D. Li, Abstract Generalized Labels for Lambda-Switch-Capable (LSC) Label
Switching Routers, IETF, 2011
https://tools.ietf.org/html/rfc6205
[54] A. Farrel, D. King, Y. Li, F. Zhang, Generalized Labels for the Flexi-Grid in Lambda
Switch Capable (LSC) Label Switching Routers, IETF Work in Progress progress
draft-ietf-ccamp-flexigrid-lambda-label-05, 2015
https://tools.ietf.org/html/draft-ietf-ccamp-flexigrid-lambda-label-05
[55] S. Das, PAC.C: A UNIFIED CONTROL ARCHITECTURE FOR PACKET AND CIRCUIT
NETWORK CONVERGENCE, 2012
http://yuba.stanford.edu/~sd2/Thesis_Saurav_Das.pdf
[56] Mininet Overview
http://mininet.org/overview
[57] Linc-OE: OpenFlow 1.3 with OE
https://github.com/FlowForwarding/LINC-Switch/blob/internals-oe-
doc/docs/LINC_internals.md#linc-oe-openflow-13-with-oe
NB If this link returns an error message, replace %20-%20 in the browser address field
with the hash character.
[58] Extensions to the OpenFlow Protocol in support of Circuit Switching. Addendum to
OpenFlow Protocol Specification (v1.0) – Circuit Switch Addendum v0.3, June 2010

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 94
References

http://archive.openflow.org/wk/images/8/81/OpenFlow_Circuit_Switch_Specific
ation_v0.3.pdf
[59] Optical Transport Protocol Extensions, Version1.0, March 15, 2015, ONF TS-022
https://www.opennetworking.org/images/stories/downloads/sdn-resources/onf-
specifications/openflow/Optical_Transport_Protocol_Extensions_V1.0.pdf
[60] F. Zhang (ed.), GMPLS Signaling Extensions for Control of Evolving G.709 Optical
Transport Networks, Request for Comments: 7139, March 2014
https://tools.ietf.org/html/rfc7139
[61] Protocol for automatic discovery in transport networks, ITU-T G.7714.1/Y.1705.1
(01/2015)
https://www.itu.int/rec/T-REC-G.7714.1/en

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 95
Glossary
1PPS Pulse Per Second
2D Two-dimensional
ABNO Application-Based Network Operations
AC Application Code
ACES Atomic Clock Ensemble in Space
ADC Analog to Digital Converter
AES Audio Engineering Society
API Application Programming Interface
ARP Address Resolution Protocol
AS Autonomous System
ASE Amplified Spontaneous Emission
ASK Amplitude-Shift Keying
AUI Attachment Unit Interface
AW Alien Wave (or Wavelength)
B100G Beyond 100G
BER Bit Error Rate (or Ratio)
BGP Border Gateway Protocol
BL Black Link
BoD Bandwidth on Demand
bPCE Back-End PCE
BPSK Pinary PSK
BVT Bandwidth Variable Transponders
C band Conventional band
CD Chromatic Dispersion
CFP C Form-factor Pluggable
CNIT Inter-University National Consortium for Telecommunications
CV Common View
CV Cross Vendor
CWDM Coarse Wavelength Division Multiplexing
dB Decibel
dBm the power ratio in decibels of the measured power referenced to one milliwatt
DCF Dispersion Compensating Fibre
DCM Dispersion Compensating Module
DEMUX Demultiplexer
DFA Doped-Fibre Amplification
DMG Differential Modal Gain
DP-QPSK Dual Polarisation-QPSK
Deliverable D13.1
Research and Study Network Technologies – 96
White Paper
Document Code: GN4-1-16-7D602
Glossary

DPSK Differential Phase-Shift Keying

DQPSK Differential Quadrature Phase Shift Keying
DSP Digital Signal Processing
DWDM Dense Wavelength Division Multiplexing
ECOC European Conference on Optical Communications
EDFA Erbium-Doped Fibre Amplifiers
EDWA Erbium-Doped Waveguide Amplifiers
E-FEC Enhanced Forward Error Correction
EON Elastic Optical Network
Er Erbium
ERO Explicit Route Object
ESA European Space Agency
ETSI European Telecommunications Standards Institute
EVM Error Vector Magnitude
FBA Fibre Brillouin Amplification
FEC Forward Error Correction
FMF Few-Mode Fibres
FON Flexible Optical Network
FTTx Fibre To The x
FWM Four-Wave Mixing
Gb Gigabit
Gb/s Gigabit per second
GE Gigabit Ethernet
G-FEC Generic Forward Error Correction
GFP Generic Framing Procedure
GHz Gigahertz
GMPLS Generalised Multi-Protocol Label Switching
GNSS Global Navigation Satellite System
GPIB General Purpose Interface Bus
GPS Global Positioning System
HD Hard Decision
HDFA Holmium-Doped Fibre Amplifiers
HG-FEC High-Grain Forward Error Correction
HO High Order
Hz Hertz
IEEE Institute of Electrical and Electronics Engineers
IEEE-SA IEEE Standards Association
IETF Internet Engineering Task Force
IGMP Internet Group Management Protocol
IL Insertion Loss
ILA In-Line Amplifier
I/O Input and Output
IP Internet Protocol
IRU Indefeasible Right of Use
ITU International Telecommunication Union
ITU-T ITU Telecommunication Standardisation Sector
JRA Joint Research Activity
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 97
Glossary

JRA1 GN4-1 Joint Research Activity 1 Future Network Topologies

kHz Kilohertz
km Kilometre
Ln Layer n
LAG Link Aggregation Group
LAN Local Area Network
L-band Long band
LINC A pure OpenFlow software switch written in Erlang
LO Low Order
LS Link State
LSP Label Switched Path
LSPDB Label Switch Path Database
MAC Media Access Control
MCF Multi-Core Fibres
MIB Management Information Base
MIMO Multiple Input Multiple Output
MMF Multi-Mode Fibres
MOMoT Multi-Domain Optical Modelling Tool
MP-AFRO Multi-Path After-Failure Repair Optimisation
MPLS Multi-Protocol Label Switching
ms milliseconds
NBI Northbound Interface
NE Network Element
NetConf Network Configuration Protocol
NG Next Generation
nm Nanometre
NMI National Metrology Institute
NMS Network Management System
NOC Network Operations Centre
NREN National Research and Education Network
NRZ Non Return to Zero
ns nanoseconds
NTP Network Time Protocol
OADM Optical Add/Drop Multiplexers
OAM Operations, Administration and Maintenance
O-band Original band
OCh Optical Channel
ODTUG Optical channel Data Tributary Unit Group
ODTUjk Optical channel Data Tributary Unit j into k
ODU Optical Channel Data Unit
ODUk Optical Channel Data Unit-k
OE Open Extension
OF OpenFlow
OFC Optical Fibre Communication Conference and Exposition
OFP OpenFlow Protocol
OIF Optical Internetworking Forum
OMU Optical Multiplex Unit
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 98
Glossary

ONF Open Networking Foundation

ON.Lab Open Networking Lab
ONOS Open Network Operating System
ONP Open Network Platform
OOK On-Off Keying
OPTIME High precision dissemination of time and frequency reference signals project
OPU Optical Channel Payload Unit
OPUk Optical Channel Payload Unit-k
OSNR Optical Signal-to-Noise Ratio
OSPF Open Shortest Path First
OSS Operations Support System
OTN Optical Transport Network
OTU Optical Channel Transport Unit
OTWG Optical Transport Working Group
OXC Optical Cross-Connects
P2P Point to Point
PAC.C Packet and Circuit Network Convergence
PAM Pulse-Amplitude Modulation
PCE Path Computation Element
PHARAO Projet d’Horloge Atomique par Refroidissement d’Atomes en Orbite
PHYs Physical Layer
PMD Polarisation Mode Dispersion
PM-QPSK Polarisation Multiplexing-QPSK
PoP Point of Presence
PoSK Polarisation-Shift Keying
PPP Precise Point Positioning
PSK Phase-Shift Keying
PTB National Metrology Institute, Germany (Physikalisch-Technische Bundesanstalt)
PTP Precision Time Protocol
QAM Quadrature Amplitude Modulation
QPSK Quadrature Phase-Shift Keying
R Receiver
REACTION Research and Experimental Assessment of Control plane archiTectures for In-
Operation flexgrid Network re-optimisation
REFIMEVE+ Metrological Fibre Network with European Vocation + (REseau FIbré MEtrologique à
Vocation Européenne+)
REST REpresentational State Transfer
RFC Request for Comments
RFQ Request for Quotation
RLS Repeater Laser Stations
ROADM Reconfigurable Optical Add-Drop Multiplexer
RSA Routing and Spectrum Allocation
S-band Short band
SBI Southbound Interface
SBS Stimulated Bouillon Scattering
S-BVT Sliceable Bandwidth Variable Transponders
SC Super-Channel
Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 99
Glossary

SD Soft Decision
SD-FEC Soft-Decision Forward Error Correction
SDH Synchronous Digital Hierarchy
SDM Space-Division Multiplexing
SDN Software-Defined Networks [or Networking]
SMF Single-Mode Fibre
SNMP Simple Network Management Protocol
SNR Signal-to-Noise Ratio
SOI Silicon On Insulator
SPM Self-Phase Modulation
SCPP Sub-Carrier Power Pre-emphasis
SRS Stimulated Raman Scattering
SSS Spectrum Selective Switch
SV Single Vendor
T Transmitter
T1 JRA1 Current and Future Network Technologies
Tb Terabit
Tb/s Terabit per second
TC Transponder Class
TCM Tandem Connection Monitoring
TDEV Time Deviation
TDFA Thulium-Doped Fibre Amplifiers
TE Traffic Engineering
TED Traffic Engineering Database
TTI Trial Termination Identifier
U-FEC Universal Forward Error Correction
USB Universal Serial Bus
UTC Coordinated Universal Time
VNT Virtual Network Topology
VNTM Virtual Network Topology Manager
VOA Variable Optical Attenuator
WAN Wide Area Network
WDM Wavelength Division Multiplexing
WSS Wavelength Selective Switch
XPM Cross-Phase Modulation
YANG A data modelling language used to model configuration and state data manipulated
by the network configuration protocol (NETCONF), NETCONF remote procedure calls,
and NETCONF notifications
ZBLAN ZrF4-BaF2-LaF3-AlF3-NaF glass
ZTC Zero-Touch Connectivity

Deliverable D13.1
Research and Study Network Technologies –
White Paper
Document Code: GN4-1-16-7D602 100