Tutorial on Networking for Digital
Substations
Harsh Vardhan – GE Grid Solutions
2019 Texas A&MProtective Relaying Conference
Agenda
• Performa nce requirements
• Good Pra ctices
• Conclusion
Digital Substation
Network (SCADA / Station Bus) PTP GOOSE MMS
Relay BCU DFR Relay BCU Relay Relay Relay
Bay 1 Bay 2 Bay 3 Bay 4 Bay 5 Bay 6 Bay 7 Bay 8
MU MU MU MU MU MU MU MU
MU MU MU MU MU MU MU MU
RIO RIO RIO RIO RIO RIO RIO RIO
Network (Process Bus) GOOSE SV PTP
– Bandwidth Requirements
Digital Substation messages
• GOOSEmessa ging – sta tus a nd control fla gs between devices
• Sampled Values (SV) messa ging – insta nta neous sa mples of current a nd volta ge
published by merging units (MUs) to other IEDs
• IEEE 1588 (PTP) messa ging – highly a ccura te time synchroniza tion over pa cket-
ba sed networks
Message Steady State Burst
GOOSE ~1 Kb/s ~1 Mb/s
~5 Mb/s (50 Hz),
SV (Protection) ~6 Mb/s (60 Hz) -NA-
~10 Mb/s (50 Hz),
SV (Metering) -NA-
~12 Mb/s (60 Hz)
PTP ~1 Kb/s -NA-
– Latency Requirements
Digital Substation messages
Message Max. Delay m
( s) Bandwidth Application
GOOSE (Trip) 3 Low Protection
GOOSE (other) 10 to 100 Low Protection
Sa mpled Va lues 4 High Process Bus
PTP - Low Genera l Pha sors, SVs
MMS (low speed) < 100 Low Control
MMS (med speed) < 500 Low Control
As per IEC 61850-90-4
Selection of Network Devices
Substa tion Ha rdened, Dua l Power Supply, Wa tchdog conta ct
Number a nd Type of Ethernet ports : Don’t forget to include spa re ports
Port Speed, Ba ndwidth : Trunk Ports, Edge ports Ba ndwidth req might be different
La yer 2 Functiona lities: MAC, VLAN, ACL for tra ffic segrega tion
PTP : Tra nspa rent clock (1-step or 2-step)
Redunda ncy : PRP, HSR, RSTP, RedBox etc.
Other fea tures :
• SNMP for monitoring
• IGMP for PMU/ R-GOOSE a pplica tion
• L3 fea tures
• Cybersecurity etc.
Good Practices
-Receiver as close as possible
Keep Sender
• Minimize La tency
• Minimize Ba ndwidth requirements on Trunk Ports
• Avoid Loss of pa ckets during congestion
- RSTP
Redundancy of Network
- PRP
Redundancy of Network
SAN DAN
RedBox
DAN DAN
SAN
- HSR
Redundancy of Network
Issues with Large
Network:
• Bigger the ring, bigger
a re the cha nces of
ha ving more tha n
single fa ilure
HSR HSR
• All the IEDs need to
support a ll the da ta
flow
– MAC Filtering
Network Traffic Segregation
A MAC a ddress is given to a network
a da pter when it is ma nufactured. It is
ha rdwired or ha rd-coded onto your
computer's network interfa ce ca rd (NIC)
a nd is unique to it
A MAC Address Table is used to define
the aging time of learned MAC
addresses, their mode of operation,
whitelisting and blacklisting devices
according to their right to access
subnetworks.
– MAC Filtering
Network Traffic Segregation
• Static MAC Filtering
• Dynamic MAC Filteringusing GMRP etc.
IED A: GOOSE Publishing Pa ra meters
– VLAN
Network Traffic Segregation
Virtua l LAN technology a llows logica l sepa ra tion of the physica l network.
- VLAN
Network Traffic Segregation
IED A: GOOSE Publishing Pa ra meters
Access Control Lists
• Block or a llow whole types of tra ffic ba sed on EtherType
• Ca n explicitly block or a llow GOOSE, SV, PTP on a ny ports
• Deny a ll other tra ffic by defa ult
– Prioritizing Traffic
Quality of Service (QoS)
incoming data outgoingdata
data packet switching data packet
functions
data packet
data packet
data packet
high priority high priority
data packet data packet
Message Priority in network QoS
PTP Medium 4
Sa mpled Va lues Medium 4
GOOSE (control) High 5
GOOSE (trip) Highest 6
MMS a nd TCP/IP comms Low 0
Network FAT
• Wha t a FAT is:
• Off-site commissioning of the substa tion network
• Performed with softwa re simula tors / test equipment
• Tested in a controlled scena rio
• Wha t a FAT does:
• Reduce commissioning time
• Reduces insta lla tion time
• On-site work is verifying communica tions ca ble connections
Summary
• Our substa tion networks don’t cha nge frequently
• Tra ffic flows a re consistent a nd predicta ble
• Network requirements only cha nge with ca pita l projects
• Simple, sta nda rd networking tools cover our needs
• Link speed to provide ba ndwidth
• Tra ffic sha ping through VLANs, MAC a ddress filtering, Access Control Lists
• QoS, tra ffic sha ping to a ddress la tency
• Network topology (RSTP, PRP, HSR) provides relia bility
• First step is good project engineering
Thank You
Questions?
