A Deep Dive into the Ethernet
1
Outline
• Cabling
• Modulation schemes: 10MBit - 100GBit
• Autonegotiation
• Energy-efficient Ethernet
• Power over Ethernet
• Flow control
2
1980 1995 1999 2006
10Mb → 100Mb → Gigabit → 10Gb
200/400
40Gb → 100Gb → Gb → Terabit
2010 2010 2017 > 2020
3
Cabling
Twisted-pair Fiber Twinaxial
4
Twisted-pair copper
RJ45 jack
Category Max Bandwidth Length
supported
Cat 3 10Mb 16MHz 100m
Cat5/5e 1Gb 100MHz 100m
Cat6 10Gb 250MHz 55m
Cat6a 10Gb 500MHz 100m
Cat8 40Gb 2GHz 36m
2017-2018
No plans for twisted-pair support at 100Gb
100Gb expected to use 20GBaud→ Use fiber
5
Data rate Bandwidth Cable
10MBit 10MHz 16MHz
100MBit 31.25MHz 100MHz
1GBit 62.5MHz 100MHz
2.5GBit 100MHz 100MHz
5GBit 200MHz 250MHz
10GBit 400MHz 500MHz
6
Rubber jacket Aluminium shield
Copper wire
drain
10/100Mb uses 2 pairs Gigabit onwards uses 4 pairs
Green + white +TX Green + white +A
Green -TX Green -A
White + orange +RX Orange + white +B
Orange -RX Orange -B
Blue + white +C
Blue -C
100MBit onwards
use full-duplex Brown + white +D
Brown -D
7
Fiber
• Isolated from external electrical noise Data rate Range
100MBit 2km
• Longer range 1GBit 1km
10GBit 400m
• 40Gb - 12 fibers 40/100GBit 150m
• 100Gb - 24 fibers
• LED at 850nm for 100MBit
• Laser for 1GBit and above
8
Shorter range due to modal dispersion
Signal spreads out in time
850 - 1300nm
50μm core
125μm cladding
1310 - 1550nm
8-10μm core
125μm cladding
More $$$ Sheet of paper is 100μm thick
10MBit
• 10MHz clock
• 1 bit every 100 nanoseconds
• Manchester phase encoding
• Polarity reversal
10
0 0 0 1 1 0 0 1 0 0 0
2.5V
-2.5V
100ns
11
100MBit (Fast Ethernet)
16 hex symbols
4 bits 4B/5B Scramble MLT-3 16 control symbols
At least 2 bit Spread across Reduce
transitions per multiple bandwidth
block for clock frequencies to
recovery reduce
interference
1V
0V
-1V
12
0: no transition, 1: transition 1→0→-1→0→1
13
4B/5B creates 125MBaud signal
MLT-3 reduces fundamental frequency to 31.25MHz
14
Gigabit Ethernet
• 125MBaud per twisted pair.
• 500MBaud in total (2 bits per symbol)
• 4D-PAM5 (5 voltage levels, spread across 4 channels)
• Trellis modulation for parity bit
15
Spread Generate
spectrum Generate symbols
parity bit
Channels are
interdependent
54=625 symbols in total
Encode 9 bit word: 8 data bits + 1 parity bit.
29=512 possible bitstrings
Remaining 113 symbols used for control or discarded
16
Parity bit generation
Trellis/convolutional encoder
XOR XOR
D flip-flops
17
2D-PAM5
Attenuation - points Noise -
move closer points blur
Subsequent symbols must have a Euclidean distance of at least X
Only specific sequences are allowed.
Decoder chooses most likely sequences using Hamming distance as
distance metric
18
Symbols in a
subset have
distance 2
# of coordinates
In 4D-PAM5 we have 16
subsets.
(XXXX, XXXY,…,YYYY)
Symbols have distance 4
19
Group 16 subsets into 8 sub-lattices:
Each pair of subsets are complements of each other
Distance between any two symbols within sub lattice >= 4
9 bit word → 1 of 512 symbols → voltage levels
20
10GBit
Each twisted pair takes care of
2.5Gbps both ways
About 400MHz bandwidth
162=256
symbols
800MBaud
Scramble
32 bits
64B/65B LDPC 2PAM16 DSQ-128
32 bits
BER: 10-12
Prune to get
2048 bits = 128 symbols
1723 data + 7 bits per
325 check symbol
21
LDPC example
1 0 1 0 1 0 Received bits
0,1 1,0 1,1 1 1 0 Checked bits
1 0 1 1 1 0 Corrected bits
Variable nodes
v0 v1 v2 v3 v4 v5 (2048)
Check nodes
c0 c1 c2 (325)
V0: V1 ⊕ V3 → 0 V1 - 0 V0 - 1 1. v nodes send bits to c nodes
2. c nodes send a v node the ⊕
V1: V0 ⊕ V3 → 1 V2 - 1 V2 - 1 of other received values
V3: V0 ⊕ V1 → 1 V4 - 1 V5 - 0 e.g. c0 sends v0 the value v1⊕v4
3. v nodes do a majority vote
22
40GBit
• 64/66 bit encoder from
10GBit
• 10.3125GBaud per lane
• Round robin across 4
PCS (physical coding
sublayer)
• PCS does coding and
negotiation
• Lanes can be optical or
twisted cable
23
100GBit
20 PCS lanes
Multiplexers
10 electrical lines
Framing, octet sync, de/scramble
Convert to bits to signal
4 optical lanes
25GBit/s
(4 different
per lane
wavelengths)
Single pair of optical cables
24
Datacenters
• Facebook (2014)
• 4x40Gbps
25
Autonegotiation
• Negotiates highest speed, duplex + other capabilities
• Only requires 2 twisted pairs
• Occurs whenever link is re-connected
• Time
• 2-3 seconds for 10/100MBit
• 5-6 seconds for Gigabit onwards
• Doesn’t check if cable is correct
• Won’t notice if you use Cat3 with 1GBASE
• Auto-downgrades if a speed can’t be established
26
Autonegotiation
• Sends a single Fast Link Pulse (similar to heartbeat)
• 33 bursts, 10ns long each
• 17 odd pulses for clock
• 16 even pulses for data:
• presence/absence => 1/0
• 16-bit link code words
27
Base page message
Next page bit to say if
there’s another message
containing additional
vendor info
LAN type: Speed, duplex, pause ability
Ethernet
Remote fault bit set if Set to 1 after
receiver senses failure receiving three
messages
28
Energy efficient ethernet (100MB-10GB)
• By default Ethernet sends IDLE symbols as a heartbeat.
• Channel is always occupied
• Low Power Idle mode
• If there are no frames: shut down
• Savings - $450 million across the US per year
• Idle link: 91% for 1G and 74% for 100MBit
• 191 links with normal bursty traffic: 15%
29
Sleep
Refresh Wake
Send LPI for Time To Sleep
Check for Send IDLE for
If received LPI
disconnection Time to Wake
Go to sleep
30
Power Over Ethernet (10MB - 1GB)
• Alt-A (common-mode): power and data share same wire
• Alt-B (spare-pair): power and data on separate wires
• Up to 25.5W per device
31
Power Over Ethernet
• Detection:
• Send 2.7-10.1V voltage check for 25kΩ resistance
• Classification
I (mA) Pout (W) Precv (W)
• Send 15.5-20.5 voltage 0-4 15.4 0.44-12.95
• Measure current draw 9-12 4 0.44-3.84
17-20 7 3.84-6.49
• Dynamic adjustment. 10 second wait
26-30 15.4 6.49-12.95
• Checks if link is connected 36-44 36 12.95-25.5
• Regulates voltage and current
32
Flow Control
• Pause frame: tell sender to stop transmitting for X time
• Used to handle NICs with small buffers
• Send pause frame to 01-80-C2-00-00-01
33
A Deep Dive into the Ethernet
34
Ethernet Packet
7x 10101010 10101011
Preamble 7 Start frame 1
MAC dst 6 MAC src 6 Header Packet
802.1Q 4 Type/length 2 12
Frame 72-1530
Payload 46-1500
64-1522
CRC 4
Interpacket gap 12
35
Q-tag: VLAN priority indicator
VLAN: LAN within a switch
Router
Subnet Switch
36
Type={IPv4, IPv6, ARP, 802.1Q}
37
