Linux allows binding of multiple network interfaces into a single channel/NIC using special
kernel module called bonding.
When bonded, two NICs appear to be the same physical device and they also have the same
MAC address. L
NIC bonding may also be called NIC Teaming.
Mode Types for EL5
Most how-to also sugget to add the following line: options bond0 mode=0 miimon=100 or options
bond0 mode=1 miimon=100
But the mode parameter has a meaning that the system administrator should choose:
Mode 0 or balance-rr is the Round-robin policy which gives fault tolerance and load balancing.
This mode sends are receives package on each node in a sequential order. So the load is
distributed on all NICs.
Mode 1 or active-backup which gives only fault tolerance without load balancing. This modes
sends all packets through the one active slave. The slave changes only if the active slave fails.
bonding modes for EL6
Several policies are available in bonding, this mode can be set using directive mode=<value>
The <value> can be one of:
balance-rr or 0 Sets a round-robin policy for fault tolerance and load balancing. Transmissions are
received and sent out sequentially on each bonded slave interface beginning with the first one
available.
active-backup or 1 Sets an active-backup policy for fault tolerance. Transmissions are received
and sent out via the first available bonded slave interface. Another bonded slave interface is only used
if the active bonded slave interface fails.
balance-xor or 2 Sets an XOR (exclusive-or) policy for fault tolerance and load balancing. Using
this method, the interface matches up the incoming request's MAC address with the MAC address for
one of the slave NICs. Once this link is established, transmissions are sent out sequentially beginning
with the first available interface.
broadcast or 3 Sets a broadcast policy for fault tolerance. All transmissions are sent on all slave
interfaces.
802.3ad or 4 Sets an IEEE 802.3ad dynamic link aggregation policy. Creates aggregation groups
that share the same speed and duplex settings. Transmits and receives on all slaves in the active
aggregator. Requires a switch that is 802.3ad compliant.
balance-tlb or 5 Sets a Transmit Load Balancing (TLB) policy for fault tolerance and load
balancing. The outgoing traffic is distributed according to the current load on each slave interface.
Incoming traffic is received by the current slave. If the receiving slave fails, another slave takes over
the MAC address of the failed slave.
balance-alb or 6 Sets an Active Load Balancing (ALB) policy for fault tolerance and load balancing.
Includes transmit and receive load balancing for IPV4 traffic. Receive load balancing is achieved
through ARP
NIC Bonding in RHEL5.6 (This config from the DB servers)
1)
2)
3)
Make sure all the NICs are up.
Get the info from N/W team that which NICs need to be bond, based on that do the bonding configuration as per below
Configuration
# cd /etc/sysconfig/network-scripts/
# cat ifcfg-bond0
DEVICE=bond0
IPADDR=10.160.62.204
NETWORK=10.160.62.0
NETMASK=255.255.255.0
USERCTL=no
ONBOOT=yes
GATEWAY=10.160.62.1
BOOTPROTO=none
BONDING_OPTS="mode=0 miimon=100"
# cat ifcfg-bond1
DEVICE=bond1
IPADDR=10.10.10.204
NETWORK=10.10.10.0
NETMASK=255.255.255.0
USERCTL=no
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="mode=0 miimon=100"
# cat ifcfg-eth0
# NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
# cat ifcfg-eth1
# NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter
DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
# cat ifcfg-eth4
# NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter
DEVICE=eth4
BOOTPROTO=none
ONBOOT=yes
MASTER=bond1
SLAVE=yes
USERCTL=no
# cat ifcfg-eth5
# NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter
DEVICE=eth5
BOOTPROTO=none
ONBOOT=yes
MASTER=bond1
SLAVE=yes
USERCTL=no
# cat /etc/modprobe.conf
alias bond0 bonding
alias bond1 bonding
# service network restart or Reboot the server.
4)
Check with the N/W team whether they did the configuration from their end as well
5)
To check the bonding is working or not
# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)
Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Speed: 100 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 3c:4a:92:e5:02:84
Slave Interface: eth1
MII Status: up
Speed: 100 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 3c:4a:92:e5:02:85
# cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)
Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth4
MII Status: up
Speed: 100 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 78:ac:c0:10:68:e8
Slave Interface: eth5
MII Status: up
Speed: 100 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 78:ac:c0:10:68:e9
# route
Kernel IP routing table
Destination Gateway
10.160.62.0 *
10.10.10.0
*
default
10.160.62.1
Genmask
255.255.255.0
255.255.255.0
0.0.0.0
Flags
U
U
UG
Metric
0
0
0
Ref
0
0
0
Use
0
0
0
# ifconfig
bond0 Link encap:Ethernet HWaddr 3C:4A:92:E5:02:84
inet addr:10.160.62.204 Bcast:10.160.62.255 Mask:255.255.255.0
inet6 addr: fe80::3e4a:92ff:fee5:284/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:134767 errors:0 dropped:0 overruns:0 frame:0
TX packets:42006 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:14078464 (13.4 MiB) TX bytes:6235358 (5.9 MiB)
bond1
Link encap:Ethernet HWaddr 78:AC:C0:10:68:E8
inet addr:10.10.10.204 Bcast:10.10.10.255 Mask:255.255.255.0
inet6 addr: fe80::7aac:c0ff:fe10:68e8/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:251442 errors:0 dropped:0 overruns:0 frame:0
TX packets:89314 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:267900904 (255.4 MiB) TX bytes:27824060 (26.5 MiB)
eth0
Link encap:Ethernet HWaddr 3C:4A:92:E5:02:84
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:66699 errors:0 dropped:0 overruns:0 frame:0
TX packets:21007 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:9752919 (9.3 MiB) TX bytes:3048426 (2.9 MiB)
Interrupt:194
eth1
Link encap:Ethernet HWaddr 3C:4A:92:E5:02:84
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
Iface
bond0
bond1
bond0
RX packets:68068 errors:0 dropped:0 overruns:0 frame:0
TX packets:21001 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4325545 (4.1 MiB) TX bytes:3187256 (3.0 MiB)
Interrupt:226
eth2
Link encap:Ethernet HWaddr 3C:4A:92:E5:02:86
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:67
eth3
Link encap:Ethernet HWaddr 3C:4A:92:E5:02:87
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:99
eth4
Link encap:Ethernet HWaddr 78:AC:C0:10:68:E8
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:44662 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:13981132 (13.3 MiB)
Interrupt:139
eth5
Link encap:Ethernet HWaddr 78:AC:C0:10:68:E8
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:251442 errors:0 dropped:0 overruns:0 frame:0
TX packets:44652 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:267900904 (255.4 MiB) TX bytes:13842928 (13.2 MiB)
Interrupt:171
eth6
Link encap:Ethernet HWaddr 78:AC:C0:10:68:EA
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:203
eth7
Link encap:Ethernet HWaddr 78:AC:C0:10:68:EB
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:235
lo
Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:174639 errors:0 dropped:0 overruns:0 frame:0
TX packets:174639 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:84543185 (80.6 MiB) TX bytes:84543185 (80.6 MiB)
NIC Bonding in RHEL 6
1)
2)
Make sure all the NICs are up.
Get the info from N/W team that which NICs need to be bond, based on that do the bonding configuration as per below
3)
Configuration
a) Do similar configuration like above, the only change is as below
b) # cat /etc/modprobe.d/bonding.conf (Create this file as this will not be available)
alias bond0 bonding
VLAN tagging Over NIC Bonding in RHEL 6
4)
5)
Once Bonding completed check whether are you able to ping the gateway.
Now need to tag the VLAN ID over bonding. (This VLAN 30 is for the physical host communication)
# cat ifcfg-bond0
DEVICE=bond0
USERCTL=no
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="mode=0 miimon=100"
#IPADDR=10.228.3.171
#NETWORK=10.228.3.0
#NETMASK=255.255.255.0
#GATEWAY=10.228.3.3
# cat ifcfg-bond0.30
DEVICE=bond0.30
IPADDR=10.228.3.171
NETWORK=10.228.3.0
NETMASK=255.255.255.0
USERCTL=no
ONBOOT=yes
GATEWAY=10.228.3.3
BOOTPROTO=static
TYPE=ethernet
VLAN="yes"
# cat ifcfg-eth2
DEVICE="eth2"
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
#HWADDR="78:AC:C0:10:48:3A"
# cat ifcfg-eth3
DEVICE="eth3"
BOOTPROTO=none
ONBOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
#HWADDR="78:AC:C0:10:48:3B"
# cat ifcfg-eth4
DEVICE="eth4"
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
#HWADDR="98:4B:E1:6E:4E:08"
# cat ifcfg-eth5
DEVICE="eth5"
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
#HWADDR="98:4B:E1:6E:4E:0A"
# service network restart or Reboot the server.
6)
7)
Check with the N/W team whether they did the configuration from their end as well
Once VLAN tagging completed check whether are you able to ping the gateway.
VLAN tagging, Bridging Over NIC Bonding in RHEL 6
1)
Now need to tag the VLAN ID 122 & bridging over bonding. (This VLAN 122 is for the communication between physical &
virtual servers.)
# cat ifcfg-br122
DEVICE=br122
BOOTPROTO=static
TYPE=Bridge
ONBOOT=yes
NETMASK=255.255.255.0
GATEWAY=10.228.12.3
DELAY=0
STP=off
# cat ifcfg-bond0.122
DEVICE=bond0.122
ONBOOT=yes
BRIDGE=br122
TYPE=ethernet
VLAN="yes"
# service network restart or Reboot the server.
# brctl show
bridge name bridge id
STP enabled interfaces
br122
8000.78acc010483a
no
bond0.122
vnet0
virbr0
8000.000000000000
yes
# ifconfig
bond0 Link encap:Ethernet HWaddr 78:AC:C0:10:48:3A
inet6 addr: fe80::7aac:c0ff:fe10:483a/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:15053790 errors:0 dropped:0 overruns:0 frame:0
TX packets:2181 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2640415223 (2.4 GiB) TX bytes:169763 (165.7 KiB)
bond0.30 Link encap:Ethernet HWaddr 78:AC:C0:10:48:3A
inet addr:10.228.3.171 Bcast:10.228.3.255 Mask:255.255.255.0
inet6 addr: fe80::7aac:c0ff:fe10:483a/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:156605 errors:0 dropped:0 overruns:0 frame:0
TX packets:806 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8266736 (7.8 MiB) TX bytes:70373 (68.7 KiB)
bond0.122 Link encap:Ethernet HWaddr 78:AC:C0:10:48:3A
inet6 addr: fe80::7aac:c0ff:fe10:483a/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:5797498 errors:0 dropped:0 overruns:0 frame:0
TX packets:1373 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1025951207 (978.4 MiB) TX bytes:92128 (89.9 KiB)
br122
Link encap:Ethernet HWaddr 78:AC:C0:10:48:3A
inet6 addr: fe80::7aac:c0ff:fe10:483a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3646053 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:213275323 (203.3 MiB) TX bytes:578 (578.0 b)
eth0
Link encap:Ethernet HWaddr 78:AC:C0:10:48:38
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:65
eth1
Link encap:Ethernet HWaddr 78:AC:C0:10:48:39
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:69
eth2
Link encap:Ethernet HWaddr 78:AC:C0:10:48:3A
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:3027008 errors:0 dropped:0 overruns:0 frame:0
TX packets:1158 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:651891641 (621.6 MiB) TX bytes:68546 (66.9 KiB)
Interrupt:73
eth3
Link encap:Ethernet HWaddr 78:AC:C0:10:48:3A
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:6486270 errors:0 dropped:0 overruns:0 frame:0
TX packets:346 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:886182677 (845.1 MiB) TX bytes:32271 (31.5 KiB)
Interrupt:77
eth4
Link encap:Ethernet HWaddr 78:AC:C0:10:48:3A
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:2156935 errors:0 dropped:0 overruns:0 frame:0
TX packets:339 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:390114790 (372.0 MiB) TX bytes:35436 (34.6 KiB)
Interrupt:30 Memory:f4000000-f4012800
eth5
Link encap:Ethernet HWaddr 78:AC:C0:10:48:3A
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:3383578 errors:0 dropped:0 overruns:0 frame:0
TX packets:340 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:712226225 (679.2 MiB) TX bytes:34090 (33.2 KiB)
Interrupt:37 Memory:f2000000-f2012800
eth6
Link encap:Ethernet HWaddr 98:4B:E1:6E:4E:1C
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:31 Memory:f8000000-f8012800
eth7
Link encap:Ethernet HWaddr 98:4B:E1:6E:4E:1E
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:39 Memory:f6000000-f6012800
lo
Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:19425 errors:0 dropped:0 overruns:0 frame:0
TX packets:19425 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:112895185 (107.6 MiB) TX bytes:112895185 (107.6 MiB)
virbr0
Link encap:Ethernet HWaddr 00:00:00:00:00:00
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:384 (384.0 b) TX bytes:3308 (3.2 KiB)
vnet0
Link encap:Ethernet HWaddr FE:54:00:39:7E:82
inet6 addr: fe80::fc54:ff:fe39:7e82/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:557 errors:0 dropped:0 overruns:0 frame:0
TX packets:8032388 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:54056 (52.7 KiB) TX bytes:1551230864 (1.4 GiB)
