We are currently running into an issue where the configured bond just freezes and restarting the network service bring it back up.
We have two bonds in each servers, both bonds are configured with two 10G network interfaces, one is used for private inter-connect (non-routable) and other is used for public (routable). Both of them are configured to use LACP with transmit has policy set to layer3+4. I will paste the individual interface configuration below.
We have four 10G ports in each server and we were using one port from each card to create a bond and it was causing issues. We recently made a change and used both ports from each card to create the bond and so far we don't see any issues.
I have been reading through the posts and have already tried the suggested solutions like network manager service is off.
Let me know if you need any other information. I will appreciate any help or pointers on this issue.
[root@shidbora123 log]# cat /etc/oracle-release
Oracle Linux Server release 6.5
[root@shidbora123 log]# uname -a
Linux shidbora123.cppib.ca 3.8.13-44.1.1.el6uek.x86_64 #2 SMP Wed Sep 10 06:10:25 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@shidbora123 network-scripts]# cat ifcfg-bond101
DEVICE=bond101
IPADDR=10.78.16.125
NETMASK=255.255.254.0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
USERCTL=no
IPV6INIT=no
BONDING_OPTS="mode=4 miimon=100 updelay=200 downdelay=200 xmit_hash_policy=layer3+4"
[root@shidbora123 network-scripts]# cat ifcfg-bond102
DEVICE=bond102
IPADDR=10.78.205.125
NETMASK=255.255.255.0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
USERCTL=no
IPV6INIT=no
BONDING_OPTS="mode=4 miimon=100 updelay=200 downdelay=200 xmit_hash_policy=layer3+4"
MTU=9000
[root@shidbora123 network-scripts]# cat ifcfg-eth0
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
MASTER=bond101
SLAVE=yes
USERCTL=no
[root@shidbora123 network-scripts]# cat ifcfg-eth1
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
MASTER=bond101
SLAVE=yes
USERCTL=no
[root@shidbora123 network-scripts]# cat ifcfg-eth8
DEVICE=eth8
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
MASTER=bond102
SLAVE=yes
USERCTL=no
[root@shidbora123 network-scripts]# cat ifcfg-eth9
DEVICE=eth9
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
MASTER=bond102
SLAVE=yes
USERCTL=no
[root@shidbora123 network-scripts]# cat /proc/net/bonding/bond101
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200
802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 3
Number of ports: 2
Actor Key: 33
Partner Key: 32794
Partner Mac Address: 00:23:04:ee:be:37
Slave Interface: eth1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: 5c:f3:fc:37:b7:c4
Aggregator ID: 3
Slave queue ID: 0
Slave Interface: eth0
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 5c:f3:fc:37:b7:c0
Aggregator ID: 3
Slave queue ID: 0
[root@shidbora123 network-scripts]# cat /proc/net/bonding/bond102
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200
802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 2
Number of ports: 2
Actor Key: 33
Partner Key: 32814
Partner Mac Address: 00:23:04:ee:be:37
Slave Interface: eth8
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: 00:0e:1e:34:05:c0
Aggregator ID: 2
Slave queue ID: 0
Slave Interface: eth9
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 3
Permanent HW addr: 00:0e:1e:34:05:c4
Aggregator ID: 2
Slave queue ID: 0