Skip to Main Content
Forums

Infrastructure Software

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Solaris 11.1 release, vlan mtu conectivity problem

CrakeenMay 10 2013 — edited Jul 31 2013
Hi guys Im having conectivity problem with my setup. Im trying to configure my system like this:

1 X86 server with 4 nics 2 bnx and 2 igb conected to HP v1910 switch configured as aggregattion with LACP Active and L2,L3 policy

then I put some vlans over the aggregation VID 101 102 103

and then i put some ip interfaces over each vlan and 1 ip interface on the aggregattion itself.

The thing is , with all configured, everything goes fine, but when i change the mtu of the aggregatiion to something bigger than 1500, i loose conectivity on the vlan interfaces not in the aggrgatted ip.

the courious thing is that when i start capturing packets on the vlan or in the aggregation with snoop (in promiscuous mode or not) everything start to work,, and as soon i stop the capture, all stop working again.

My system info:

root@BKP-SERVER:~# cat /etc/release

Oracle Solaris 11.1 X86
Copyright (c) 1983, 2012, Oracle and/or its affiliates. All rights reserved.
Assembled 19 September 2012

root@BKP-SERVER:~# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 1000 full bnx0
net1 Ethernet up 1000 full bnx1
net2 Ethernet up 1000 full igb0
net3 Ethernet up 1000 full igb1

root@BKP-SERVER:~# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net1 phys 1500 up --
net2 phys 1500 up --
net3 phys 1500 up --
aggr1 aggr 1500 up net0 net1 net2 net3
vlan101 vlan 1500 up aggr1
vlan102 vlan 1500 up aggr1
vlan103 vlan 1500 up aggr1

root@BKP-SERVER:~# dladm show-vlan
LINK VID OVER FLAGS
vlan101 101 aggr1 -----
vlan102 102 aggr1 -----
vlan103 103 aggr1 -----


root@BKP-SERVER:~# ipadm show-if
IFNAME CLASS STATE ACTIVE OVER
lo0 loopback ok yes --
aggr1 ip ok yes --
vlan101 ip ok yes --
vlan102 ip ok yes --
vlan103 ip ok yes --


root@BKP-SERVER:~# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
aggr1/v4 static ok 10.9.0.43/16
vlan101/v4 static ok 192.167.1.1/24
vlan101/v4a static ok 192.167.1.2/24
vlan101/v4b static ok 192.167.1.3/24
vlan101/v4c static ok 192.167.1.4/24
vlan102/v4 static ok 192.167.2.1/24
vlan102/v4a static ok 192.167.2.2/24
vlan102/v4b static ok 192.167.2.3/24
vlan102/v4c static ok 192.167.2.4/24
vlan103/v4 static ok 192.167.3.1/24
vlan103/v4a static ok 192.167.3.2/24
vlan103/v4b static ok 192.167.3.3/24
vlan103/v4c static ok 192.167.3.4/24
lo0/v6 static ok ::1/128

pinging from mi S11 box to a server with ip 192.167.2.20 with already tested jumbo frames support and assigned to vlan 102

root@BKP-SERVER:~# ping -s 192.167.2.20 1472 4
PING 192.167.2.20: 1472 data bytes
1480 bytes from 192.167.2.20: icmp_seq=0. time=0.485 ms
1480 bytes from 192.167.2.20: icmp_seq=1. time=0.353 ms
1480 bytes from 192.167.2.20: icmp_seq=2. time=0.528 ms
1480 bytes from 192.167.2.20: icmp_seq=3. time=0.359 ms

pinging from server over the aggregated link ip 10.9.0.43 to 10.9.1.1

root@BKP-SERVER:~# ping -s 10.9.1.1 1472 4
PING 10.9.1.1: 1472 data bytes
1480 bytes from 10.9.1.1: icmp_seq=0. time=1.462 ms
1480 bytes from 10.9.1.1: icmp_seq=1. time=2.959 ms
1480 bytes from 10.9.1.1: icmp_seq=2. time=0.394 ms
1480 bytes from 10.9.1.1: icmp_seq=3. time=0.330 ms

----10.9.1.1 PING Statistics----
4 packets transmitted, 4 packets received, 0% packet loss
round-trip (ms) min/avg/max/stddev = 0.330/1.286/2.959/1.230

now i change the mtu ofthe aggregation:

root@BKP-SERVER:~# dladm set-linkprop -p mtu=9000 aggr1

root@BKP-SERVER:~# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 9000 up --
net1 phys 9000 up --
net2 phys 9000 up --
net3 phys 9000 up --
aggr1 aggr 9000 up net0 net1 net2 net3
vlan101 vlan 1500 up aggr1
vlan102 vlan 1500 up aggr1
vlan103 vlan 1500 up aggr1

now the ping over vlan:

root@BKP-SERVER:~# ping -s 192.167.2.20 1472 4
PING 192.167.2.20: 1472 data bytes

----192.167.2.20 PING Statistics----
4 packets transmitted, 0 packets received, 100% packet loss

now the ping over the aggregated interface

root@BKP-SERVER:~# ping -s 10.9.1.1 1472 4
PING 10.9.1.1: 1472 data bytes
1480 bytes from 10.9.1.1: icmp_seq=0. time=0.496 ms
1480 bytes from 10.9.1.1: icmp_seq=1. time=0.818 ms
1480 bytes from 10.9.1.1: icmp_seq=2. time=0.422 ms
1480 bytes from 10.9.1.1: icmp_seq=3. time=0.427 ms

----10.9.1.1 PING Statistics----
4 packets transmitted, 4 packets received, 0% packet loss
round-trip (ms) min/avg/max/stddev = 0.422/0.541/0.818/0.188

now trying jumbo frames over vlan:

root@BKP-SERVER:~# ping -s 192.167.2.20 8500 4
PING 192.167.2.20: 8500 data bytes

----192.167.2.20 PING Statistics----
4 packets transmitted, 0 packets received, 100% packet loss

now trying jumbo frames over the aggregated link

root@BKP-SERVER:~# ping -s 10.9.1.1 8500 4
PING 10.9.1.1: 8500 data bytes
8508 bytes from 10.9.1.1: icmp_seq=0. time=1.270 ms
8508 bytes from 10.9.1.1: icmp_seq=1. time=0.922 ms
8508 bytes from 10.9.1.1: icmp_seq=2. time=1.612 ms
8508 bytes from 10.9.1.1: icmp_seq=3. time=0.859 ms

----10.9.1.1 PING Statistics----
4 packets transmitted, 4 packets received, 0% packet loss
round-trip (ms) min/avg/max/stddev = 0.859/1.166/1.612/0.348

now as soon i start capture packets with any of this

snoop -V -P -d aggr1 icmp (not promiscuous)
snoop -V -P -d vlan102 icmp (not promiscuous)
snoop -V -d aggr1 icmp
snoop -V -d vlan102 icmp

root@BKP-SERVER:~# ping -s 192.167.2.20 8500 4
PING 192.167.2.20: 8500 data bytes
8508 bytes from 192.167.2.20: icmp_seq=0. time=0.825 ms
8508 bytes from 192.167.2.20: icmp_seq=1. time=0.781 ms
8508 bytes from 192.167.2.20: icmp_seq=2. time=0.838 ms
8508 bytes from 192.167.2.20: icmp_seq=3. time=0.787 ms

----192.167.2.20 PING Statistics----
4 packets transmitted, 4 packets received, 0% packet loss
round-trip (ms) min/avg/max/stddev = 0.781/0.808/0.838/0.028

now i change the mtu on the vlans itselfs

root@BKP-SERVER:~# dladm set-linkprop -p mtu=9000 vlan101
root@BKP-SERVER:~# dladm set-linkprop -p mtu=9000 vlan102
root@BKP-SERVER:~# dladm set-linkprop -p mtu=9000 vlan103

root@BKP-SERVER:~# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 9000 up --
net1 phys 9000 up --
net2 phys 9000 up --
net3 phys 9000 up --
aggr1 aggr 9000 up net0 net1 net2 net3
vlan101 vlan 9000 up aggr1
vlan102 vlan 9000 up aggr1
vlan103 vlan 9000 up aggr1

ping over vlan again and same result

root@BKP-SERVER:~# ping -s 192.167.2.20 8000 4
PING 192.167.2.20: 8000 data bytes

----192.167.2.20 PING Statistics----
4 packets transmitted, 0 packets received, 100% packet loss


with packet capture on:

root@BKP-SERVER:~# ping -s 192.167.2.20 8000 4
PING 192.167.2.20: 8000 data bytes
8008 bytes from 192.167.2.20: icmp_seq=0. time=0.928 ms
8008 bytes from 192.167.2.20: icmp_seq=1. time=0.917 ms
8008 bytes from 192.167.2.20: icmp_seq=2. time=0.930 ms
8008 bytes from 192.167.2.20: icmp_seq=3. time=0.780 ms

----192.167.2.20 PING Statistics----
4 packets transmitted, 4 packets received, 0% packet loss
round-trip (ms) min/avg/max/stddev = 0.780/0.889/0.930/0.073

packet capture output:

192.167.2.1 -> 192.167.2.20 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.1 -> 192.167.2.20 IP D=192.167.2.20 S=192.167.2.1 LEN=8028, ID=52183, TOS=0x0, TTL=255
192.167.2.1 -> 192.167.2.20 ICMP Echo request (ID: 1372 Sequence number: 3)
________________________________
192.167.2.20 -> 192.167.2.1 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.20 -> 192.167.2.1 IP D=192.167.2.1 S=192.167.2.20 LEN=8028, ID=8028, TOS=0x0, TTL=255
192.167.2.20 -> 192.167.2.1 ICMP Echo reply (ID: 1372 Sequence number: 3)
________________________________
192.167.2.1 -> 192.167.2.20 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.1 -> 192.167.2.20 IP D=192.167.2.20 S=192.167.2.1 LEN=8028, ID=52187, TOS=0x0, TTL=255
192.167.2.1 -> 192.167.2.20 ICMP Echo request (ID: 1373 Sequence number: 0)
________________________________
192.167.2.20 -> 192.167.2.1 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.20 -> 192.167.2.1 IP D=192.167.2.1 S=192.167.2.20 LEN=8028, ID=8029, TOS=0x0, TTL=255
192.167.2.20 -> 192.167.2.1 ICMP Echo reply (ID: 1373 Sequence number: 0)
________________________________
192.167.2.1 -> 192.167.2.20 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.1 -> 192.167.2.20 IP D=192.167.2.20 S=192.167.2.1 LEN=8028, ID=52189, TOS=0x0, TTL=255
192.167.2.1 -> 192.167.2.20 ICMP Echo request (ID: 1373 Sequence number: 1)
________________________________
192.167.2.20 -> 192.167.2.1 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.20 -> 192.167.2.1 IP D=192.167.2.1 S=192.167.2.20 LEN=8028, ID=8030, TOS=0x0, TTL=255
192.167.2.20 -> 192.167.2.1 ICMP Echo reply (ID: 1373 Sequence number: 1)
________________________________
192.167.2.1 -> 192.167.2.20 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.1 -> 192.167.2.20 IP D=192.167.2.20 S=192.167.2.1 LEN=8028, ID=52190, TOS=0x0, TTL=255
192.167.2.1 -> 192.167.2.20 ICMP Echo request (ID: 1373 Sequence number: 2)
________________________________
192.167.2.20 -> 192.167.2.1 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.20 -> 192.167.2.1 IP D=192.167.2.1 S=192.167.2.20 LEN=8028, ID=8031, TOS=0x0, TTL=255
192.167.2.20 -> 192.167.2.1 ICMP Echo reply (ID: 1373 Sequence number: 2)
________________________________
192.167.2.1 -> 192.167.2.20 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.1 -> 192.167.2.20 IP D=192.167.2.20 S=192.167.2.1 LEN=8028, ID=52191, TOS=0x0, TTL=255
192.167.2.1 -> 192.167.2.20 ICMP Echo request (ID: 1373 Sequence number: 3)
________________________________
192.167.2.20 -> 192.167.2.1 ETHER Type=0800 (IP), VLAN ID=102, size=8046 bytes
192.167.2.20 -> 192.167.2.1 IP D=192.167.2.1 S=192.167.2.20 LEN=8028, ID=8032, TOS=0x0, TTL=255
192.167.2.20 -> 192.167.2.1 ICMP Echo reply (ID: 1373 Sequence number: 3)

there is any solution for this problem? or there is any related bug for this?

any idea? :(

Ty very much in advance and sorry for my english.

Edited by: 957695 on 10-may-2013 6:00
Comments
Locked Post
New comments cannot be posted to this locked post.
Post Details
Locked on Aug 28 2013
Added on May 10 2013
#network, #oracle-solaris, #solaris, #solaris-11
3 comments
1,566 views