Solaris 11: interface numbering and PCIe network card issues
I have SunFire 4170M3 with four 10GbE interfaces on-board plus PCIe card with two additioanl 10GbE interfaces (these two not connected at the moment so 'down' state is ok).
After installing Solaris 11 I was able to see only two on-board interfaces and two PCIe interfaces:
-----
root@marcus:~# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net4 Ethernet up 10 full usbecm0
net0 Ethernet up 1000 full ixgbe0
net2 Ethernet down 0 unknown ixgbe2
net1 Ethernet up 1000 full ixgbe1
net3 Ethernet down 0 unknown ixgbe3
root@marcus:~# dladm show-phys -L
LINK DEVICE LOC
net0 ixgbe0 /SYS/MB
net1 ixgbe1 /SYS/MB
net2 ixgbe2 /SYS/MB/RISER3/PCIE3
net3 ixgbe3 /SYS/MB/RISER3/PCIE3
net4 usbecm0 --
root@marcus:~# grep ixgbe /etc/path_to_inst
"/pci@0,0/pci8086,3c04@2/pci108e,484d@0" 0 "ixgbe"
"/pci@0,0/pci8086,3c04@2/pci108e,484d@0,1" 1 "ixgbe"
"/pci@0,0/pci8086,3c08@3/pci108e,7b11@0" 2 "ixgbe"
"/pci@0,0/pci8086,3c08@3/pci108e,7b11@0,1" 3 "ixgbe"
root@marcus:~# prtconf -D | grep ixgbe
pci108e,484d, instance #0 (driver name: ixgbe)
pci108e,484d, instance #1 (driver name: ixgbe)
pci108e,7b11, instance #2 (driver name: ixgbe)
pci108e,7b11, instance #3 (driver name: ixgbe)
pci108e,484d, instance #4 (driver name: ixgbe)
pci108e,484d, instance #5 (driver name: ixgbe)
-----
After turning off 3-rd PCI slot (where net card is installed) in BIOS two missing on-board interfaces appeared:
-----
root@marcus:~# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net4 Ethernet up 10 full usbecm0
net0 Ethernet up 1000 full ixgbe0
net2 Ethernet down 0 unknown ixgbe2
net1 Ethernet up 1000 full ixgbe1
net3 Ethernet down 0 unknown ixgbe3
net7 Ethernet up 1000 full ixgbe4
net5 Ethernet up 1000 full ixgbe5
root@marcus:~# dladm show-phys -L
LINK DEVICE LOC
net0 ixgbe0 /SYS/MB
net1 ixgbe1 /SYS/MB
net2 ixgbe2 /SYS/MB/RISER3/PCIE3
net3 ixgbe3 /SYS/MB/RISER3/PCIE3
net7 ixgbe4 /SYS/MB
net5 ixgbe5 /SYS/MB
net4 usbecm0 --
root@marcus:~# grep ixgbe /etc/path_to_inst
"/pci@0,0/pci8086,3c04@2/pci108e,484d@0" 0 "ixgbe"
"/pci@0,0/pci8086,3c04@2/pci108e,484d@0,1" 1 "ixgbe"
"/pci@0,0/pci8086,3c08@3/pci108e,7b11@0" 2 "ixgbe"
"/pci@0,0/pci8086,3c08@3/pci108e,7b11@0,1" 3 "ixgbe"
"/pci@78,0/pci8086,3c02@1/pci108e,484d@0" 4 "ixgbe"
"/pci@78,0/pci8086,3c02@1/pci108e,484d@0,1" 5 "ixgbe"
root@marcus:~# prtconf -D | grep ixgbe
pci108e,484d, instance #0 (driver name: ixgbe)
pci108e,484d, instance #1 (driver name: ixgbe)
pci108e,7b11, instance #2 (driver name: ixgbe)
pci108e,7b11, instance #3 (driver name: ixgbe)
pci108e,484d, instance #4 (driver name: ixgbe)
pci108e,484d, instance #5 (driver name: ixgbe)
-----
So far questions are
* why 2 from 4 on-board interfaces where missing initially? prtconf shows them, so looks like they got 'overrided' by PCIe-card interfaces somehow. Reboots or attempts to reconfigure didn't get them initialized. Looks like bug.
* how do I get on-board interfaces listed first (ixgbe0-3), then PCIe (ixgbe4-5) ?
* why link name net6 not appeared while net5 and net7 both appeared? I can rename though.
* what is usbecm0? is it keyboard redirection when using ilom?