We have a bunch of older Sun servers (Sunfire models), with OL6 and OL7 kernels not seeming to support the server h/w (Mellanox Infiniband and NVidia Ethernet specifically).
I tried OL6 on these servers, but the Mellanox Cougar Cub (PCI Infiniband) cards failed to work (were not detected). Installing OL5 resulted in a working Infiniband layer. Server details:
[root@old-rac-03 ~]# dmidecode -t1
# dmidecode 2.12
SMBIOS 2.3 present.
Handle 0x0001, DMI type 1, 25 bytes
System Information
Manufacturer: Sun Microsystems
Product Name: Sun Fire X4100 Server
Version:
Serial Number: xxxxxxxxxxxxxxx
UUID: FF200008-FFFF-FFFF-FFFF-FCFF394F1400
Wake-up Type: Power Switch
[root@old-rac-03 ~]#
[root@old-rac-03 ~]# lspci | grep -i infiniband
07:00.0 InfiniBand: Mellanox Technologies MT23108 InfiniHost (rev a1)
[root@old-rac-03 ~]#
[root@old-rac-03 ~]# lspci -vv -s 07:00
07:00.0 InfiniBand: Mellanox Technologies MT23108 InfiniHost (rev a1)
Subsystem: Mellanox Technologies MT23108 InfiniHost
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 37
Region 0: Memory at ed800000 (64-bit, non-prefetchable) [size=1M]
Region 2: Memory at fe000000 (64-bit, prefetchable) [size=8M]
Region 4: Memory at f0000000 (64-bit, prefetchable) [size=128M]
Capabilities: [40] MSI-X: Enable- Count=32 Masked-
Vector table: BAR=0 offset=00082000
PBA: BAR=0 offset=00082200
Capabilities: [50] Vital Product Data
Product Name: Cougar cub
Read-only fields:
[PN] Part number: 99-00025-01
[EC] Engineering changes: Rev: C04
[SN] Serial number: xxxxxxxxxxxxxxx
[V0] Vendor specific: PW=12W;PCI 66MHZ;PCI-X 133MHZ
[V2] Vendor specific: 0449
[RV] Reserved: checksum good, 0 byte(s) reserved
Read/write fields:
[V1] Vendor specific: N/A
[YA] Asset tag: N/A
[RW] Read-write area: 94 byte(s) free
End
Capabilities: [60] MSI: Enable- Count=1/32 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [70] PCI-X non-bridge device
Command: DPERE- ERO- RBC=512 OST=2
Status: Dev=07:00.0 64bit+ 133MHz+ SCD- USC- DC=simple DMMRBC=4096 DMOST=2 DMCRS=8 RSCEM- 266MHz- 533MHz-
Kernel driver in use: ib_mthca
Kernel modules: ib_mthca
[root@old-rac-03 ~]#
[root@old-rac-03 ~]# ibstat
CA 'mthca0'
CA type: MT23108
Number of ports: 2
Firmware version: 3.5.917
Hardware version: a1
Node GUID: 0x0005ad0000035a3c
System image GUID: 0x0005ad000100d050
Port 1:
State: Active
Physical state: LinkUp
Rate: 10
Base lid: 7
LMC: 0
SM lid: 2
Capability mask: 0x02510a68
Port GUID: 0x0005ad0000035a3d
Link layer: IB
Port 2:
State: Active
Physical state: LinkUp
Rate: 10
Base lid: 4
LMC: 0
SM lid: 2
Capability mask: 0x02510a68
Port GUID: 0x0005ad0000035a3e
Link layer: IB
[root@old-rac-03 ~]#
Last week I dusted off some older Sunfire servers for lab work. Installed OL7, resulting in no NIC devices for configuring the network. Installed OL6, and all 4 NVidia Ethernet ports were detected, and available. Server details:
[root@lab2 ~]# dmidecode -t1
# dmidecode 2.12
SMBIOS 2.5 present.
Handle 0x0001, DMI type 1, 27 bytes
System Information
Manufacturer: Sun Microsystems
Product Name: Sun Fire X4140
Version:
Serial Number: xxxxxxxxxxxxx
UUID: 080020FF-FFFF-FFFF-FFFF-44F8ED4F1400
Wake-up Type: Power Switch
SKU Number: 4151844-1
Family: Sun Fire X86 Server
[root@lab2 ~]#
[root@lab2 ~]# lspci | grep -i ethernet
00:08.0 Bridge: NVIDIA Corporation MCP55 Ethernet (rev a3)
00:09.0 Bridge: NVIDIA Corporation MCP55 Ethernet (rev a3)
80:08.0 Bridge: NVIDIA Corporation MCP55 Ethernet (rev a3)
80:09.0 Bridge: NVIDIA Corporation MCP55 Ethernet (rev a3)
[root@lab2 ~]#
[root@lab2 ~]# lspci -v -s *:0:08
00:08.0 Bridge: NVIDIA Corporation MCP55 Ethernet (rev a3)
Subsystem: NVIDIA Corporation Device cb84
Flags: bus master, 66MHz, fast devsel, latency 0, IRQ 37
Memory at defba000 (32-bit, non-prefetchable) [size=4K]
I/O ports at 8400 [size=8]
Memory at defbe800 (32-bit, non-prefetchable) [size=256]
Memory at defbe400 (32-bit, non-prefetchable) [size=16]
Capabilities: [44] Power Management version 2
Capabilities: [70] MSI-X: Enable- Count=8 Masked-
Capabilities: [50] MSI: Enable+ Count=1/8 Maskable+ 64bit+
Capabilities: [6c] HyperTransport: MSI Mapping Enable- Fixed+
Kernel driver in use: forcedeth
[root@lab2 ~]#
[root@lab2 ~]# ethtool --driver eth0
driver: forcedeth
version: 0.64
firmware-version:
bus-info: 0000:00:08.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
[root@lab2 ~]#
Would be great if later OL kernel versions can be used, supporting this older hardware.