EMC SmartFabric OS10 Switch Configuration Guide for VxRail 4.7
Page 3
... contents Revisions...2 1 Introduction...5 1.1 Supported switches and operating systems ...6 1.2 Typographical conventions ...6 1.3 Attachments...6 2 Hardware overview...7 2.1 Dell EMC PowerSwitch S5248F-ON ...7 2.2 Dell EMC PowerSwitch S3048-ON...7 2.3 Dell EMC VxRail 14G nodes ...7 2.3.1 Dell EMC VxRail P Series node ...8 2.3.2 Dell EMC VxRail V Series node ...8 2.3.3 Dell EMC VxRail S Series node ...8 2.3.4 Dell EMC VxRail E Series node ...8 2.3.5 VxRail node network adapters...8 3 Topology options ...10 3.1 Dual switch ...10...
... contents Revisions...2 1 Introduction...5 1.1 Supported switches and operating systems ...6 1.2 Typographical conventions ...6 1.3 Attachments...6 2 Hardware overview...7 2.1 Dell EMC PowerSwitch S5248F-ON ...7 2.2 Dell EMC PowerSwitch S3048-ON...7 2.3 Dell EMC VxRail 14G nodes ...7 2.3.1 Dell EMC VxRail P Series node ...8 2.3.2 Dell EMC VxRail V Series node ...8 2.3.3 Dell EMC VxRail S Series node ...8 2.3.4 Dell EMC VxRail E Series node ...8 2.3.5 VxRail node network adapters...8 3 Topology options ...10 3.1 Dual switch ...10...
EMC SmartFabric OS10 Switch Configuration Guide for VxRail 4.7
Page 7
2 2.1 Hardware overview This section briefly describes the hardware used to validate this guide. Appendix A contains a complete listing of -band (OOB) management traffic. 2.3 Dell EMC PowerSwitch S3048-ON Dell EMC VxRail 14G nodes Current Dell EMC VxRail P, V, S, and E Series nodes are built on 14th generation (14G) PowerEdge servers. VxRail P, V, and S Series nodes use a 1-RU form factor...
2 2.1 Hardware overview This section briefly describes the hardware used to validate this guide. Appendix A contains a complete listing of -band (OOB) management traffic. 2.3 Dell EMC PowerSwitch S3048-ON Dell EMC VxRail 14G nodes Current Dell EMC VxRail P, V, S, and E Series nodes are built on 14th generation (14G) PowerEdge servers. VxRail P, V, and S Series nodes use a 1-RU form factor...
EMC SmartFabric OS10 Switch Configuration Guide for VxRail 4.7
Page 14
... as VLT peers. Note: The OOB Management network is an isolated network for VxRail 4.7 It is installed in each S3048-ON for use the 100.67.0.0/16 IP address block. 14 Dell EMC SmartFabric OS10 Switch Configuration Guide for remote management of -band (OOB) Management network is not to hardware management ports...
... as VLT peers. Note: The OOB Management network is an isolated network for VxRail 4.7 It is installed in each S3048-ON for use the 100.67.0.0/16 IP address block. 14 Dell EMC SmartFabric OS10 Switch Configuration Guide for remote management of -band (OOB) Management network is not to hardware management ports...
EMC SmartFabric OS10 Switch Configuration Guide for VxRail 4.7
Page 44
... SmartFabric OS10 Switch Configuration Guide for S5248F-ON using the information provided below. Dell EMC PowerSwitches Switches and operating system versions Qty Item 2 Dell EMC PowerSwitch S5248F-ON leaf switches 1 Dell EMC PowerSwitch S3048-ON management switch OS version 10.5.0.0 10.5.0.0 DAC Cables DAC cables Qty Item 8 DAC-SFP28-25G-3.0M (Node connections) 2 DAC...
... SmartFabric OS10 Switch Configuration Guide for S5248F-ON using the information provided below. Dell EMC PowerSwitches Switches and operating system versions Qty Item 2 Dell EMC PowerSwitch S5248F-ON leaf switches 1 Dell EMC PowerSwitch S3048-ON management switch OS version 10.5.0.0 10.5.0.0 DAC Cables DAC cables Qty Item 8 DAC-SFP28-25G-3.0M (Node connections) 2 DAC...
EMC SmartFabric OS10 Switch Configuration Guide for VxRail 4.7
Page 48
... 10.5.0 Manuals and documentation for Dell EMC PowerSwitch S3048-ON Manuals and documentation for Dell EMC PowerSwitch S5248F-ON Dell EMC Networking Layer 3 Leaf-Spine Deployment and Best Practices with OS10 Dell EMC VxRail Network Planning Guide Dell EMC VxRail support and documentation (account required) Dell EMC VxRail Support Matrix (account required) Dell EMC VxRail Appliance 4.5.x and 4.7.x Event...
... 10.5.0 Manuals and documentation for Dell EMC PowerSwitch S3048-ON Manuals and documentation for Dell EMC PowerSwitch S5248F-ON Dell EMC Networking Layer 3 Leaf-Spine Deployment and Best Practices with OS10 Dell EMC VxRail Network Planning Guide Dell EMC VxRail support and documentation (account required) Dell EMC VxRail Support Matrix (account required) Dell EMC VxRail Appliance 4.5.x and 4.7.x Event...
VCF on VxRail Multirack Deployment using BGP EVPN
Page 3
... Center...9 1.4 Prerequisites...10 1.5 Supported switches and operating systems...10 1.6 Typographical conventions ...10 1.7 Attachments...10 2 Hardware overview...11 2.1 Dell EMC VxRail E560...11 2.2 Dell EMC PowerSwitch S5248F-ON ...11 2.3 Dell EMC PowerSwitch Z9264F-ON...11 2.4 Dell EMC PowerSwitch S3048-ON...12 3 Network transport ...13 3.1 Layer 3 leaf and spine topology...13 3.2 BGP EVPN VXLAN overview ...13 3.2.1 The...
... Center...9 1.4 Prerequisites...10 1.5 Supported switches and operating systems...10 1.6 Typographical conventions ...10 1.7 Attachments...10 2 Hardware overview...11 2.1 Dell EMC VxRail E560...11 2.2 Dell EMC PowerSwitch S5248F-ON ...11 2.3 Dell EMC PowerSwitch Z9264F-ON...11 2.4 Dell EMC PowerSwitch S3048-ON...12 3 Network transport ...13 3.1 Layer 3 leaf and spine topology...13 3.2 BGP EVPN VXLAN overview ...13 3.2.1 The...
VCF on VxRail Multirack Deployment using BGP EVPN
Page 12
In this document, one S3048-ON supports out-of-band (OOB) management traffic for all examples. 2.4 Dell EMC PowerSwitch S3048-ON The Dell EMC PowerSwitch S3048-ON is a 1-Rack Unit (RU) switch with forty-eight 1GbE BASE-T ports and four 10GbE SFP+ ports. Dell EMC PowerSwitch S3048-ON 12 VCF on VxRail Multirack Deployment using BGP EVPN
In this document, one S3048-ON supports out-of-band (OOB) management traffic for all examples. 2.4 Dell EMC PowerSwitch S3048-ON The Dell EMC PowerSwitch S3048-ON is a 1-Rack Unit (RU) switch with forty-eight 1GbE BASE-T ports and four 10GbE SFP+ ports. Dell EMC PowerSwitch S3048-ON 12 VCF on VxRail Multirack Deployment using BGP EVPN
VCF on VxRail Multirack Deployment using BGP EVPN
Page 21
...B GRN=10G ACT/ LNK A GRN=10G ACT/ LNK B GRN=10G ACT/ LNK A GRN=10G ACT/ LNK B VxRail E node sfo01w02vxrail01 VxRail E node sfo01w02vxrail03 Rack 1 Dell EMC VxRail multirack Rack 1 physical connectivity 21 VCF on VxRail Multirack Deployment using two QSFP28-DD 200 GbE direct-access cables (DAC) forming a VLT interconnect... which can be set up with varying levels of Rack 1. The 2x 25 GbE PCIe, shown here in slot 2, is dedicated to an S3048-ON OOB management switch. Resiliency is used for a total throughput of 400 GbE. The S5248F-ON leaf switches are connected using BGP EVPN Each...
...B GRN=10G ACT/ LNK A GRN=10G ACT/ LNK B GRN=10G ACT/ LNK A GRN=10G ACT/ LNK B VxRail E node sfo01w02vxrail01 VxRail E node sfo01w02vxrail03 Rack 1 Dell EMC VxRail multirack Rack 1 physical connectivity 21 VCF on VxRail Multirack Deployment using two QSFP28-DD 200 GbE direct-access cables (DAC) forming a VLT interconnect... which can be set up with varying levels of Rack 1. The 2x 25 GbE PCIe, shown here in slot 2, is dedicated to an S3048-ON OOB management switch. Resiliency is used for a total throughput of 400 GbE. The S5248F-ON leaf switches are connected using BGP EVPN Each...
VCF on VxRail Multirack Deployment using BGP EVPN
Page 59
...CPU @ 3.00GHz, 12 cores - 12 16GB DDR4 DIMMs (192GB total) - 2 800GB SAS SSD - 8 1.2TB SAS HDD EF03 1 Dell HBA330 Storage Controller 15.17.09.06 1 Boot Optimized Storage Solution (BOSS) 2.5.13.3016 Controller w/ 2x240GB SATA SSDs 1 Broadcom 57414 NDC ... using BGP EVPN BIOS 2.1.8 - A Validated components A.1 Dell EMC PowerSwitch models Switches and operating system versions Qty Item 4 Dell EMC PowerSwitch S5248F-ON leaf switches 2 Dell EMC PowerSwitch Z9264F-ON spine switches 2 Dell EMC PowerSwitch S3048-ON OOB mgmt switches Version 10.4.3.5 10.4.3.5 10.4.3.5 A.2...
...CPU @ 3.00GHz, 12 cores - 12 16GB DDR4 DIMMs (192GB total) - 2 800GB SAS SSD - 8 1.2TB SAS HDD EF03 1 Dell HBA330 Storage Controller 15.17.09.06 1 Boot Optimized Storage Solution (BOSS) 2.5.13.3016 Controller w/ 2x240GB SATA SSDs 1 Broadcom 57414 NDC ... using BGP EVPN BIOS 2.1.8 - A Validated components A.1 Dell EMC PowerSwitch models Switches and operating system versions Qty Item 4 Dell EMC PowerSwitch S5248F-ON leaf switches 2 Dell EMC PowerSwitch Z9264F-ON spine switches 2 Dell EMC PowerSwitch S3048-ON OOB mgmt switches Version 10.4.3.5 10.4.3.5 10.4.3.5 A.2...
EMC Solutions for Microsoft Azure Stack HCI Networking Guide
Page 3
Table of contents Revisions...2 1 Introduction...6 1.1 Typographical conventions...6 1.2 Attachments...6 2 Hardware Overview...7 2.1 Dell EMC Networking S5248F-ON ...7 2.2 Dell EMC Networking S3048-ON ...7 2.3 Dell EMC S2D Ready Nodes...7 2.4 Management system ...8 2.5 Supported Dell EMC Networking switches...8 3 Topology...9 3.1 Connections to leaf switches ...10 3.2 Connections to OOB management switch ...11 4 ... VLANs and VRRP...26 7.4 Configure storage interfaces ...27 7.5 Configure in-band management interfaces ...28 3 Dell EMC Solutions for Microsoft Azure Stack HCI Networking Guide
Table of contents Revisions...2 1 Introduction...6 1.1 Typographical conventions...6 1.2 Attachments...6 2 Hardware Overview...7 2.1 Dell EMC Networking S5248F-ON ...7 2.2 Dell EMC Networking S3048-ON ...7 2.3 Dell EMC S2D Ready Nodes...7 2.4 Management system ...8 2.5 Supported Dell EMC Networking switches...8 3 Topology...9 3.1 Connections to leaf switches ...10 3.2 Connections to OOB management switch ...11 4 ... VLANs and VRRP...26 7.4 Configure storage interfaces ...27 7.5 Configure in-band management interfaces ...28 3 Dell EMC Solutions for Microsoft Azure Stack HCI Networking Guide
EMC Solutions for Microsoft Azure Stack HCI Networking Guide
Page 7
...R740xd, R640, and R440 servers and are available in Hybrid (mix of -band (OOB) management traffic. 2.3 Dell EMC Networking S3048-ON Dell EMC S2D Ready Nodes Dell EMC Microsoft Storage Spaces Direct Ready Nodes are used to validate the deployment examples in -band traffic. 2 2.1 ...leaf switches for in this document. This guide uses two S5248F-ONs as leaf switches for S2D. 2.2 Dell EMC Networking S5248F-ON Dell EMC Networking S3048-ON The Dell EMC Networking S3048-ON is a 1-Rack Unit (RU), multilayer switch with certified components that simplify ordering and reduce deployment risks...
...R740xd, R640, and R440 servers and are available in Hybrid (mix of -band (OOB) management traffic. 2.3 Dell EMC Networking S3048-ON Dell EMC S2D Ready Nodes Dell EMC Microsoft Storage Spaces Direct Ready Nodes are used to validate the deployment examples in -band traffic. 2 2.1 ...leaf switches for in this document. This guide uses two S5248F-ONs as leaf switches for S2D. 2.2 Dell EMC Networking S5248F-ON Dell EMC Networking S3048-ON The Dell EMC Networking S3048-ON is a 1-Rack Unit (RU), multilayer switch with certified components that simplify ordering and reduce deployment risks...
EMC Solutions for Microsoft Azure Stack HCI Networking Guide
Page 8
...part of -band management traffic. It also serves as the S2D management system. For switch configuration examples using OS9, see Support Matrix for Dell EMC Solutions for out-of the S2D cluster and does not contribute storage to deploy and manage the S2D cluster. 2.4 Management system A ...management system is used to it has network connectivity to OS10EE. One Dell EMC Networking S3048-ON switch is used as the Active Directory (AD) domain controller and DNS server for Microsoft Azure Stack HCI Networking Guide In this...
...part of -band management traffic. It also serves as the S2D management system. For switch configuration examples using OS9, see Support Matrix for Dell EMC Solutions for out-of the S2D cluster and does not contribute storage to deploy and manage the S2D cluster. 2.4 Management system A ...management system is used to it has network connectivity to OS10EE. One Dell EMC Networking S3048-ON switch is used as the Active Directory (AD) domain controller and DNS server for Microsoft Azure Stack HCI Networking Guide In this...
EMC Solutions for Microsoft Azure Stack HCI Networking Guide
Page 9
... are limited to 16 nodes. 3 Topology S2D clusters are cabled as shown in a single rack along with two leaf switches and one OOB management switch. S3048-ON OOB Mgmt S5248F-ON Leaf 1A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27... Node 4 25GbE Storage traffic 10GbE In-band mgmt and VM traffic 2x100GbE VLTi (QSFP28-DD DAC) 1GbE OOB mgmt All cabling for the S2D environment 9 Dell EMC Solutions for Microsoft Azure Stack HCI Networking Guide
... are limited to 16 nodes. 3 Topology S2D clusters are cabled as shown in a single rack along with two leaf switches and one OOB management switch. S3048-ON OOB Mgmt S5248F-ON Leaf 1A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27... Node 4 25GbE Storage traffic 10GbE In-band mgmt and VM traffic 2x100GbE VLTi (QSFP28-DD DAC) 1GbE OOB mgmt All cabling for the S2D environment 9 Dell EMC Solutions for Microsoft Azure Stack HCI Networking Guide
EMC Solutions for Microsoft Azure Stack HCI Networking Guide
Page 11
... at least one of servers, switches, and other devices. Note: When running OS10EE, the S3048-ON will function as shown in Figure 7. At a minimum, Dell EMC recommends changing the admin password to a complex password during the first login. S3048-ON OOB Management 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ... has forty-eight 1GbE Base-T ports for iDRAC connections must be connected to available ports on port-groups. 11 Dell EMC Solutions for more information on either one S3048-ON switch per -VLAN spanning tree plus (RPVST+) is enabled. The S5248F-ON ports used as VLT peers. Use...
... at least one of servers, switches, and other devices. Note: When running OS10EE, the S3048-ON will function as shown in Figure 7. At a minimum, Dell EMC recommends changing the admin password to a complex password during the first login. S3048-ON OOB Management 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ... has forty-eight 1GbE Base-T ports for iDRAC connections must be connected to available ports on port-groups. 11 Dell EMC Solutions for more information on either one S3048-ON switch per -VLAN spanning tree plus (RPVST+) is enabled. The S5248F-ON ports used as VLT peers. Use...
EMC Solutions for Microsoft Azure Stack HCI Networking Guide
Page 34
...51 S5248F-Leaf1B ethernet1/1/51 3c:2c:30:10:34:00 ethernet1/1/52 S5248F-Leaf1B ethernet1/1/52 3c:2c:30:10:34:00 mgmt1/1/1 R168U39-S3048 ethernet1/1/38 74:e6:e2:f5:c8:80 8.1.4 show vrrp brief command shows the master and virtual IP addresses and whether the switch is ... Group Priority Prempt State Version Master addr(s) Virtual addr vlan1611 IPv4 11 150 true master 2 172.16.11.252 172.16.11.254 34 Dell EMC Solutions for management and VM traffic. Note: Mellanox ConnectX-4 Lx adapters set to provide gateway redundancy for Microsoft Azure Stack HCI Networking Guide ...
...51 S5248F-Leaf1B ethernet1/1/51 3c:2c:30:10:34:00 ethernet1/1/52 S5248F-Leaf1B ethernet1/1/52 3c:2c:30:10:34:00 mgmt1/1/1 R168U39-S3048 ethernet1/1/38 74:e6:e2:f5:c8:80 8.1.4 show vrrp brief command shows the master and virtual IP addresses and whether the switch is ... Group Priority Prempt State Version Master addr(s) Virtual addr vlan1611 IPv4 11 150 true master 2 172.16.11.252 172.16.11.254 34 Dell EMC Solutions for management and VM traffic. Note: Mellanox ConnectX-4 Lx adapters set to provide gateway redundancy for Microsoft Azure Stack HCI Networking Guide ...
EMC Solutions for Microsoft Azure Stack HCI Networking Guide
Page 56
... the examples in Table 15 for Windows Server 2016 and Table 16 for Microsoft Azure Stack HCI. Dell EMC Networking Switches Switches and OS versions Qty Item 2 Dell EMC Networking S5248F-ON leaf switch 1 Dell EMC Networking S3048-ON management switch OS Version 10.4.2.2 10.4.2.2 A.2 Note: See Table 1 for Microsoft Azure Stack HCI Networking...
... the examples in Table 15 for Windows Server 2016 and Table 16 for Microsoft Azure Stack HCI. Dell EMC Networking Switches Switches and OS versions Qty Item 2 Dell EMC Networking S5248F-ON leaf switch 1 Dell EMC Networking S3048-ON management switch OS Version 10.4.2.2 10.4.2.2 A.2 Note: See Table 1 for Microsoft Azure Stack HCI Networking...
EMC Solutions for Microsoft Azure Stack HCI Networking Guide
Page 61
... Release 10.4.2.0 Manuals and documentation for Dell EMC Networking S3048-ON Manuals and documentation for Dell EMC Networking S5248F-ON Dell EMC Solutions for Microsoft Azure Stack HCI page Dell EMC Solutions for Microsoft Azure Stack HCI Deployment Guide Support Matrix for Dell EMC Solutions for Microsoft Azure Stack HCI Dell EMC Networking Layer 3 Leaf-Spine Deployment...
... Release 10.4.2.0 Manuals and documentation for Dell EMC Networking S3048-ON Manuals and documentation for Dell EMC Networking S5248F-ON Dell EMC Solutions for Microsoft Azure Stack HCI page Dell EMC Solutions for Microsoft Azure Stack HCI Deployment Guide Support Matrix for Dell EMC Solutions for Microsoft Azure Stack HCI Dell EMC Networking Layer 3 Leaf-Spine Deployment...
EMC Networking Virtualization Overlay with BGP EVPN
Page 3
Table of contents Revisions...2 1 Introduction...6 1.1 Typographical conventions...6 1.2 Attachments...6 2 Hardware Overview...7 2.1 Dell EMC PowerSwitch S5248F-ON ...7 2.2 Dell EMC PowerSwitch S4148U-ON ...7 2.3 Dell EMC PowerSwitch Z9264F-ON...8 2.4 Dell EMC PowerSwitch S3048-ON...8 2.5 Dell EMC PowerEdge R740xd server ...8 3 BGP EVPN VXLAN overview ...9 3.1 The VXLAN protocol...10 3.2 BGP EVPN... configuration ...22 6.4 VLAN configuration...24 6.5 Downstream interface configuration ...25 6.6 Upstream interface configuration...26 3 Dell EMC Networking Virtualization Overlay with BGP EVPN
Table of contents Revisions...2 1 Introduction...6 1.1 Typographical conventions...6 1.2 Attachments...6 2 Hardware Overview...7 2.1 Dell EMC PowerSwitch S5248F-ON ...7 2.2 Dell EMC PowerSwitch S4148U-ON ...7 2.3 Dell EMC PowerSwitch Z9264F-ON...8 2.4 Dell EMC PowerSwitch S3048-ON...8 2.5 Dell EMC PowerEdge R740xd server ...8 3 BGP EVPN VXLAN overview ...9 3.1 The VXLAN protocol...10 3.2 BGP EVPN... configuration ...22 6.4 VLAN configuration...24 6.5 Downstream interface configuration ...25 6.6 Upstream interface configuration...26 3 Dell EMC Networking Virtualization Overlay with BGP EVPN
EMC Networking Virtualization Overlay with BGP EVPN
Page 8
This guide uses two Z9264F-ONs as spine switches. 2.4 Dell EMC PowerSwitch Z9264F-ON Dell EMC PowerSwitch S3048-ON The Dell EMC PowerSwitch S3048-ON is a 1-RU switch with up to 128 10/25/40/50GbE ports using supported breakout cables. The R740xd server ...The server adds extraordinary storage capacity options, making it well-suited for out-of-band (OOB) management traffic. 2.5 Dell EMC PowerSwitch S3048-ON Dell EMC PowerEdge R740xd server The Dell EMC PowerEdge R740xd server is used in each rack for dataintensive applications that brings scalable storage performance and data set ...
This guide uses two Z9264F-ONs as spine switches. 2.4 Dell EMC PowerSwitch Z9264F-ON Dell EMC PowerSwitch S3048-ON The Dell EMC PowerSwitch S3048-ON is a 1-RU switch with up to 128 10/25/40/50GbE ports using supported breakout cables. The R740xd server ...The server adds extraordinary storage capacity options, making it well-suited for out-of-band (OOB) management traffic. 2.5 Dell EMC PowerSwitch S3048-ON Dell EMC PowerEdge R740xd server The Dell EMC PowerEdge R740xd server is used in each rack for dataintensive applications that brings scalable storage performance and data set ...
EMC Networking Virtualization Overlay with BGP EVPN
Page 17
..., each server has an additional 1GbE OOB management connection for connections to carry heartbeat messages sent between leaf switches configured as shown in Figure 13. S3048-ON OOB management S5248F-ON Leaf 1A S5248F-ON Leaf 1B Z9264F-ON Spine 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16... Networking Virtualization Overlay with BGP EVPN It is an isolated network for OOB management network connections. Dell EMC recommends using at least one S3048-ON switch per 42-RU rack for remote management of servers, switches, and other devices. 4.4 OOB management network connections The OOB ...
..., each server has an additional 1GbE OOB management connection for connections to carry heartbeat messages sent between leaf switches configured as shown in Figure 13. S3048-ON OOB management S5248F-ON Leaf 1A S5248F-ON Leaf 1B Z9264F-ON Spine 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16... Networking Virtualization Overlay with BGP EVPN It is an isolated network for OOB management network connections. Dell EMC recommends using at least one S3048-ON switch per 42-RU rack for remote management of servers, switches, and other devices. 4.4 OOB management network connections The OOB ...