EMC Networking Virtualization Overlay with BGP EVPN
Page 3
...1 Introduction...6 1.1 Typographical conventions...6 1.2 Attachments...6 2 Hardware Overview...7 2.1 Dell EMC PowerSwitch S5248F-ON ...7 2.2 Dell EMC PowerSwitch S4148U-ON ...7 2.3 Dell EMC PowerSwitch Z9264F-ON...8 2.4 Dell EMC PowerSwitch S3048-ON...8 2.5 Dell EMC PowerEdge R740xd server ...8 3 BGP EVPN VXLAN overview ...9 3.1 The ... OS version...19 5.2 Verify license installation ...19 5.3 Factory default configuration ...20 5.4 Configure switch port profile (S4148U-ON only 20 6 Configure leaf switches ...21 6.1 Initial configuration settings ...21 6.2 VLT configuration...22 6.3 Virtual...
...1 Introduction...6 1.1 Typographical conventions...6 1.2 Attachments...6 2 Hardware Overview...7 2.1 Dell EMC PowerSwitch S5248F-ON ...7 2.2 Dell EMC PowerSwitch S4148U-ON ...7 2.3 Dell EMC PowerSwitch Z9264F-ON...8 2.4 Dell EMC PowerSwitch S3048-ON...8 2.5 Dell EMC PowerEdge R740xd server ...8 3 BGP EVPN VXLAN overview ...9 3.1 The ... OS version...19 5.2 Verify license installation ...19 5.3 Factory default configuration ...20 5.4 Configure switch port profile (S4148U-ON only 20 6 Configure leaf switches ...21 6.1 Initial configuration settings ...21 6.2 VLT configuration...22 6.3 Virtual...
EMC Networking Virtualization Overlay with BGP EVPN
Page 7
... a 1-Rack Unit (RU), multilayer switch with BGP EVPN Leaf switches used to validate the deployment examples in this document. Dell EMC PowerSwitch S4148U-ON 7 Dell EMC Networking Virtualization Overlay with forty-eight SFP28 25GbE ports, two QSFP28-DD ports (two 100GbE interfaces per port), and four... for 25GbE and 10GbE compute and storage. This guide uses two S5248F-ONs as leaf switches. 2.2 Dell EMC PowerSwitch S5248F-ON Dell EMC PowerSwitch S4148U-ON The Dell EMC PowerSwitch S4148U-ON is used in and out of this writing, this guide. As of tunnels (RIOT). This guide...
... a 1-Rack Unit (RU), multilayer switch with BGP EVPN Leaf switches used to validate the deployment examples in this document. Dell EMC PowerSwitch S4148U-ON 7 Dell EMC Networking Virtualization Overlay with forty-eight SFP28 25GbE ports, two QSFP28-DD ports (two 100GbE interfaces per port), and four... for 25GbE and 10GbE compute and storage. This guide uses two S5248F-ONs as leaf switches. 2.2 Dell EMC PowerSwitch S5248F-ON Dell EMC PowerSwitch S4148U-ON The Dell EMC PowerSwitch S4148U-ON is used in and out of this writing, this guide. As of tunnels (RIOT). This guide...
EMC Networking Virtualization Overlay with BGP EVPN
Page 12
...Note: For this topology will scale to 32 racks. With Z9264F-ON switches as border leafs. For additional leaf-spine network information, refer to Dell EMC PowerSwitch Layer 3 Leaf-Spine Deployment and Best Practices with BGP EVPN BGP is used to load balance traffic across the Layer 3 connections. ...in the remaining two racks. Equal cost multi-path routing (ECMP) is an additional switch attached to be used as shown in Rack 1, and four S4148U-ON switches are used as leafs in Figure 8. The Layer 3/Layer 2 (L3/L2) boundary is considered a best practice. VLT allows all connections ...
...Note: For this topology will scale to 32 racks. With Z9264F-ON switches as border leafs. For additional leaf-spine network information, refer to Dell EMC PowerSwitch Layer 3 Leaf-Spine Deployment and Best Practices with BGP EVPN BGP is used to load balance traffic across the Layer 3 connections. ...in the remaining two racks. Equal cost multi-path routing (ECMP) is an additional switch attached to be used as shown in Rack 1, and four S4148U-ON switches are used as leafs in Figure 8. The Layer 3/Layer 2 (L3/L2) boundary is considered a best practice. VLT allows all connections ...
EMC Networking Virtualization Overlay with BGP EVPN
Page 15
Connections from hosts to each spine. 100GbE uplink ports on S4148U-ONs also used to S4148U-ON leaf switches are 10GbE (not shown). This requires two QSFP28-DD DAC cables. For this guide. 4.2 Underlay network connections The physical underlay network connections ... has one connection to S5248F-ON leaf switches are used as a VLT port channel on S5248F-ON switches, may be used for the VLTi. 15 Dell EMC Networking Virtualization Overlay with BGP EVPN Z9264F-ON Spine 1 Z9264F-ON Spine 2 Stack ID Reset Stack ID Reset S5248F-ON Stack ID Leaf 1A...
Connections from hosts to each spine. 100GbE uplink ports on S4148U-ONs also used to S4148U-ON leaf switches are 10GbE (not shown). This requires two QSFP28-DD DAC cables. For this guide. 4.2 Underlay network connections The physical underlay network connections ... has one connection to S5248F-ON leaf switches are used as a VLT port channel on S5248F-ON switches, may be used for the VLTi. 15 Dell EMC Networking Virtualization Overlay with BGP EVPN Z9264F-ON Spine 1 Z9264F-ON Spine 2 Stack ID Reset Stack ID Reset S5248F-ON Stack ID Leaf 1A...
EMC Networking Virtualization Overlay with BGP EVPN
Page 16
....222.1 S5248F-1a VLTi S5248F-1b 172.16.14.253 17V2N.1I6.1165.12453 VLT VM R740xd-1 VM VM R740xd-2 VM VTEP 10.222.222.2 S4148U-2a VLTi S4148U-2b 172.16.14.253 172.16.15.253 VLT VM R740xd-3 VM VM R740xd-4 VM eBGP ECMP eBGP VTEP 10.222.222..., VMs on all on VNI 1614, for routing to external networks, such as their local leaf switches will always be in the range 1-16777215. 16 Dell EMC Networking Virtualization Overlay with anycast gateways and an indirect gateway Rack n Physical L3 connection Physical L2 connection Virtual L2 connection Virtual L2 connection Virtual...
....222.1 S5248F-1a VLTi S5248F-1b 172.16.14.253 17V2N.1I6.1165.12453 VLT VM R740xd-1 VM VM R740xd-2 VM VTEP 10.222.222.2 S4148U-2a VLTi S4148U-2b 172.16.14.253 172.16.15.253 VLT VM R740xd-3 VM VM R740xd-4 VM eBGP ECMP eBGP VTEP 10.222.222..., VMs on all on VNI 1614, for routing to external networks, such as their local leaf switches will always be in the range 1-16777215. 16 Dell EMC Networking Virtualization Overlay with anycast gateways and an indirect gateway Rack n Physical L3 connection Physical L2 connection Virtual L2 connection Virtual L2 connection Virtual...
EMC Networking Virtualization Overlay with BGP EVPN
Page 20
... of OS10 Enterprise Edition User Guide Release 10.4.3.0 for the VLTi) on each S4148U-ON switch. Switch port profile-3 is specific to delete startup-configuration [confirm yes/no ]:y 20 Dell EMC Networking Virtualization Overlay with the leaf switches at its factory default configuration. All... OS10# write memory OS10# reload Proceed to a complex password during the first login. Dell EMC PowerSwitches running OS10EE can be wiped out after a save and reload. Configure switch port profile (S4148U-ON only) Note: This section is used as leafs in this deployment example.
... of OS10 Enterprise Edition User Guide Release 10.4.3.0 for the VLTi) on each S4148U-ON switch. Switch port profile-3 is specific to delete startup-configuration [confirm yes/no ]:y 20 Dell EMC Networking Virtualization Overlay with the leaf switches at its factory default configuration. All... OS10# write memory OS10# reload Proceed to a complex password during the first login. Dell EMC PowerSwitches running OS10EE can be wiped out after a save and reload. Configure switch port profile (S4148U-ON only) Note: This section is used as leafs in this deployment example.
EMC Networking Virtualization Overlay with BGP EVPN
Page 21
...scaled-overlay-routing bfd enable ip vrf tenant1 hardware overlay-routing-profile scaled-overlay-routing bfd enable ip vrf tenant1 21 Dell EMC Networking Virtualization Overlay with the configure terminal command. 2. Section 1.2 describes how to scaled-overlay-routing. Configuration differences... for more switch hardware resources to this .pdf. After setting to factory defaults, S4148U-ON switches are set the hardware overlay routing profile to access .pdf attachments. This example uses RSTP. See the L3 VXLAN...
...scaled-overlay-routing bfd enable ip vrf tenant1 hardware overlay-routing-profile scaled-overlay-routing bfd enable ip vrf tenant1 21 Dell EMC Networking Virtualization Overlay with the configure terminal command. 2. Section 1.2 describes how to scaled-overlay-routing. Configuration differences... for more switch hardware resources to this .pdf. After setting to factory defaults, S4148U-ON switches are set the hardware overlay routing profile to access .pdf attachments. This example uses RSTP. See the L3 VXLAN...
EMC Networking Virtualization Overlay with BGP EVPN
Page 26
...the IP prefix list items named spine-leaf. 3. Configure a route map named spine-leaf. 2. This range includes all VTEP IP addresses. 26 Dell EMC Networking Virtualization Overlay with a mask greater than or equal to 32. 6.6 6.7 Upstream interface configuration Each upstream (spine-connected) interface is ...by running the no shutdown Note: The remaining leaf switches are configured in the 10.222.222.0/24 address range with BGP EVPN S4148U-ON leaf switches use different ports, ethernet 1/1/25-1/1/26, for the 100GbE connections to redistribute all leaf router IDs. Route map ...
...the IP prefix list items named spine-leaf. 3. Configure a route map named spine-leaf. 2. This range includes all VTEP IP addresses. 26 Dell EMC Networking Virtualization Overlay with a mask greater than or equal to 32. 6.6 6.7 Upstream interface configuration Each upstream (spine-connected) interface is ...by running the no shutdown Note: The remaining leaf switches are configured in the 10.222.222.0/24 address range with BGP EVPN S4148U-ON leaf switches use different ports, ethernet 1/1/25-1/1/26, for the 100GbE connections to redistribute all leaf router IDs. Route map ...
EMC Networking Virtualization Overlay with BGP EVPN
Page 27
... same manner. To reduce traffic loss when a VLT peer boots up as downstream ports. 1. For example, if all ports marked as upstream ports go down. S4148U-ON leaf switches use different downstream ports, ethernet 1/1/251/1/26, for Leafs 3a and 3b. Create an uplink state group. 2. Enable the uplink state group... the heartbeat is still up and joins an existing VLT domain, or if all traffic to those shown above in response to the spines. 27 Dell EMC Networking Virtualization Overlay with identical settings to the VLT primary peer until at least one VLTi link is lost.
... same manner. To reduce traffic loss when a VLT peer boots up as downstream ports. 1. For example, if all ports marked as upstream ports go down. S4148U-ON leaf switches use different downstream ports, ethernet 1/1/251/1/26, for Leafs 3a and 3b. Create an uplink state group. 2. Enable the uplink state group... the heartbeat is still up and joins an existing VLT domain, or if all traffic to those shown above in response to the spines. 27 Dell EMC Networking Virtualization Overlay with identical settings to the VLT primary peer until at least one VLTi link is lost.
EMC Networking Virtualization Overlay with BGP EVPN
Page 58
... connection Virtual L2 connection 11.1 Validate tunneled Layer 2 bridging This test is shown in Figure 23. 58 Dell EMC Networking Virtualization Overlay with each other and the external network. VNI 1616 VM VM on VNI 1614,...1165.12453 VLT VM-1614-1 R740xd-1 VM-1616-1 R740xd-2 VTEP 10.222.222.2 S4148U-2a VLTi S4148U-2b 172.16.14.253 172.16.15.253 VLT VM-1614-3 R740xd-3 VM-1615-3 R740xd-4 ...VTEP 10.222.222.3 S4148U-3a VLTi S4148U-3b Border leafs VLT 172.16.16.253 Gateway/ Firewall Loopback0 1.1.1.1/32 Rack 1 Rack 2 172...
... connection Virtual L2 connection 11.1 Validate tunneled Layer 2 bridging This test is shown in Figure 23. 58 Dell EMC Networking Virtualization Overlay with each other and the external network. VNI 1616 VM VM on VNI 1614,...1165.12453 VLT VM-1614-1 R740xd-1 VM-1616-1 R740xd-2 VTEP 10.222.222.2 S4148U-2a VLTi S4148U-2b 172.16.14.253 172.16.15.253 VLT VM-1614-3 R740xd-3 VM-1615-3 R740xd-4 ...VTEP 10.222.222.3 S4148U-3a VLTi S4148U-3b Border leafs VLT 172.16.16.253 Gateway/ Firewall Loopback0 1.1.1.1/32 Rack 1 Rack 2 172...
EMC Networking Virtualization Overlay with BGP EVPN
Page 64
...version A04 VMware vCenter Server Appliance 6.7 Update 1 - iDRAC with BGP EVPN build 10244845 (Includes vSphere Web Client) 64 Dell EMC Networking Virtualization Overlay with Lifecycle Controller Firmware 16.17.00.03 14.07.50 18.8.9 1.6.12 3.21.26.22 ... 1 Intel X710 2x10GbE SFP+/i350 2x1GbE BASE-T rNDC - Dell EMC PowerSwitches Switches and OS versions Qty Item 2 Dell EMC PowerSwitch S5248F-ON leaf switch 4 Dell EMC PowerSwitch S4148U-ON leaf switch 2 Dell EMC PowerSwitch Z9264-ON spine switch 1 Dell EMC PowerSwitch S3048-ON management switch OS Version 10.4.3.1.154 10...
...version A04 VMware vCenter Server Appliance 6.7 Update 1 - iDRAC with BGP EVPN build 10244845 (Includes vSphere Web Client) 64 Dell EMC Networking Virtualization Overlay with Lifecycle Controller Firmware 16.17.00.03 14.07.50 18.8.9 1.6.12 3.21.26.22 ... 1 Intel X710 2x10GbE SFP+/i350 2x1GbE BASE-T rNDC - Dell EMC PowerSwitches Switches and OS versions Qty Item 2 Dell EMC PowerSwitch S5248F-ON leaf switch 4 Dell EMC PowerSwitch S4148U-ON leaf switch 2 Dell EMC PowerSwitch Z9264-ON spine switch 1 Dell EMC PowerSwitch S3048-ON management switch OS Version 10.4.3.1.154 10...
EMC Networking Virtualization Overlay with BGP EVPN
Page 65
... Deployment and Best Practices with OS10 Manuals and documentation for Dell EMC PowerSwitch S3048-ON Manuals and documentation for Dell EMC PowerSwitch S4148U-ON Manuals and documentation for Dell EMC PowerSwitch S5248F-ON Manuals and documentation for Dell EMC PowerSwitch Z9264F-ON Manuals and documentation for Dell EMC PowerEdge R740xd VMware product manuals and technical guides...
... Deployment and Best Practices with OS10 Manuals and documentation for Dell EMC PowerSwitch S3048-ON Manuals and documentation for Dell EMC PowerSwitch S4148U-ON Manuals and documentation for Dell EMC PowerSwitch S5248F-ON Manuals and documentation for Dell EMC PowerSwitch Z9264F-ON Manuals and documentation for Dell EMC PowerEdge R740xd VMware product manuals and technical guides...
EMC Networking ONIE Quick Start Guide
Page 4
... all operations that are performed can also be done on all data center switches that end in this effort by using the specified Dell EMC Networking switches and operating systems. Also, the steps can be leveraged for bare metal networking switches. This guide covers the menu... within ONIE and the operations that contains a "-ON" suffix, such as the Dell EMC Networking Z9100-ON and the Dell EMC Networking S4148U-ON, have ONIE enabled. The Dell EMC Networking S4148U-ON and the Dell EMC Networking S4048-ON are available within ONIE. See https://www.opencomputerproject.org/ for...
... all operations that are performed can also be done on all data center switches that end in this effort by using the specified Dell EMC Networking switches and operating systems. Also, the steps can be leveraged for bare metal networking switches. This guide covers the menu... within ONIE and the operations that contains a "-ON" suffix, such as the Dell EMC Networking Z9100-ON and the Dell EMC Networking S4148U-ON, have ONIE enabled. The Dell EMC Networking S4148U-ON and the Dell EMC Networking S4048-ON are available within ONIE. See https://www.opencomputerproject.org/ for...
EMC Networking ONIE Quick Start Guide
Page 12
...Version Programming passed. TlvInfo Header: Id String: TlvInfo Version: 1 Total Length: 180 TLV Name Code Len Value Product Name 0x21 9 S4148U-ON Part Number 0x22 6 0943V5 Serial Number 0x23 20 CN0943V5282987BQ0005 Base MAC Address 0x24 6 E4:F0:04:6B:04:42 Manufacture Date ...0x26 1 1 Label Revision 0x27 3 A00 MAC Addresses 0x2A 2 256 Manufacturer 0x2B 5 28298 Country Code 0x2C 2 CN Vendor Name 0x2D 8 Dell EMC Service Tag 0x2F 7 4S0RPK2 Vendor Extension 0xFD 4 0x00 0x00 0x02 0xA2 Platform Name 0x28 30 x86_64-dellemc_s4148u_c2338-r0 ONIE Version 0x29 10 3....
...Version Programming passed. TlvInfo Header: Id String: TlvInfo Version: 1 Total Length: 180 TLV Name Code Len Value Product Name 0x21 9 S4148U-ON Part Number 0x22 6 0943V5 Serial Number 0x23 20 CN0943V5282987BQ0005 Base MAC Address 0x24 6 E4:F0:04:6B:04:42 Manufacture Date ...0x26 1 1 Label Revision 0x27 3 A00 MAC Addresses 0x2A 2 256 Manufacturer 0x2B 5 28298 Country Code 0x2C 2 CN Vendor Name 0x2D 8 Dell EMC Service Tag 0x2F 7 4S0RPK2 Vendor Extension 0xFD 4 0x00 0x00 0x02 0xA2 Platform Name 0x28 30 x86_64-dellemc_s4148u_c2338-r0 ONIE Version 0x29 10 3....
EMC Networking ONIE Quick Start Guide
Page 21
... .deb (3.33.4.1-6) ... Compare with the following steps to the switch. Download the latest version of the CPLD image from support.dell.com and unzip. 2. If this process is unsuccessful, use the following command: root@dellemc-diag-os:~# cpldupgradetool --cpldver System CPLD Version... Run the following steps to upgrade the individual BIOS component using the steps within the ONIE: Rescue Firmware update section. Platform detected S4148U [12] Copying config dir contents Copying bin dir contents EDA-DIAG BIOS upgrade Upgrade the BIOS image using the smbiostool command: 1....
... .deb (3.33.4.1-6) ... Compare with the following steps to the switch. Download the latest version of the CPLD image from support.dell.com and unzip. 2. If this process is unsuccessful, use the following command: root@dellemc-diag-os:~# cpldupgradetool --cpldver System CPLD Version... Run the following steps to upgrade the individual BIOS component using the steps within the ONIE: Rescue Firmware update section. Platform detected S4148U [12] Copying config dir contents Copying bin dir contents EDA-DIAG BIOS upgrade Upgrade the BIOS image using the smbiostool command: 1....
EMC Networking ONIE Quick Start Guide
Page 29
Dell EMC networking switches Versions Qty Item OS/Firmware version 1 Dell EMC Networking S4148U-ON ONIE software DIAG OS DIAG tools Firmware 1 Dell EMC Networking S4048-ON ONIE software DIAG OS Firmware 3.33.1.1-4 3.33.3.0-1 3.33.4.1-6 3.33.5.1-19 3.21.5.3 1.0.0.14 3.21.5.3 29 Dell EMC Networking ONIE Quick Start Guide D D.1 Validated hardware and software versions The following tables list the hardware and components used to configure and validate the example configurations in this guide.
Dell EMC networking switches Versions Qty Item OS/Firmware version 1 Dell EMC Networking S4148U-ON ONIE software DIAG OS DIAG tools Firmware 1 Dell EMC Networking S4048-ON ONIE software DIAG OS Firmware 3.33.1.1-4 3.33.3.0-1 3.33.4.1-6 3.33.5.1-19 3.21.5.3 1.0.0.14 3.21.5.3 29 Dell EMC Networking ONIE Quick Start Guide D D.1 Validated hardware and software versions The following tables list the hardware and components used to configure and validate the example configurations in this guide.
EMC Networking ONIE Quick Start Guide
Page 34
.../19/drive rs/driversdetails?driverId Onie-firmware =FT8DP BIOS CPLD Dell EMC Networking S4148U-ON Open Networking Hardware Diagnostic Guide ONIE Software Diagnostics OS Diagnostic Tools Firmware updater https://downloads.dell.co .pdf m/manuals/allproducts/esuprt_ser_stor _net/esuprt_networking/ networking-s4148uon_specifications_enus.pdf https://www.dell.com/sup .zip port/home/us/en/19/drive rs...
.../19/drive rs/driversdetails?driverId Onie-firmware =FT8DP BIOS CPLD Dell EMC Networking S4148U-ON Open Networking Hardware Diagnostic Guide ONIE Software Diagnostics OS Diagnostic Tools Firmware updater https://downloads.dell.co .pdf m/manuals/allproducts/esuprt_ser_stor _net/esuprt_networking/ networking-s4148uon_specifications_enus.pdf https://www.dell.com/sup .zip port/home/us/en/19/drive rs...
EMC Networking OS10 Enterprise Edition Storage Overview
Page 4
... ...16 7.2 Layer 3 leaf-spine network ...17 8 Fibre Channel storage deployment options ...18 8.1 Fibre Channel storage with S4148U-ON in F_port mode 18 8.2 FCoE with S4148U-ON in NPG mode...19 8.3 FCoE to Fibre Channel with S4148U-ON in F_Port mode 19 8.4 FCoE with FIP snooping bridge...20 9 Ethernet-based storage...22 9.1 iSCSI SAN...-spine data center with iSCSI or SDS 23 9.2.1 Leaf-spine deployment ...23 9.2.2 vSAN deployment with SDS example ...23 9.2.3 iSCSI configuration ...24 A Support and feedback ...25 4 Dell EMC Networking OS10 Enterprise Edition Storage Overview
... ...16 7.2 Layer 3 leaf-spine network ...17 8 Fibre Channel storage deployment options ...18 8.1 Fibre Channel storage with S4148U-ON in F_port mode 18 8.2 FCoE with S4148U-ON in NPG mode...19 8.3 FCoE to Fibre Channel with S4148U-ON in F_Port mode 19 8.4 FCoE with FIP snooping bridge...20 9 Ethernet-based storage...22 9.1 iSCSI SAN...-spine data center with iSCSI or SDS 23 9.2.1 Leaf-spine deployment ...23 9.2.2 vSAN deployment with SDS example ...23 9.2.3 iSCSI configuration ...24 A Support and feedback ...25 4 Dell EMC Networking OS10 Enterprise Edition Storage Overview
EMC Networking OS10 Enterprise Edition Storage Overview
Page 18
... To obtain the detailed deployment guide with step-by-step configuration of the S4148U-ON switches, see Dell EMC Networking Fibre Channel Deployment with a high performing, stable solution. Dell EMC Networking provides several options to FC HBAs in F_port Mode 18 Dell EMC Networking OS10 Enterprise Edition Storage Overview The advantages of Fibre Channel...
... To obtain the detailed deployment guide with step-by-step configuration of the S4148U-ON switches, see Dell EMC Networking Fibre Channel Deployment with a high performing, stable solution. Dell EMC Networking provides several options to FC HBAs in F_port Mode 18 Dell EMC Networking OS10 Enterprise Edition Storage Overview The advantages of Fibre Channel...
EMC Networking OS10 Enterprise Edition Storage Overview
Page 19
...The solution in this section uses a Dell EMC PowerSwitch S4148U-ON switch for both Ethernet and FC traffic. This solution provides a scalable design that takes advantage of the performance advantages of the S4148U-ON switches, see Dell EMC Networking FCoE Deployment with S4148U-ON in NPG Mode FCoE to a... traditional FC switch. The S4148U-ON acts as both a leaf and FCoE to FC bridge to Fibre Channel ...
...The solution in this section uses a Dell EMC PowerSwitch S4148U-ON switch for both Ethernet and FC traffic. This solution provides a scalable design that takes advantage of the performance advantages of the S4148U-ON switches, see Dell EMC Networking FCoE Deployment with S4148U-ON in NPG Mode FCoE to a... traditional FC switch. The S4148U-ON acts as both a leaf and FCoE to FC bridge to Fibre Channel ...