HP InfiniBand Solution for Oracle RAC Environments white paper
Page 9
... interface using the following command: ifup ib0 9 In a redundant configuration, each node in the cluster should be configured. Prior to ALL of the Dual-Port 4x Adapter User Guide to prevent the eth device from starting up automatically on boot. Once the HCA drivers have been installed, follow the instructions in the Configuring IPoIB Drivers section of HP and Oracle software pieces. Create the /etc/sysconfig/network-scripts/ifcfg-ib0 file...
... interface using the following command: ifup ib0 9 In a redundant configuration, each node in the cluster should be configured. Prior to ALL of the Dual-Port 4x Adapter User Guide to prevent the eth device from starting up automatically on boot. Once the HCA drivers have been installed, follow the instructions in the Configuring IPoIB Drivers section of HP and Oracle software pieces. Create the /etc/sysconfig/network-scripts/ifcfg-ib0 file...
Fabric Clustering System 24-port 4x Fabric Copper Switch Installation Guide, March 2005
Page 5
... the hardware and software that support these products, see "http://www.infinibandta.org". Click on supported servers, HP-UX versions, drivers, and patches, is built around switched fabric architecture, providing the bandwidth necessary for purchasing HP I /O Cards or related documentation. 1 HP-UX Integrity Installation Overview & Planning This chapter contains HP-UX installation information for the 376227-B21 HP 24-port 4x Fabric Copper Switch and contains the following : • Technical Computing Clusters supporting HP Message Passing Interface...
... the hardware and software that support these products, see "http://www.infinibandta.org". Click on supported servers, HP-UX versions, drivers, and patches, is built around switched fabric architecture, providing the bandwidth necessary for purchasing HP I /O Cards or related documentation. 1 HP-UX Integrity Installation Overview & Planning This chapter contains HP-UX installation information for the 376227-B21 HP 24-port 4x Fabric Copper Switch and contains the following : • Technical Computing Clusters supporting HP Message Passing Interface...
Fabric Clustering System 24-port 4x Fabric Copper Switch Installation Guide, March 2005
Page 19
... name. 3. InfiniBand provides higher speed, lower network latency and uses less CPU than other industry standard protocols (for example, Fibre Channel and Gigabit Ethernet). This document contains the following : • Technical Computing Clusters supporting HP Message Passing Interface (MPI) based applications Kit Contents • HP 24-Port Fabric Copper Switch • Switch Bezel • Serial management cable, DB-9, 10 ft (3m) • Two sets of how to install the HP 24-Port 4x Fabric Copper Switch. Select Software and Driver downloads. 2. For...
... name. 3. InfiniBand provides higher speed, lower network latency and uses less CPU than other industry standard protocols (for example, Fibre Channel and Gigabit Ethernet). This document contains the following : • Technical Computing Clusters supporting HP Message Passing Interface (MPI) based applications Kit Contents • HP 24-Port Fabric Copper Switch • Switch Bezel • Serial management cable, DB-9, 10 ft (3m) • Two sets of how to install the HP 24-Port 4x Fabric Copper Switch. Select Software and Driver downloads. 2. For...
Fabric Clustering System 24-port 4x Fabric Copper Switch Installation Guide, March 2005
Page 20
... power cables, IEC C14 • This installation guide • Important Safety Instructions booklet • Limited Warranty and Material Limitations Documentation HP Fabric Clustering System 24-Port Switch 376227-B21 HP 24-port 4x Fabric Copper Switch Figure 2-1 Front View of Switch Figure 2-2 Rear View of 24-Port Switch Figure 2-3 Power Plug Stylized View of Switch Connectors (24) Switch Cluster Connections Power Plug Management Port Console Port Install HP Fabric Clustering System Switches This section describes how to "Remove Shipping Sleeve" on page 23. 20 Chapter 2 ProLiant...
... power cables, IEC C14 • This installation guide • Important Safety Instructions booklet • Limited Warranty and Material Limitations Documentation HP Fabric Clustering System 24-Port Switch 376227-B21 HP 24-port 4x Fabric Copper Switch Figure 2-1 Front View of Switch Figure 2-2 Rear View of 24-Port Switch Figure 2-3 Power Plug Stylized View of Switch Connectors (24) Switch Cluster Connections Power Plug Management Port Console Port Install HP Fabric Clustering System Switches This section describes how to "Remove Shipping Sleeve" on page 23. 20 Chapter 2 ProLiant...
Fabric Clustering System 24-port 4x Fabric Copper Switch Installation Guide, March 2005
Page 24
... Connectivity 1. Figure 2-11 Remove Support Bracket Support Bracket Mount the bracket to the rack using a program such as HyperTerminal for Windows. use the cable kit that the bracket orientation matches your terminal parameters to another location, the bracket is mounted in place. Open a terminal emulation windows using four screws. For more information, see the HP 24-Port 4x Fabric Copper Switch Hardware User Guide. Attach the serial management cable. Insert the power cords. Connect the serial management cable from which the power supplies are accessible...
... Connectivity 1. Figure 2-11 Remove Support Bracket Support Bracket Mount the bracket to the rack using a program such as HyperTerminal for Windows. use the cable kit that the bracket orientation matches your terminal parameters to another location, the bracket is mounted in place. Open a terminal emulation windows using four screws. For more information, see the HP 24-Port 4x Fabric Copper Switch Hardware User Guide. Attach the serial management cable. Insert the power cords. Connect the serial management cable from which the power supplies are accessible...
Fabric Clustering System 24-port 4x Fabric Copper Switch Installation Guide, March 2005
Page 25
...)# exit HP(config)# exit HP# copy running-config startup-config Attach Cables Copper cables for InfiniBand should be installed in a horizontal fashion. For example: HP(config-if-mgmt-ethernet)# ip address 10.10.0.22 255.255.255.0 g. HP(config-if-mgmt-ethernet)# no shutdown. Enter the default user name super and password super. Enable the management port by the netmask. ProLiant Linux Installation Overview & Planning Attach Cables 3. HP(config)# interface mgmt-ethernet HP(config-if-mgmt-ethernet)# f. Cable Guide Installation Instructions are provided as a separate document...
...)# exit HP(config)# exit HP# copy running-config startup-config Attach Cables Copper cables for InfiniBand should be installed in a horizontal fashion. For example: HP(config-if-mgmt-ethernet)# ip address 10.10.0.22 255.255.255.0 g. HP(config-if-mgmt-ethernet)# no shutdown. Enter the default user name super and password super. Enable the management port by the netmask. ProLiant Linux Installation Overview & Planning Attach Cables 3. HP(config)# interface mgmt-ethernet HP(config-if-mgmt-ethernet)# f. Cable Guide Installation Instructions are provided as a separate document...
24-Port 4x Fabric Copper Switch Chassis Manager User Guide
Page 91
...23 speed, configuring 28 view all 23 view internal gateway 22 ports, InfiniBand viewing 51 Ports, InfiniBand Ports properties 51 power supply status 29 prepare your switch 9 prerequisites 9 printed circuit assembly (PCA 21 R RADIUS adding 41 configuring 40 deleting 42 viewing 40 RADIUS server configuring encryption key 40 configuring max retry value 41 configuring timeout 41 configuring UDP port 40 rebooting 38 reloading 38 S saving configuration files 38 serial number 21 services (basic), configuring 38 setup 9 slot ID 19 software, installing 36 speed, port speed 28...
...23 speed, configuring 28 view all 23 view internal gateway 22 ports, InfiniBand viewing 51 Ports, InfiniBand Ports properties 51 power supply status 29 prepare your switch 9 prerequisites 9 printed circuit assembly (PCA 21 R RADIUS adding 41 configuring 40 deleting 42 viewing 40 RADIUS server configuring encryption key 40 configuring max retry value 41 configuring timeout 41 configuring UDP port 40 rebooting 38 reloading 38 S saving configuration files 38 serial number 21 services (basic), configuring 38 setup 9 slot ID 19 software, installing 36 speed, port speed 28...
24-Port 4x Fabric Copper Switch Command Line Reference Guide
Page 12
... -write access for flexible control. Use this login (refer to individual user accounts. 2 • Baud: 9600 b/s • Data Bits: 8 • Parity: None • Stop Bits: 1 • Flow control: None 3. Table 1-2 lists and describes these roles. When you can change only its own user information, such as super, admin, or guest. This user configures the console and management ports for initial configuration. NOTE: The CLI is case-sensitive. The HP 24-Port 4x Fabric Copper Switch User Guide describes this...
... -write access for flexible control. Use this login (refer to individual user accounts. 2 • Baud: 9600 b/s • Data Bits: 8 • Parity: None • Stop Bits: 1 • Flow control: None 3. Table 1-2 lists and describes these roles. When you can change only its own user information, such as super, admin, or guest. This user configures the console and management ports for initial configuration. NOTE: The CLI is case-sensitive. The HP 24-Port 4x Fabric Copper Switch User Guide describes this...
24-Port 4x Fabric Copper Switch Command Line Reference Guide
Page 14
... telnet topspin-360 Login: super Password: xxxx Topspin-360> enable Topspin-360# Mode changes are reflected in submodes. All Ethernet, FC, and IB configuration occurs in changes to the CLI prompt. Example Topspin-360(config)# interface ethernet 4/1-4/4 Topspin-360(config-if-ether-4/1-4/4)# The commands that element. Example Topspin-360(config)# interface mgmt-ethernet Topspin-360(config-if-mgmt-ethernet)# You can enter configuration modes. When you can assign IP addresses to interface gateway ports, set connection speeds, set connection types, etc. 4 mode, you transition from User...
... telnet topspin-360 Login: super Password: xxxx Topspin-360> enable Topspin-360# Mode changes are reflected in submodes. All Ethernet, FC, and IB configuration occurs in changes to the CLI prompt. Example Topspin-360(config)# interface ethernet 4/1-4/4 Topspin-360(config-if-ether-4/1-4/4)# The commands that element. Example Topspin-360(config)# interface mgmt-ethernet Topspin-360(config-if-mgmt-ethernet)# You can enter configuration modes. When you can assign IP addresses to interface gateway ports, set connection speeds, set connection types, etc. 4 mode, you transition from User...
24-Port 4x Fabric Copper Switch Command Line Reference Guide
Page 51
... Availability: HP 24-Port 4x Fabric Copper Switch, Topspin 90, Topspin 120, Topspin 270, Topspin 360, Topspin IB Server Switch Module Command Modes: Ethernet Management Interface Configuration (config-if-mgmt-ethernet) submode, InfiniBand Management Interface Configuration (config-if-mgmt-ib) submode. Usage Guidelines: The gateway that you configure the management interfaces. Topspin-360(config-if-mgmt-ethernet)# gateway 10.3.0.94 The following example assigns a default IP gateway to the port. Topspin-360(config-if-mgmt-ib)# gateway 10.3.0.2 Defaults: The gateway address defaults to...
... Availability: HP 24-Port 4x Fabric Copper Switch, Topspin 90, Topspin 120, Topspin 270, Topspin 360, Topspin IB Server Switch Module Command Modes: Ethernet Management Interface Configuration (config-if-mgmt-ethernet) submode, InfiniBand Management Interface Configuration (config-if-mgmt-ib) submode. Usage Guidelines: The gateway that you configure the management interfaces. Topspin-360(config-if-mgmt-ethernet)# gateway 10.3.0.94 The following example assigns a default IP gateway to the port. Topspin-360(config-if-mgmt-ib)# gateway 10.3.0.2 Defaults: The gateway address defaults to...
24-Port 4x Fabric Copper Switch Command Line Reference Guide
Page 57
...Server Switch Module Command Modes: Global Configuration (config) mode. For instance, enter Ethernet Management Interface Configuration submode to configure the attributes of a port, a list of ports, or a range of slot#/port# pairs to create Ethernet trunk groups. mgmt-ib Enters InfiniBand Management Interface Configuration submode to administer your switch. Privilege Level: Unrestricted read-write user. Usage Guidelines: Enter an Interface Configuration submode to enter the ip and ID number (integer) of the appropriate type. You can use this port to configure...
...Server Switch Module Command Modes: Global Configuration (config) mode. For instance, enter Ethernet Management Interface Configuration submode to configure the attributes of a port, a list of ports, or a range of slot#/port# pairs to create Ethernet trunk groups. mgmt-ib Enters InfiniBand Management Interface Configuration submode to administer your switch. Privilege Level: Unrestricted read-write user. Usage Guidelines: Enter an Interface Configuration submode to enter the ip and ID number (integer) of the appropriate type. You can use this port to configure...
24-Port 4x Fabric Copper Switch Command Line Reference Guide
Page 82
... 2 Gbps. Platform Availability: HP 24-Port 4x Fabric Copper Switch, Topspin 90, Topspin 120, Topspin 270, Topspin 360, Topspin IB Server Switch Module Command Modes: Fibre Channel Interface Configuration (config-if-fc) mode, Ethernet Interface Configuration (config-if-ether) submode, InfiniBand Interface Configuration (config-if-ib) submode (select switches). Enter the no auto-negotiate command before you enable auto-negotiation. You must disable the auto-negotiation feature to manually configure speed. Ethernet: The speed command sets the administrative speed (the speed that you...
... 2 Gbps. Platform Availability: HP 24-Port 4x Fabric Copper Switch, Topspin 90, Topspin 120, Topspin 270, Topspin 360, Topspin IB Server Switch Module Command Modes: Fibre Channel Interface Configuration (config-if-fc) mode, Ethernet Interface Configuration (config-if-ether) submode, InfiniBand Interface Configuration (config-if-ib) submode (select switches). Enter the no auto-negotiate command before you enable auto-negotiation. You must disable the auto-negotiation feature to manually configure speed. Ethernet: The speed command sets the administrative speed (the speed that you...
24-Port 4x Fabric Copper Switch Command Line Reference Guide
Page 275
...: HP 24-Port 4x Fabric Copper Switch, Topspin 90, Topspin 120, Topspin 270, Topspin 360, Topspin IB Server Switch Module Command Modes: User Exec mode, Privileged Exec mode. This port must be changed. This configuration cannot be configured and a management station attached before any interaction with the switch chassis and is possible. Examples: Topspin-90# show interface mgmt-serial command in User Exec mode or Privileged Exec mode. The Serial Console port is the initial connection point with the switch chassis is used...
...: HP 24-Port 4x Fabric Copper Switch, Topspin 90, Topspin 120, Topspin 270, Topspin 360, Topspin IB Server Switch Module Command Modes: User Exec mode, Privileged Exec mode. This port must be changed. This configuration cannot be configured and a management station attached before any interaction with the switch chassis and is possible. Examples: Topspin-90# show interface mgmt-serial command in User Exec mode or Privileged Exec mode. The Serial Console port is the initial connection point with the switch chassis is used...
24-Port 4x Fabric Copper Switch Quick Setup Instructions
Page 1
... rails to rack • Four rubber mounting feet for desktop installations • Rack support bracket • Two power cables, IEC C14 • This installation card • Important Safety Instructions booklet • Limited Warranty and Material Limitations Documentation Requirements In addition to the accessories provided with the sliding rail assembled to the HP 24-Port 4x Fabric Copper Switch Hardware User Guide. Each set forth in the kit. 2. Install the rack support bracket, if...
... rails to rack • Four rubber mounting feet for desktop installations • Rack support bracket • Two power cables, IEC C14 • This installation card • Important Safety Instructions booklet • Limited Warranty and Material Limitations Documentation Requirements In addition to the accessories provided with the sliding rail assembled to the HP 24-Port 4x Fabric Copper Switch Hardware User Guide. Each set forth in the kit. 2. Install the rack support bracket, if...
24-Port 4x Fabric Copper Switch Quick Setup Instructions
Page 2
... User Interface (GUI) • HP 24-Port 4x Fabric Copper Switch Command Line Reference Guide-for information on the commands used to manage and troubleshoot the switch • HP 24-Port 4x Fabric Copper Switch Chassis Manager User Guide-for information on how to use the web-based GUI to manage and troubleshoot the switch • HP Serial Management Cable Guide-for Windows®. Insert the power cords. It is not pre-configured when it between reboots. When the switch has completed booting, press Enter several times to enter config-if-mgmtethernet mode. Login: super Password: super HP...
... User Interface (GUI) • HP 24-Port 4x Fabric Copper Switch Command Line Reference Guide-for information on the commands used to manage and troubleshoot the switch • HP 24-Port 4x Fabric Copper Switch Chassis Manager User Guide-for information on how to use the web-based GUI to manage and troubleshoot the switch • HP Serial Management Cable Guide-for Windows®. Insert the power cords. It is not pre-configured when it between reboots. When the switch has completed booting, press Enter several times to enter config-if-mgmtethernet mode. Login: super Password: super HP...
24-Port 4x Fabric Copper Switch User Guide
Page 24
...Multiple Switches Multiple switches can be active and able to the following methods: • Command Line Interface (CLI)-a text-based interface accessible through a direct serial connection, Telnet over IP, or SSH over IP • Chassis Manager (GUI)-A web-based graphic user interface. • Element Manager (GUI)-A graphic interface installed on configuring the SM and Database Synchronization. It is recommended that can manage the IB server-switch system using the following : • HP 24-Port 4x Fabric Copper Switch User Guide • HP 24-Port 4x Fabric Copper Switch Command Line...
...Multiple Switches Multiple switches can be active and able to the following methods: • Command Line Interface (CLI)-a text-based interface accessible through a direct serial connection, Telnet over IP, or SSH over IP • Chassis Manager (GUI)-A web-based graphic user interface. • Element Manager (GUI)-A graphic interface installed on configuring the SM and Database Synchronization. It is recommended that can manage the IB server-switch system using the following : • HP 24-Port 4x Fabric Copper Switch User Guide • HP 24-Port 4x Fabric Copper Switch Command Line...
24-Port 4x Fabric Copper Switch User Guide
Page 3
... HP 24-Port 4x Fabric Copper Switch User Guide 1 Maximize Server Networks ...1 What is InfiniBand? ...2 How Does InfiniBand Work? ...2 Possible Components ...2 Protocols ...2 Architectural Elements ...3 Understanding the Subnet Manager (SM 4 Understanding Subnet Manager Routing 5 2: Getting Started 9 Overview of Set-up Procedures ...9 Install HCAs and Drivers in Hosts ...10 Install and Power on the InfiniBand Chassis 10 Configure Basic Connectivity...10 Configuring an Ethernet Management IP Address 10 Configuring a Direct Serial-Console Connection 11 Configuring an InfiniBand Management...
... HP 24-Port 4x Fabric Copper Switch User Guide 1 Maximize Server Networks ...1 What is InfiniBand? ...2 How Does InfiniBand Work? ...2 Possible Components ...2 Protocols ...2 Architectural Elements ...3 Understanding the Subnet Manager (SM 4 Understanding Subnet Manager Routing 5 2: Getting Started 9 Overview of Set-up Procedures ...9 Install HCAs and Drivers in Hosts ...10 Install and Power on the InfiniBand Chassis 10 Configure Basic Connectivity...10 Configuring an Ethernet Management IP Address 10 Configuring a Direct Serial-Console Connection 11 Configuring an InfiniBand Management...
24-Port 4x Fabric Copper Switch User Guide
Page 26
... Manager) HP 24-Port 4x Fabric Copper Switches can be managed visually through the CLI. Remote Telnet Login You can reach a host on the network, enter either the user-execute or privileged-execute mode on the InfiniBand device, then enter the ping command. Success rate is an In-band procedure only. Use the following methods: • "Direct Serial-Console Connection" on page 14 • "Remote Telnet Login" on page 14 • "Remote SSH Login" on page 15 Direct Serial-Console Connection Refer...
... Manager) HP 24-Port 4x Fabric Copper Switches can be managed visually through the CLI. Remote Telnet Login You can reach a host on the network, enter either the user-execute or privileged-execute mode on the InfiniBand device, then enter the ping command. Success rate is an In-band procedure only. Use the following methods: • "Direct Serial-Console Connection" on page 14 • "Remote Telnet Login" on page 14 • "Remote SSH Login" on page 15 Direct Serial-Console Connection Refer...
24-Port 4x Fabric Copper Switch User Guide
Page 30
...-login : Topspin-90> Topspin-90> show user command to page 47). 2. Log in your fabric: • Exchange the partition configuration between switches by enabling database synchronization, if you have multiple InfiniBand switches in to "Enable/Disable Database Synchronization" on page 84. Exit the global-configuration mode. 7. Enter the global-configuration mode. 4. Only a user with unrestricted privileges may view user information. If you have not already been changed (refer to change the default passwords after initial configuration. Set User...
...-login : Topspin-90> Topspin-90> show user command to page 47). 2. Log in your fabric: • Exchange the partition configuration between switches by enabling database synchronization, if you have multiple InfiniBand switches in to "Enable/Disable Database Synchronization" on page 84. Exit the global-configuration mode. 7. Enter the global-configuration mode. 4. Only a user with unrestricted privileges may view user information. If you have not already been changed (refer to change the default passwords after initial configuration. Set User...
Dual-port 4x Fabric Adapter User Guide
Page 79
... Server 1, using the ping command. Also, check the LEDs on page 61. 67 IPoIB Setup About IPoIB IPoIB (IP over InfiniBand) is simply that: IP packets running over IP. If the system failed to communicate over the IB fabric. If the drivers did not install, it is plugged into the fabric between the driver suite and the installed kernel. 4. To test connectivity, attempt to configure ib0. On Server 1, use...
... Server 1, using the ping command. Also, check the LEDs on page 61. 67 IPoIB Setup About IPoIB IPoIB (IP over InfiniBand) is simply that: IP packets running over IP. If the system failed to communicate over the IB fabric. If the drivers did not install, it is plugged into the fabric between the driver suite and the installed kernel. 4. To test connectivity, attempt to configure ib0. On Server 1, use...