Troubleshooting Guide
Page 7
Contents Send documentation comments to mdsfeedback-doc@cisco.com Port Index Availability 5-4 Combining Modules and Supervisors 5-7 Initial Troubleshooting Checklist 5-7 Generation 1 and Generation 2 Issues 5-7 Module Does Not Come Online ...Licenses in Use for Multiple Modules 6-8 Grace Period Alerts 6-9 Checking in the Fabric Manager Server License From Device Manager 6-10 License Listed as Missing 6-11 7 C H A P T E R Troubleshooting Cisco Fabric Services 7-1 Overview 7-1 Initial Troubleshooting Checklist 7-2 Verifying CFS Using Fabric Manager 7-3 Verifying CFS Using the CLI 7-3 OL-9285-05...
Contents Send documentation comments to mdsfeedback-doc@cisco.com Port Index Availability 5-4 Combining Modules and Supervisors 5-7 Initial Troubleshooting Checklist 5-7 Generation 1 and Generation 2 Issues 5-7 Module Does Not Come Online ...Licenses in Use for Multiple Modules 6-8 Grace Period Alerts 6-9 Checking in the Fabric Manager Server License From Device Manager 6-10 License Listed as Missing 6-11 7 C H A P T E R Troubleshooting Cisco Fabric Services 7-1 Overview 7-1 Initial Troubleshooting Checklist 7-2 Verifying CFS Using Fabric Manager 7-3 Verifying CFS Using the CLI 7-3 OL-9285-05...
Troubleshooting Guide
Page 11
Contents Send documentation comments to mdsfeedback-doc@cisco.com Assigning a New Domain ID Using the CLI 11-19 Using Fabric Reconfiguration for Domain ID Assignments 11-20 CFS Distribution of Domain ID List Fails 11-22 Allowed Domain ID List Incorrect After a VSAN Merge 11-23 Changes to fcdomain Do Not Take Effect 11... 13-1 Configuration Guidelines 13-1 Transit VSANs 13-2 Border Switches 13-2 Limitations and Restrictions 13-2 Initial Troubleshooting Checklist 13-3 Verifying IVR Configuration Using Fabric Manager 13-3 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x xi
Contents Send documentation comments to mdsfeedback-doc@cisco.com Assigning a New Domain ID Using the CLI 11-19 Using Fabric Reconfiguration for Domain ID Assignments 11-20 CFS Distribution of Domain ID List Fails 11-22 Allowed Domain ID List Incorrect After a VSAN Merge 11-23 Changes to fcdomain Do Not Take Effect 11... 13-1 Configuration Guidelines 13-1 Transit VSANs 13-2 Border Switches 13-2 Limitations and Restrictions 13-2 Initial Troubleshooting Checklist 13-3 Verifying IVR Configuration Using Fabric Manager 13-3 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x xi
Troubleshooting Guide
Page 19
...-64 iSLB Zone Set Activation Failed 20-64 iSLB CFS Commit Fails 20-65 Resolving an iSLB Merge Failure 20-65 21 C H A P T E R Troubleshooting IP Access Lists 21-1 Overview 21-1 Protocol Information 21-1 Address Information 21-2 Port Information 21-2 ICMP Information 21-3 ToS Information 21-3 Initial Troubleshooting Checklist 21-4 Common Troubleshooting Tools... Common Troubleshooting Tools in Fabric Manager 22-4 Common Troubleshooting Commands in the CLI 22-5 IPsec Issues 22-5 Verifying IKE Configuration Compatibility 22-6 OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x xix
...-64 iSLB Zone Set Activation Failed 20-64 iSLB CFS Commit Fails 20-65 Resolving an iSLB Merge Failure 20-65 21 C H A P T E R Troubleshooting IP Access Lists 21-1 Overview 21-1 Protocol Information 21-1 Address Information 21-2 Port Information 21-2 ICMP Information 21-3 ToS Information 21-3 Initial Troubleshooting Checklist 21-4 Common Troubleshooting Tools... Common Troubleshooting Tools in Fabric Manager 22-4 Common Troubleshooting Commands in the CLI 22-5 IPsec Issues 22-5 Verifying IKE Configuration Compatibility 22-6 OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x xix
Troubleshooting Guide
Page 25
...in the website mentioned above. Select and view the troubleshooting guide pertinent to address each new feature and the release in Cisco SAN-OS Release 3.x. The latest version of each new and changed guideline in 3.3(1). 3.3(1) Chapter 23, "Troubleshooting SANTap... troubleshooting information for CFS Regions feature. 3.2(1) Chapter 7, "Troubleshooting Cisco Fabric Services" Updated configuration limits for the Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x, and tells you where they are also listed in your switch. Each guide addresses the features introduced in or...
...in the website mentioned above. Select and view the troubleshooting guide pertinent to address each new feature and the release in Cisco SAN-OS Release 3.x. The latest version of each new and changed guideline in 3.3(1). 3.3(1) Chapter 23, "Troubleshooting SANTap... troubleshooting information for CFS Regions feature. 3.2(1) Chapter 7, "Troubleshooting Cisco Fabric Services" Updated configuration limits for the Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x, and tells you where they are also listed in your switch. Each guide addresses the features introduced in or...
Troubleshooting Guide
Page 26
... Tools and Methodology" Added information about troubleshooting 3.1(2) nondisruptive upgrades on Cisco MDS 9124 Fabric Switch SAN Device Virtualization SNMP test traps Digital Certificates iSCSI Load Balancing Mixed Generation Hardware AAA Call Home FC-SP, port security, fabric binding IP Access Lists Users and Roles Description Changed in Release Where Documented Updated FICON...
... Tools and Methodology" Added information about troubleshooting 3.1(2) nondisruptive upgrades on Cisco MDS 9124 Fabric Switch SAN Device Virtualization SNMP test traps Digital Certificates iSCSI Load Balancing Mixed Generation Hardware AAA Call Home FC-SP, port security, fabric binding IP Access Lists Users and Roles Description Changed in Release Where Documented Updated FICON...
Troubleshooting Guide
Page 28
...Describes how to troubleshoot role-based access control. Document Organization Preface Send documentation comments to mdsfeedback-doc@cisco.com Chapter Chapter 10 Title Description Troubleshooting PortChannels and Describes how to identify and resolve problems Trunking ...and Roles Troubleshooting FC-SP, Port Security, and Fabric Binding Troubleshooting IP Storage Services Troubleshooting IP Access Lists Troubleshooting IPsec Troubleshooting SANTap Troubleshooting Digital Certificates Troubleshooting Call Home Troubleshooting Fabric Manager Before Contacting Technical Support ...
...Describes how to troubleshoot role-based access control. Document Organization Preface Send documentation comments to mdsfeedback-doc@cisco.com Chapter Chapter 10 Title Description Troubleshooting PortChannels and Describes how to identify and resolve problems Trunking ...and Roles Troubleshooting FC-SP, Port Security, and Fabric Binding Troubleshooting IP Storage Services Troubleshooting IP Access Lists Troubleshooting IPsec Troubleshooting SANTap Troubleshooting Digital Certificates Troubleshooting Call Home Troubleshooting Fabric Manager Before Contacting Technical Support ...
Troubleshooting Guide
Page 29
...boldface font italic font [ ] [ x | y | z ] string Description Commands and keywords are in boldface screen font. OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x xxix italic screen font Arguments for which you might do something that could result in equipment damage or ...loss of data. This document uses the following conventions: Note Means reader take note. Lists configuration limits for the Cisco MDS 9000 Family product. In this situation, you supply values are in italic screen font. < > Nonprinting characters, ...
...boldface font italic font [ ] [ x | y | z ] string Description Commands and keywords are in boldface screen font. OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x xxix italic screen font Arguments for which you might do something that could result in equipment damage or ...loss of data. This document uses the following conventions: Note Means reader take note. Lists configuration limits for the Cisco MDS 9000 Family product. In this situation, you supply values are in italic screen font. < > Nonprinting characters, ...
Troubleshooting Guide
Page 32
... support, providing documentation feedback, security guidelines, and also recommended aliases and general Cisco documents, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at: http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html xxxii Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05
... support, providing documentation feedback, security guidelines, and also recommended aliases and general Cisco documents, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at: http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html xxxii Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05
Troubleshooting Guide
Page 36
...accounting log Note Use the show running -config CLI command is no longer consolidated. The interface configuration as seen in Cisco SAN-OS Release 3.0(1) or later. Fabric Manager and Device Manager also provide the following questions to see the LUNs...an account that is a member of the network-admin group. If yes, is the server allowed to mdsfeedback-doc@cisco.com • View traffic statistics-In Device Manager, click the Device tab, select one or more ports, right-... access? • If LUN masking software is used, is the host's pWWN listed in the LUN masking database?
...accounting log Note Use the show running -config CLI command is no longer consolidated. The interface configuration as seen in Cisco SAN-OS Release 3.0(1) or later. Fabric Manager and Device Manager also provide the following questions to see the LUNs...an account that is a member of the network-admin group. If yes, is the server allowed to mdsfeedback-doc@cisco.com • View traffic statistics-In Device Manager, click the Device tab, select one or more ports, right-... access? • If LUN masking software is used, is the host's pWWN listed in the LUN masking database?
Troubleshooting Guide
Page 37
...Verifying SAN Element Registration Answer the following questions to verify that your end devices are members of the fabric configuration: OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 1-5 Fibre Channel End-to-End Connectivity Answering the following questions will help to determine the ...> Edit Full Zone Database and select the active zone set (in the FCNS database then ISLs are identical. - Verify that both devices are listed in bold) for an N port? In Fabric Manager, choose Tools > Ping or Tools > Traceroute (or use the show fcns commands. ...
...Verifying SAN Element Registration Answer the following questions to verify that your end devices are members of the fabric configuration: OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 1-5 Fibre Channel End-to-End Connectivity Answering the following questions will help to determine the ...> Edit Full Zone Database and select the active zone set (in the FCNS database then ISLs are identical. - Verify that both devices are listed in bold) for an N port? In Fabric Manager, choose Tools > Ping or Tools > Traceroute (or use the show fcns commands. ...
Troubleshooting Guide
Page 42
... • Implementing Syslog with Fabric Manager, page 1-11 • Implementing Syslog with your customer support representative for a list of time or when the Cisco MDS switch is not accessible. Please contact your system. The action may involve a fix or a recommendation to contact technical... problems with links, internal hardware, or the system software. This portion of the message log to locations in the Cisco MDS 9000 Family System Messages Reference. This section contains the following example: Error Message PORT-3-IF_UNSUPPORTED_TRANSCEIVER: Transceiver for more ...
... • Implementing Syslog with Fabric Manager, page 1-11 • Implementing Syslog with your customer support representative for a list of time or when the Cisco MDS switch is not accessible. Please contact your system. The action may involve a fix or a recommendation to contact technical... problems with links, internal hardware, or the system software. This portion of the message log to locations in the Cisco MDS 9000 Family System Messages Reference. This section contains the following example: Error Message PORT-3-IF_UNSUPPORTED_TRANSCEIVER: Transceiver for more ...
Troubleshooting Guide
Page 44
... creating an event on the syslog server. End with the CLI To configure a syslog server using PCs that the IP address of the switch is listed in brackets. # tail -f /var/adm/MDS_logs Sep 17 11:07:41 [172.22.36.142.2.2] : 2004 Sep 17 11:17:29 pacific: %PORT-5-... [172.22.36.142.2.2] : 2004 Sep 17 11:17:39 pacific: %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configuring console from pts/0 (dhcp-171-71-49-125.cisco.com) 1-12 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 In this situation. however the Device Manager prompts you manually change it does detect this...
... creating an event on the syslog server. End with the CLI To configure a syslog server using PCs that the IP address of the switch is listed in brackets. # tail -f /var/adm/MDS_logs Sep 17 11:07:41 [172.22.36.142.2.2] : 2004 Sep 17 11:17:29 pacific: %PORT-5-... [172.22.36.142.2.2] : 2004 Sep 17 11:17:39 pacific: %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configuring console from pts/0 (dhcp-171-71-49-125.cisco.com) 1-12 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 In this situation. however the Device Manager prompts you manually change it does detect this...
Troubleshooting Guide
Page 46
...15:12:07 172.20.150.82 %MODULE-2-MOD_FAIL: Initialization of memory constraints, these syslog messages in the switch's syslog server list. Hardware messages are sortable. Viewing NVRAM logs System messages that contains notice or greater severity messages. Please wait until completion... 2005... command to view the logs on Device Manager. Troubleshooting with Logs Chapter 1 Troubleshooting Overview Send documentation comments to mdsfeedback-doc@cisco.com Viewing the Log from the Supervisor You can view system messages from the same workstation as the Fabric Manager Server. ...
...15:12:07 172.20.150.82 %MODULE-2-MOD_FAIL: Initialization of memory constraints, these syslog messages in the switch's syslog server list. Hardware messages are sortable. Viewing NVRAM logs System messages that contains notice or greater severity messages. Please wait until completion... 2005... command to view the logs on Device Manager. Troubleshooting with Logs Chapter 1 Troubleshooting Overview Send documentation comments to mdsfeedback-doc@cisco.com Viewing the Log from the Supervisor You can view system messages from the same workstation as the Fabric Manager Server. ...
Troubleshooting Guide
Page 50
... on images in the directories of your fabric. Guidelines Chapter 2 Troubleshooting Installs, Upgrades, and Reboots Send documentation comments to mdsfeedback-doc@cisco.com Guidelines This sections lists general guidelines for performing Cisco SAN-OS software installations, image upgrade and downgrade procedures, and reboots, and it includes the following topics: • Guidelines for Installations...
... on images in the directories of your fabric. Guidelines Chapter 2 Troubleshooting Installs, Upgrades, and Reboots Send documentation comments to mdsfeedback-doc@cisco.com Guidelines This sections lists general guidelines for performing Cisco SAN-OS software installations, image upgrade and downgrade procedures, and reboots, and it includes the following topics: • Guidelines for Installations...
Troubleshooting Guide
Page 58
... switches you want to install images on LINUX). If any switch has insufficient space, you cannot transfer files from a drop-down list of switches you can see the Software Install Wizard. Click the row under System, Kickstart, Asm-sfn, or Ssi and select from... the local host to the switch. 2-10 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 Troubleshooting Cisco SAN-OS Software Upgrades and Downgrades Chapter 2 Troubleshooting Installs, Upgrades, and Reboots Send documentation comments to mdsfeedback...
... switches you want to install images on LINUX). If any switch has insufficient space, you cannot transfer files from a drop-down list of switches you can see the Software Install Wizard. Click the row under System, Kickstart, Asm-sfn, or Ssi and select from... the local host to the switch. 2-10 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 Troubleshooting Cisco SAN-OS Software Upgrades and Downgrades Chapter 2 Troubleshooting Installs, Upgrades, and Reboots Send documentation comments to mdsfeedback...
Troubleshooting Guide
Page 61
Troubleshooting Cisco SAN-OS Software System Reboots This section lists possible problems and solutions for software reboots and includes the following topics: • Power On or Switch Reboot Hangs, page 2-13 • Corrupted Bootflash ...the standby supervisor. -- Power on or switch reboot The bootflash is corrupted. See the "Recovery Using BIOS Setup for dual supervisor configuration. OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 2-13 Solution See the "Recovery for Switches with standby terminated. Table 2-5 Power-on or Switch Reboot Hangs ...
Troubleshooting Cisco SAN-OS Software System Reboots This section lists possible problems and solutions for software reboots and includes the following topics: • Power On or Switch Reboot Hangs, page 2-13 • Corrupted Bootflash ...the standby supervisor. -- Power on or switch reboot The bootflash is corrupted. See the "Recovery Using BIOS Setup for dual supervisor configuration. OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 2-13 Solution See the "Recovery for Switches with standby terminated. Table 2-5 Power-on or Switch Reboot Hangs ...
Troubleshooting Guide
Page 67
...prompt, and press Enter. loader> net --ip=172.16.1.2 Specify the subnet mask. OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 2-19 The switch(boot)# prompt indicates that list. Issue the init system command at this procedure. switch(boot)# init system Caution Be sure that you ...the switch(boot)# Prompt" section on that you have made a backup of the configuration files before you issue this prompt or to display a list of the kickstart image file that exists on page 2-21. loader> net --gw=172.16.1.1 Boot the kickstart image file from the regular ...
...prompt, and press Enter. loader> net --ip=172.16.1.2 Specify the subnet mask. OL-9285-05 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x 2-19 The switch(boot)# prompt indicates that list. Issue the init system command at this procedure. switch(boot)# init system Caution Be sure that you ...the switch(boot)# Prompt" section on that you have made a backup of the configuration files before you issue this prompt or to display a list of the kickstart image file that exists on page 2-21. loader> net --gw=172.16.1.1 Boot the kickstart image file from the regular ...
Troubleshooting Guide
Page 68
...= "2.1(2 Image verification OK Starting kernel... done. INIT: Going single user INIT: Sending processes the TERM signal switch(boot)# 2-20 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 The loader> prompt is not supported, so you want the command to appear...feature does not work at this prompt and may result in that list. Be sure that you begin this procedure. INIT: version 2.78 booting Checking all filesystems..... Entering single-user mode... Troubleshooting Cisco SAN-OS Software System Reboots Chapter 2 Troubleshooting Installs, Upgrades, and...
...= "2.1(2 Image verification OK Starting kernel... done. INIT: Going single user INIT: Sending processes the TERM signal switch(boot)# 2-20 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 The loader> prompt is not supported, so you want the command to appear...feature does not work at this prompt and may result in that list. Be sure that you begin this procedure. INIT: version 2.78 booting Checking all filesystems..... Entering single-user mode... Troubleshooting Cisco SAN-OS Software System Reboots Chapter 2 Troubleshooting Installs, Upgrades, and...
Troubleshooting Guide
Page 72
...172.16.10.100 Gateway: 172.16.1.1 Booting: /kick-282 console=ttyS0,9600n8nn quiet loader_ver= "2.1(2 Image verification OK Starting kernel... 2-24 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 Caution The loader> prompt is different from the required server. Wait..... If you ....1.2 Netmask: 255.255.255.0 Server: 0.0.0.0 Gateway: 0.0.0.0 Specify the IP address of commands available at this prompt and may result in that list. loader> ip address 172.16.1.2 255.255.255.0 Found Intel EtherExpressPro100 82559ER at this prompt or to run SETUP.... Step 2 Step 3 ...
...172.16.10.100 Gateway: 172.16.1.1 Booting: /kick-282 console=ttyS0,9600n8nn quiet loader_ver= "2.1(2 Image verification OK Starting kernel... 2-24 Cisco MDS 9000 Family Troubleshooting Guide, Release 3.x OL-9285-05 Caution The loader> prompt is different from the required server. Wait..... If you ....1.2 Netmask: 255.255.255.0 Server: 0.0.0.0 Gateway: 0.0.0.0 Specify the IP address of commands available at this prompt and may result in that list. loader> ip address 172.16.1.2 255.255.255.0 Found Intel EtherExpressPro100 82559ER at this prompt or to run SETUP.... Step 2 Step 3 ...
Troubleshooting Guide
Page 84
...FCWA, NASB, SANTap, and ISAPI virtualization) Layer 3 Intelligent Storage Yes Services (such as FCWA, NASB, SANTap, ISAPI virtualization) only 1. Cisco MDS SAN-OS Release Module 2.0(1b) 2.0(2b), 2.0(3), 2.0(4), and 2.0(4a) 2.1(1a) 2.1(2) and later SSM None Fibre Channel switching Fibre ...Modules" section on page 3-9) and SSI boot image version 2.1(2). You can use dual fabric configuration to mdsfeedback-doc@cisco.com Table 3-1 lists the features supported for Fibre Channel switching traffic1 1. As shown in Table 3-2, Layer 3 Intelligent Storage Services traffic ...
...FCWA, NASB, SANTap, and ISAPI virtualization) Layer 3 Intelligent Storage Yes Services (such as FCWA, NASB, SANTap, ISAPI virtualization) only 1. Cisco MDS SAN-OS Release Module 2.0(1b) 2.0(2b), 2.0(3), 2.0(4), and 2.0(4a) 2.1(1a) 2.1(2) and later SSM None Fibre Channel switching Fibre ...Modules" section on page 3-9) and SSI boot image version 2.1(2). You can use dual fabric configuration to mdsfeedback-doc@cisco.com Table 3-1 lists the features supported for Fibre Channel switching traffic1 1. As shown in Table 3-2, Layer 3 Intelligent Storage Services traffic ...