Troubleshooting Guide
Page 1
Cisco Nexus 5000 Series Troubleshooting Guide December 15, 2011 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883 Text Part Number: OL-25300-01 Send document comments to nexus5k-docfeedback@cisco.com.
Cisco Nexus 5000 Series Troubleshooting Guide December 15, 2011 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883 Text Part Number: OL-25300-01 Send document comments to nexus5k-docfeedback@cisco.com.
Troubleshooting Guide
Page 2
... FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. Third party trademarks mentioned are not intended to nexus5k-docfeedback@cisco.com. Cisco Nexus 5000 Series Troubleshooting Guide © 2011 Cisco Systems, Inc. All rights reserved.
... FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. Third party trademarks mentioned are not intended to nexus5k-docfeedback@cisco.com. Cisco Nexus 5000 Series Troubleshooting Guide © 2011 Cisco Systems, Inc. All rights reserved.
Troubleshooting Guide
Page 3
... Protocol 3-3 Multicast 3-5 VLANs 3-6 Registers and Counters 3-10 Troubleshooting QoS Issues 4-1 Policy Maps 4-1 Improper Configurations 4-2 PFC 4-8 Registers and Counters 4-10 Troubleshooting SAN Switching Issues 5-1 Overview 5-1 NPV 5-2 Zoning 5-7 SAN Port Channels 5-13 FC Services 5-16 Cisco Fabric Services 5-31 VSANs 5-40 Registers and Counters 5-48 Cisco Nexus 5000 Series Troubleshooting Guide iii Send document comments to nexus5k-docfeedback...
... Protocol 3-3 Multicast 3-5 VLANs 3-6 Registers and Counters 3-10 Troubleshooting QoS Issues 4-1 Policy Maps 4-1 Improper Configurations 4-2 PFC 4-8 Registers and Counters 4-10 Troubleshooting SAN Switching Issues 5-1 Overview 5-1 NPV 5-2 Zoning 5-7 SAN Port Channels 5-13 FC Services 5-16 Cisco Fabric Services 5-31 VSANs 5-40 Registers and Counters 5-48 Cisco Nexus 5000 Series Troubleshooting Guide iii Send document comments to nexus5k-docfeedback...
Troubleshooting Guide
Page 4
Troubleshooting Security Issues 6-1 Roles 6-1 AAA 6-4 Troubleshooting System Management Issues 7-1 SNMP 7-1 Logging 7-3 Traps 7-4 DNS 7-4 Troubleshooting Virtual Port Channel Issues 8-1 Improper Configurations 8-1 Troubleshooting Config-Sync Issues 9-1 Commit Failure 9-1 Import Failure 9-3 Merge Failure 9-5 Switch-profile Deletion Failure 9-6 Verify Failure 9-8 Cisco Nexus 5000 Series Troubleshooting Guide iv OL-25300-01 Contents Send document comments to nexus5k-docfeedback@cisco.com.
Troubleshooting Security Issues 6-1 Roles 6-1 AAA 6-4 Troubleshooting System Management Issues 7-1 SNMP 7-1 Logging 7-3 Traps 7-4 DNS 7-4 Troubleshooting Virtual Port Channel Issues 8-1 Improper Configurations 8-1 Troubleshooting Config-Sync Issues 9-1 Commit Failure 9-1 Import Failure 9-3 Merge Failure 9-5 Switch-profile Deletion Failure 9-6 Verify Failure 9-8 Cisco Nexus 5000 Series Troubleshooting Guide iv OL-25300-01 Contents Send document comments to nexus5k-docfeedback@cisco.com.
Troubleshooting Guide
Page 5
... resolve problems that can occur with Layer 2 switching in the Cisco Nexus 5000 Series switch. Troubleshooting QoS Issues Describes how to identify and resolve problems that may occur when configuring and using the Cisco Nexus 5000 Series switch. OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide v Preface This preface describes the audience, organization, and conventions of the Cisco Nexus 5000 Series Troubleshooting Guide, and how to nexus5k-docfeedback...
... resolve problems that can occur with Layer 2 switching in the Cisco Nexus 5000 Series switch. Troubleshooting QoS Issues Describes how to identify and resolve problems that may occur when configuring and using the Cisco Nexus 5000 Series switch. OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide v Preface This preface describes the audience, organization, and conventions of the Cisco Nexus 5000 Series Troubleshooting Guide, and how to nexus5k-docfeedback...
Troubleshooting Guide
Page 6
... Command descriptions use these conventions: Convention boldface font italic font [ ] [ x | y | z ] string Description Commands and keywords are grouped in the Cisco Nexus 5000 Series switch. Elements in the manual. Do not use quotation marks around the string or the string will help you solve a problem. This document uses the following ...with security in brackets and separated by vertical bars. Describes how to identify and resolve problems that can occur with system management and the Cisco Nexus 5000 Series switch. A nonquoted set of code indicates a comment line.
... Command descriptions use these conventions: Convention boldface font italic font [ ] [ x | y | z ] string Description Commands and keywords are grouped in the Cisco Nexus 5000 Series switch. Elements in the manual. Do not use quotation marks around the string or the string will help you solve a problem. This document uses the following ...with security in brackets and separated by vertical bars. Describes how to identify and resolve problems that can occur with system management and the Cisco Nexus 5000 Series switch. A nonquoted set of code indicates a comment line.
Troubleshooting Guide
Page 7
... are related Cisco Nexus 5000 Series and Cisco Nexus 2000 Series Fabric Extender documents: Release Notes Cisco Nexus 5000 Series and Cisco Nexus 2000 Series Release Notes Cisco Nexus 5000 Series Switch Release Notes Configuration Guides Cisco Nexus 5000 Series Configuration Limits for Cisco NX-OS Release 5.0(2)N1(1) Cisco Nexus 5000 Series Configuration Limits for Cisco NX-OS Release 4.2(1)N1(1) and Release 4.2(1)N2(1) Cisco Nexus 5000 Series NX-OS Fibre Channel over Ethernet Configuration Guide Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide Cisco Nexus 5000 Series NX-OS...
... are related Cisco Nexus 5000 Series and Cisco Nexus 2000 Series Fabric Extender documents: Release Notes Cisco Nexus 5000 Series and Cisco Nexus 2000 Series Release Notes Cisco Nexus 5000 Series Switch Release Notes Configuration Guides Cisco Nexus 5000 Series Configuration Limits for Cisco NX-OS Release 5.0(2)N1(1) Cisco Nexus 5000 Series Configuration Limits for Cisco NX-OS Release 4.2(1)N1(1) and Release 4.2(1)N2(1) Cisco Nexus 5000 Series NX-OS Fibre Channel over Ethernet Configuration Guide Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide Cisco Nexus 5000 Series NX-OS...
Troubleshooting Guide
Page 8
Preface Regulatory Compliance and Safety Information for the Cisco Nexus 5000 Series Switches and Cisco Nexus 2000 Series Fabric Extenders Licensing Guide Cisco NX-OS Licensing Guide Command References Cisco Nexus 5000 Series Command Reference Technical References Cisco Nexus 5000 Series and Cisco Nexus 2000 Series Fabric Extender MIBs Reference Error and System Messages Cisco NX-OS System Messages Reference Troubleshooting Guide Cisco Nexus 5000 Troubleshooting Guide Cisco Nexus 5000 Series Troubleshooting Guide viii OL-25300-01
Preface Regulatory Compliance and Safety Information for the Cisco Nexus 5000 Series Switches and Cisco Nexus 2000 Series Fabric Extenders Licensing Guide Cisco NX-OS Licensing Guide Command References Cisco Nexus 5000 Series Command Reference Technical References Cisco Nexus 5000 Series and Cisco Nexus 2000 Series Fabric Extender MIBs Reference Error and System Messages Cisco NX-OS System Messages Reference Troubleshooting Guide Cisco Nexus 5000 Troubleshooting Guide Cisco Nexus 5000 Series Troubleshooting Guide viii OL-25300-01
Troubleshooting Guide
Page 9
... 3 Gather information that defines the specific symptoms. Identify all potential problems that may occur when configuring and using the Cisco Nexus 5000 Series switch. To identify the possible problems, you determine the cause of the crash: • Use the show version or ... triggered due to potential problems. Troubleshooting a Switch Crash When a switch crashes, the cause might be causing the symptoms. Systematically eliminate each potential problem (from the failure of a process, and results in a reload of Reset OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-1
... 3 Gather information that defines the specific symptoms. Identify all potential problems that may occur when configuring and using the Cisco Nexus 5000 Series switch. To identify the possible problems, you determine the cause of the crash: • Use the show version or ... triggered due to potential problems. Troubleshooting a Switch Crash When a switch crashes, the cause might be causing the symptoms. Systematically eliminate each potential problem (from the failure of a process, and results in a reload of Reset OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-1
Troubleshooting Guide
Page 10
... show cores command for a Nexus 5000 switch.) switch#show cores Module-num Instance-num ---------- ------------ 1 1 Process-name -----------clis PID --4023 Core-create-time Nov 6 15:20 • Use the copy core://module-id/PID ftp: command to export the file and contact the TAC to determine if a core file was created. Cisco Nexus 5000 Series Troubleshooting Guide 1-2 OL-25300...
... show cores command for a Nexus 5000 switch.) switch#show cores Module-num Instance-num ---------- ------------ 1 1 Process-name -----------clis PID --4023 Core-create-time Nov 6 15:20 • Use the copy core://module-id/PID ftp: command to export the file and contact the TAC to determine if a core file was created. Cisco Nexus 5000 Series Troubleshooting Guide 1-2 OL-25300...
Troubleshooting Guide
Page 11
...practices are the recommended steps you should take to ensure the proper operation of your switch. • Maintain a consistent Cisco NX-OS release across all your Cisco Nexus 5000 switches. • Refer to the release notes for your Cisco SAN-OS release for the latest features, limitations, and caveats. • Enable... Bus Adapter N Port Virtualizer Virtual Network Link Fabric Extender Port Analyzer Adapter Reconfigure Fabric Request State Change Notification Cisco FCoE MUX ASIC Fibre Channel Protocol Fabric Shortest Path First OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-3
...practices are the recommended steps you should take to ensure the proper operation of your switch. • Maintain a consistent Cisco NX-OS release across all your Cisco Nexus 5000 switches. • Refer to the release notes for your Cisco SAN-OS release for the latest features, limitations, and caveats. • Enable... Bus Adapter N Port Virtualizer Virtual Network Link Fabric Extender Port Analyzer Adapter Reconfigure Fabric Request State Change Notification Cisco FCoE MUX ASIC Fibre Channel Protocol Fabric Shortest Path First OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-3
Troubleshooting Guide
Page 12
... interface fc2/4 interface Ethernet1/1 speed 1000 Displaying within the configuration mode, so there is required from the configuration switch# show logging | egrep Egrep grep Grep head Stream Editor last Display last lines less Stream Editor no need to back out...that are commonly used to the symptoms and possible problems covered in that match Cisco Nexus 5000 Series Troubleshooting Guide 1-4 OL-25300-01 The following chapters in this guide may use to nexus5k-docfeedback@cisco.com. Fabric Manager Tools and CLI Commands Chapter 1 Troubleshooting Overview Send document...
... interface fc2/4 interface Ethernet1/1 speed 1000 Displaying within the configuration mode, so there is required from the configuration switch# show logging | egrep Egrep grep Grep head Stream Editor last Display last lines less Stream Editor no need to back out...that are commonly used to the symptoms and possible problems covered in that match Cisco Nexus 5000 Series Troubleshooting Guide 1-4 OL-25300-01 The following chapters in this guide may use to nexus5k-docfeedback@cisco.com. Fabric Manager Tools and CLI Commands Chapter 1 Troubleshooting Overview Send document...
Troubleshooting Guide
Page 13
... Select source filesystem Select source filesystem Redirecting output NX-OS allows you to redirect outputs to files and flash areas in the switch. bootflash: core: debug: ftp: licenses log: modflash: nvram: running-config scp: sftp: startup-config system: tftp: ... SD no system default switchport shutdown Copy command switch# copy ? OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-5 Using the pipe command to only display required keyword switch# show tech-support details command to nexus5k-docfeedback@cisco.com. Chapter 1 Troubleshooting Overview Fabric Manager Tools...
... Select source filesystem Select source filesystem Redirecting output NX-OS allows you to redirect outputs to files and flash areas in the switch. bootflash: core: debug: ftp: licenses log: modflash: nvram: running-config scp: sftp: startup-config system: tftp: ... SD no system default switchport shutdown Copy command switch# copy ? OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-5 Using the pipe command to only display required keyword switch# show tech-support details command to nexus5k-docfeedback@cisco.com. Chapter 1 Troubleshooting Overview Fabric Manager Tools...
Troubleshooting Guide
Page 14
...In the following examples, the logging command and the Device Manager display severity information: Viewing Severity Information with the CLI switch(config)# show cli list | include debug | include interface Narrowing scope of a keyword. Copy the file from source... case difference when comparing strings -x Print only lines where the match is a whole line WORD Search for the expression switch# show logging Cisco Nexus 5000 Series Troubleshooting Guide 1-6 OL-25300-01 bootflash: Select destination filesystem debug: Select destination filesystem ftp: Select destination filesystem log: ...
...In the following examples, the logging command and the Device Manager display severity information: Viewing Severity Information with the CLI switch(config)# show cli list | include debug | include interface Narrowing scope of a keyword. Copy the file from source... case difference when comparing strings -x Print only lines where the match is a whole line WORD Search for the expression switch# show logging Cisco Nexus 5000 Series Troubleshooting Guide 1-6 OL-25300-01 bootflash: Select destination filesystem debug: Select destination filesystem ftp: Select destination filesystem log: ...
Troubleshooting Guide
Page 15
... via this tool. Ethanalyzer Ethanalyzer is a command-line version of Wireshark that captures and decodes packets. OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-7 SPAN is a tool that collects frames that are transient to the switch are destined to a second port for analysis. debugging Size - 4194304 Viewing Severity Levels in the Device Manager Ethanalyzer...
... via this tool. Ethanalyzer Ethanalyzer is a command-line version of Wireshark that captures and decodes packets. OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-7 SPAN is a tool that collects frames that are transient to the switch are destined to a second port for analysis. debugging Size - 4194304 Viewing Severity Levels in the Device Manager Ethanalyzer...
Troubleshooting Guide
Page 16
...Captures packets sent or received by the supervisor and provides a summary of NX-OS Ethanalyzer. Examples switch# ethanalyzer local sniff-interface No matches in current mode, matching in the command captures output until you... comments to a file. ethanalyzer local sniff-interface decode-internal Decodes the internal frame header for switch domain controller. switch# ethanalyzer local sniff-interface inbound-hi brief limit-captured-frames 0 Capturing on eth0 2008-08-...:0d:ec:6b:cd:41 Cost = 0 Port = 0x8093 [snip] Cisco Nexus 5000 Series Troubleshooting Guide 1-8 OL-25300-01
...Captures packets sent or received by the supervisor and provides a summary of NX-OS Ethanalyzer. Examples switch# ethanalyzer local sniff-interface No matches in current mode, matching in the command captures output until you... comments to a file. ethanalyzer local sniff-interface decode-internal Decodes the internal frame header for switch domain controller. switch# ethanalyzer local sniff-interface inbound-hi brief limit-captured-frames 0 Capturing on eth0 2008-08-...:0d:ec:6b:cd:41 Cost = 0 Port = 0x8093 [snip] Cisco Nexus 5000 Series Troubleshooting Guide 1-8 OL-25300-01
Troubleshooting Guide
Page 17
..., or other Remote Monitoring (RMON) probes. You can choose the SPAN traffic in a SPAN session. Note For the Cisco Nexus 5548 Switch, Fibre Channel ports cannot be configured as ingress source ports in the ingress direction, the egress direction, or both directions ...port is marked: False] [Protocols in the specified VLAN or VSAN are included as SPAN sources. OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-9 The Cisco Nexus 5000 Series switch supports Ethernet, virtual Ethernet, Fibre Channel, virtual Fibre Channel, port channels, SAN port channels, VLANs, and VSANs ...
..., or other Remote Monitoring (RMON) probes. You can choose the SPAN traffic in a SPAN session. Note For the Cisco Nexus 5548 Switch, Fibre Channel ports cannot be configured as ingress source ports in the ingress direction, the egress direction, or both directions ...port is marked: False] [Protocols in the specified VLAN or VSAN are included as SPAN sources. OL-25300-01 Cisco Nexus 5000 Series Troubleshooting Guide 1-9 The Cisco Nexus 5000 Series switch supports Ethernet, virtual Ethernet, Fibre Channel, virtual Fibre Channel, port channels, SAN port channels, VLANs, and VSANs ...
Troubleshooting Guide
Page 18
...as ingress or egress source ports. • Source ports can be configured with Cisco NX-OS Release 5.0(2)N1(1). SPAN Destinations SPAN destinations refer to monitor. The Cisco Nexus 5000 Series switch supports Ethernet and Fibre Channel interfaces as source ports. • The Cisco Nexus 5010 switch supports a maximum of any port type: Ethernet, virtual Ethernet, Fibre Channel, virtual... Channel ports cannot be destination ports. • Cannot be a source port. • Cannot be a port channel or SAN port channel group. 1-10 Cisco Nexus 5000 Series Troubleshooting Guide OL-25300-01
...as ingress or egress source ports. • Source ports can be configured with Cisco NX-OS Release 5.0(2)N1(1). SPAN Destinations SPAN destinations refer to monitor. The Cisco Nexus 5000 Series switch supports Ethernet and Fibre Channel interfaces as source ports. • The Cisco Nexus 5010 switch supports a maximum of any port type: Ethernet, virtual Ethernet, Fibre Channel, virtual... Channel ports cannot be destination ports. • Cannot be a source port. • Cannot be a port channel or SAN port channel group. 1-10 Cisco Nexus 5000 Series Troubleshooting Guide OL-25300-01
Troubleshooting Guide
Page 19
...switch# show monitor session SESSION STATE REASON 1 up The session is not monitored if it can affect traffic forwarding on the monitor source with a 0 VLAN tag (priority tag). • For Ethernet destination, the monitor session is up DESCRIPTION OL-25300-01 Cisco Nexus 5000 Series... Troubleshooting Guide 1-11 If a destination port is oversubscribed, it belongs to nexus5k-docfeedback@cisco.com. • Does not participate in on one destination port can be a...
...switch# show monitor session SESSION STATE REASON 1 up The session is not monitored if it can affect traffic forwarding on the monitor source with a 0 VLAN tag (priority tag). • For Ethernet destination, the monitor session is up DESCRIPTION OL-25300-01 Cisco Nexus 5000 Series... Troubleshooting Guide 1-11 If a destination port is oversubscribed, it belongs to nexus5k-docfeedback@cisco.com. • Does not participate in on one destination port can be a...
Troubleshooting Guide
Page 20
... ports : fc3/2 Debugging Command-Line Debugging Available debugs depend on debugs. Determine the destination of the debug file. 1-12 Cisco Nexus 5000 Series Troubleshooting Guide OL-25300-01 There are many different options to screen via console, Telnet, or SSH. Fabric Manager Tools and...Send document comments to run from the CLI. You must have administrator privileges to nexus5k-docfeedback@cisco.com. Debug Logging Set the log file as CiscoLive_debugs, using the debug logfile command. switch# show monitor session 1 session 1 type : local state : up source intf : rx...
... ports : fc3/2 Debugging Command-Line Debugging Available debugs depend on debugs. Determine the destination of the debug file. 1-12 Cisco Nexus 5000 Series Troubleshooting Guide OL-25300-01 There are many different options to screen via console, Telnet, or SSH. Fabric Manager Tools and...Send document comments to run from the CLI. You must have administrator privileges to nexus5k-docfeedback@cisco.com. Debug Logging Set the log file as CiscoLive_debugs, using the debug logfile command. switch# show monitor session 1 session 1 type : local state : up source intf : rx...