IBM Support

Troubleshoot Unexpected Reloads on Catalyst 9000 Switches

General Page

This guide explains how to troubleshoot unexpected reloads (or silent reload events) on Cisco Catalyst 9000 switches. It details what information to collect post-event to help identify the root cause.

Requirements and Components Used

  • Requirements: None specific to hardware/software versions.
  • Lab Environment: Information was tested with devices in a lab environment using default configurations. Exercise caution in a live network.

Background

An unexpected or silent reload can occur for many reasons. To determine the cause, you need the exact date/time of the event and must gather relevant logs and technical details as soon as the device reloads. This prevents critical log information from being overwritten.

Common Reload Reasons

Reload Reason Explanation Recommendation
PowerOn Power supply or inlet circuit was shut off and on (manually or due to a power event). If unexpected, verify physical power sources (power supplies, UPS, battery backups).
Stack Merge due to Incompatibility Standby switch lost communication with Active switch, prompting a new Active election. Check/replace stacking cables or modules.
Reload Command A user executed the reload command. Expected if command was intentional. If unexpected, check parser traces/logs.
LocalSoft or CPU Catastrophic Err Software/CPU error caused a forced reset. Gather logs, upgrade software if a bug is identified, and open a TAC case if needed.
Power failure or Unknown Sudden loss of power supply or circuit, causing a momentary outage. Check power supply LEDs, cables, UPS, circuit health; open a TAC case if no issues are found.
Redundancy force-switchover User ran redundancy force-switchover, causing the Standby to become Active. No action needed; this reload is expected.
Image Install Reload triggered by a new Cisco IOS® XE image installation. No action needed; this reload is expected.

Collect Relevant Data

  • Exact date/time of the occurrence.
  • What changed prior to the reload? (Configuration change, login, or idle?)
  • Check syslogs for any errors before the event (both on the switch and external Syslog servers).
  • Collect show tech-support output immediately.
  • Any other insights: power failures, environmental factors, etc.

Important: Gather information immediately after the reload to ensure logs are not overwritten.

Last Reload Reason

Use the following commands to see why the device reloaded and if there have been multiple reloads:

show version | include uptime|returned|reload|Version

Displays uptime, last reload reason, software version.

show logging onboard switch <id> uptime detail

Shows historical uptime data and past reset reasons.

Tech-Support Command

show tech-support gathers extensive system info. Methods to capture this data:

  • Terminal Session: Log the session to a text file, run show tech-support, then provide the file to TAC.
  • Redirect to a File:
    show tech-support | redirect flash:ShowTech.txt
    copy flash:ShowTech.txt tftp:
    
    Transfer the file via TFTP, FTP, SCP, etc.

Commands to Collect for TAC

  • Trace Logs: Internal logs of each Cisco IOS® XE process.
    • Generate via: request platform software trace archive
    • Produces a file (e.g., .tar.gz) in flash: or bootflash:
  • System Reports: Automatically generated compressed files when the switch crashes due to software errors.
    • Located in crashinfo: directory.
    • Named system-report_...
    • Encrypted and require TAC to decode.

Troubleshoot Scenarios

1. PowerOn

Cause: Switch was manually powered off/on or encountered a long power interruption.
Remediation:

  • Check if it was intentional.
  • Verify secure connection of power cables.
  • Inspect inlet circuit or UPS logs.
  • If no problem found, open a TAC case.

2. Stack Merge due to Incompatibility

Cause: Standby lost communication with Active, prompting a merge.
Remediation:

  • Ensure stacking cables/modules are seated properly.
  • Replace cables/modules if issue persists.
  • If still ongoing, open TAC case.

3. Reload Command

Cause: User executed reload.
Remediation: No direct fix—this is expected if intentional. Check logs if unexpected.

4. LocalSoft or CPU Catastrophic Err

Cause: Software/CPU error forced reset (e.g., bug CSCvr77861).
Remediation:

  • Check for known bugs and upgrade to a fixed version.
  • Gather trace logs and open TAC case if it recurs.
5. Power Failure or Unknown

Cause: Temporary power supply or circuit outage.
Remediation:

  • Check power supply LEDs and cables.
  • Check UPS logs.
  • Verify circuit stability (no voltage/current flaps).
  • If no issue found, open TAC case.
6. Redundancy Force-Switchover

Cause: redundancy force-switchover command was issued.
Remediation: No action required; this is expected.

7. Image Install

Cause: Reload as part of new Cisco IOS® XE image installation.
Remediation: No action required; this is expected.

8. Core File Present or Crash

Cause: Software crash generating core or system-report files.
Remediation:

  • Locate core or crash files in flash:core/ or crashinfo:.
  • Open a TAC case to have files analyzed.

Cisco Bug IDs

Some known bugs that cause unexpected reloads:

  • CSCvp56289: Silent reload with certain power supplies.
  • CSCvv27849: FED process caused unexpected reload.
  • CSCvr77861: Switch may reload with last reason “LocalSoft or CpuCatastrophicErr.”
  • CSCvu97189: Using show platform forward leads to device freeze/reload.
  • CSCwe11818: C9300L stack merge occurs after switchover.

Note: You must be a registered Cisco user to access internal bug details.

[{"Type":"MASTER","Line of Business":{"code":"LOB66","label":"Technology Lifecycle Services"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSWG8KZ","label":"MVS Network - CISCO"},"ARM Category":[{"code":"a8m3p000000Got7AAC","label":"MVS Network - CISCO-\u003ECisco Switches-\u003ECATALYST-\u003ECAT9200"},{"code":"a8m3p000000GosdAAC","label":"MVS Network - CISCO-\u003ECisco Switches-\u003ECATALYST-\u003ECAT9300"},{"code":"a8m3p000000GotBAAS","label":"MVS Network - CISCO-\u003ECisco Switches-\u003ECATALYST-\u003ECAT9400"},{"code":"a8m3p000000GotEAAS","label":"MVS Network - CISCO-\u003ECisco Switches-\u003ECATALYST-\u003ECAT9500"},{"code":"a8m3p000000GotCAAS","label":"MVS Network - CISCO-\u003ECisco Switches-\u003ECATALYST-\u003ECAT9600"}],"Platform":[{"code":"PF025","label":"Platform Independent"}]}]

Document Information

Modified date:
04 March 2025

UID

ibm17184774