IBM Support

Alert "1048 Unexpected enclosure fault", event ID 85157 "Unexpected encryption error" after reboot of an encrypted FlashSystem

Troubleshooting


Problem

After a reboot of an encrypted FlashSystem, a single flash card might fail to unlock during the encryption verification. Alert "1048 Unexpected enclosure fault", event ID 85157 (or 085157) "Unexpected encryption error" occurs and a restart of the encryption manager is required to retry the unlock operation.

Symptom

The following event is present in the log after the FlashSystem is rebooted or powered on:

Error Log Entry 38
 Node Identifier       : node1
 Object Type           : enclosure
 Object ID             : 1
 Error Count           : 1
 Error ID              : 85157 : Unexpected encryption error <<
 Error Code            : 1048 : Unexpected enclosure fault   <<


 One or more drives are shown as offline in the output of lsdrive:

  svcinfo lsdrive -delim :
  id status   use    tech_type  enclosure_id slot_id health_state encrypted
  0  online   member sas_ssd    1            1       good         yes      
  1  online   member sas_ssd    1            2       good         yes      
  2  online   member sas_ssd    1            3       good         yes      
  3  online   member sas_ssd    1            4       good         yes      
  4  online   member sas_ssd    1            5       good         yes      
  5  online   member sas_ssd    1            6       good         yes      
  6  online   member sas_ssd    1            7       good         yes      
  7  online   member sas_ssd    1            8       good         yes      
  8  online   member sas_ssd    1            9       good         yes      
  9  offline  member sas_ssd    1            10      good         no        
  10 offline  member sas_ssd    1            11      good         no        
  11 offline  spare  sas_ssd    1            12      good         no

The multiple offline flash cards cause the array to be offline:

  svcinfo lsarray -delim :
  mdisk_id mdisk_name status  mdisk_grp_id mdisk_grp_name capacity raid_status raid_level tier encrypt enclosure_id
  0        array0     offline 0            mdiskgrp0      52.0TB   offline     raid5      ssd  yes     1

Cause

The SSL handshake failed (errno - 11 SSL_get_error - 2) during the unlocking portion of the encryption verification. The system does not attempt to retry the handshake until the encryption manager is restarted (which is accomplished by another reboot).

Environment

Any FlashSystem storage enclosure at any firmware release level:

  • 840
  • 900
  • V9000
  • V840
  • A9000 and A9000R

Resolving The Problem

If the situation is as described, do not perform any actions against the offline flash cards (for example, running a certify operation or reseating them).

In the cases we saw with this pattern, another reboot brought back the encryption manager and cleared the situation. In an array offline situation, the system does not allow the shutdown without using the -force flag.

Action plan:

  1. Collect a snap option 4
  2. Perform another reboot of the FlashSystem enclosure via "stopsystem -reboot -force" (FlashSystem 840 / 900 / A9000 enclosure)
  • For V9000 use "stopsystem -reboot -enclosure <enclosure id> -force"

[{"Product":{"code":"STKMQB","label":"IBM FlashSystem 900"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":"Flash Module","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"STKMQB","label":"IBM FlashSystem 900"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"STKMQB","label":"IBM FlashSystem 900"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"STJKMM","label":"IBM FlashSystem A9000"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"STJKN5","label":"IBM FlashSystem A9000R"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"ST2HTZ","label":"IBM FlashSystem Software"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Product":{"code":"STKMQV","label":"IBM FlashSystem V9000"},"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]

Document Information

Modified date:
17 February 2023

UID

ssg1S1010029

Page Feedback