White Papers
Abstract
The AIX Multipath I/O (MPIO) framework manages multiple paths to storage devices, ensuring redundancy and high availability.
This document explains the Closed Path Recovery feature and further describes how to
- Initiate automatic path recovery and obtain up-to-date disk path status without keeping disks open for extended period.
- Control the feature dynamically.
- Understand its behavior and resource implications.
Content
Traditionally, AIX PCM performs path health checks and subsequent path status updates only when a disk is actively in use - referred to as being "open" by an application or process - and the disk's hcheck_interval attribute value is greater than zero.
When the disk is no longer in use (i.e., "closed"), health checks and path status updates cease, and tools like lsmpio and lspath report the last known state from when the disk was closed. This can lead to outdated information.
For example:
- a path that fails while the disk is open but recovers after closure remains reported as "Failed" until the disk is reopened or path is reconfigured.
- a path that fails while the disk is closed remains reported as "Failed" until the disk is reopened or path is reconfigured.
Closed Path Recovery Feature
Introduced in AIX 7.2 TL4, the Closed Path Recovery feature enhances AIX MPIO by providing up-to-date disk path status reporting without requiring disks to remain in open state.
The feature is disabled by default to maintain system efficiency and prevent unnecessary processing overhead from inactive disks.
When enabled, it periodically attempts to recover paths marked as "Failed" even when the associated disk is closed and health checks cannot be performed.
The dynamic control and on-demand updates via lsmpio -o command, offer flexibility and improved visibility over the AIX MPIO devices path status.
The dynamic control and on-demand updates via lsmpio -o command, offer flexibility and improved visibility over the AIX MPIO devices path status.
Third-Party PCM Support
Closed Path Recovery feature is fully functional only with AIX PCM unless third-party PCMs explicitly support it.
Please check with your storage vendor for compatibility.
Enabling The Closed Path Recovery Feature
The Closed Path Recovery is controlled via the dk_closed_path_recovery tunable, managed through the ioo utility.
The tunable allows dynamic enabling or disabling of the feature without requiring a system reboot.
ioo -o dk_closed_path_recovery Displays the current value (0 or 1)
ioo -o dk_closed_path_recovery=0 Disables the feature (default setting)
ioo -o dk_closed_path_recovery=1 Enables the feature
With dk_closed_path_recovery=1, attempts are made every 60 seconds to recover the closed paths that are in "Failed" state. However, disks that were already closed at the time when the feature is enabled won’t benefit until they are reopened (i.e., using the "lsmpio -o" command).
In addition, "dk_closed_path_recovery" can be modified using the System Management Interface Tool (SMIT) panel as shown in the code block below:
smitty ChShCurIO
Performance & Resource Scheduling
Tuning Kernel & Network Parameters
Tuning IO Parameters
Change / Show Current Parameters
Usage Considerations
The table below compares the behavior and implications of the "dk_closed_path_recovery" tunable set to "0" versus "1" on closed path recovery, as well as the resource utilization.
| Aspect | dk_closed_path_recovery=0 | dk_closed_path_recovery=1 |
|---|---|---|
| Expected Behavior |
No recovery occurs for closed disks.
Path status reflects the last known state from disk closure.
|
Background attempts to recover failed paths occur every 60 seconds for closed disks, but only for those open at the enablement or opened later. |
| Resource Implications and Memory Usage |
No noticeable performance overhead.
|
Additional 20 KB for 2000 disks, regardless of failures. Negligible in healthy path environment. The overhead is structural and not directly tied to the tunable’s state, though it’s only relevant when failures occur. Slight increases in memory use during path failures. Adds minor overhead: Up to 320 KB for 2000 disks with 16 paths each (only when failures occur). |
In conclusion, memory overhead is minimal in a stable environment and increases only during path failures.
On-Demand Path Status Updates
The lsmpio -o command provides an alternative way to refresh path statuses.
Syntax: lsmpio -o -l hdisk# (e.g., lsmpio -o -l hdisk0)
Behavior: Opens the specified disk, refreshes all path states, closes the disk, and displays the updated status. If the operation fails, it shows a warning with the last-known states.
Caution: In large environments with many paths or disks, "lsmpio -o" response time ranges from seconds to minutes, depending on the disk count, command scope and the IO failure conditions. Use this command carefully to avoid delays.
For more details, please see lsmpio man page.
Usage Example
In the following example, hdisk1 is configured with four paths, split across two parent controllers.
All paths are in a closed (Clo) state, indicating they are not actively in use for I/O operations.
All paths are in a closed (Clo) state, indicating they are not actively in use for I/O operations.
# lsmpio -l hdisk1
name path_id status path_status parent connection
hdisk1 0 Enabled Clo fscsi0 50050768102xxxxx,x000000000000
hdisk1 1 Enabled Clo fscsi0 50050768101xxxxx,x000000000000
hdisk1 2 Enabled Clo fscsi1 50050768102xxxxx,x000000000000
hdisk1 3 Enabled Clo fscsi1 50050768101xxxxx,x000000000000
The Closed Path Recovery is disabled by default:
# ioo -o dk_closed_path_recovery
dk_closed_path_recovery = 0
A connectivity issue occurred on the SAN, resulting in two of the paths transitioning to a "Failed" state:
# lsmpio -l hdisk1
name path_id status path_status parent connection
hdisk1 0 Failed Clo fscsi0 50050768102xxxxx,x000000000000
hdisk1 1 Enabled Clo fscsi0 50050768101xxxxx,x000000000000
hdisk1 2 Failed Clo fscsi1 50050768102xxxxx,x000000000000
hdisk1 3 Enabled Clo fscsi1 50050768101xxxxx,x000000000000
The "dk_closed_path_recovery" is enabled using the "ioo" utility and the change is logged in the system error report as seen in the summ output:
# ioo -o dk_closed_path_recovery=1
Setting dk_closed_path_recovery to 1
# errpt -a|summ
Mar 7 12:46:05 closed_path_rec I SC_MPIO_INFO1 MPIO Tunable changed successfully
However, the paths were in a "Clo" state when the feature was enabled.
As a result, after the connectivity problem resolve, the paths continue to report as "Failed":
As a result, after the connectivity problem resolve, the paths continue to report as "Failed":
# lsmpio -l hdisk1
name path_id status path_status parent connection
hdisk1 0 Failed Clo fscsi0 50050768102xxxxx,x000000000000
hdisk1 1 Enabled Clo fscsi0 50050768101xxxxx,x000000000000
hdisk1 2 Failed Clo fscsi1 50050768102xxxxx,x000000000000
hdisk1 3 Enabled Clo fscsi1 50050768101xxxxx,x000000000000
In this scenario, the "lsmpio -o" command can be executed to manually open the disk paths and update the status.
# lsmpio -o
name path_id status path_status parent connection
hdisk1 0 Enabled Clo fscsi0 50050768102xxxxx,x000000000000
hdisk1 1 Enabled Clo fscsi0 50050768101xxxxx,x000000000000
hdisk1 2 Enabled Clo fscsi1 50050768102xxxxx,x000000000000
hdisk1 3 Enabled Clo fscsi1 50050768101xxxxx,x000000000000
Opening the paths also enabled the automatic recovery(dk_closed_path_recovery), ensuring paths will get updated in future.
For further details, please refer to the IBM Documentation or contact IBM Support.
Recommended Documentation
[{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"ARM Category":[{"code":"a8m0z000000cw0FAAQ","label":"IO Device Drivers"}],"Platform":[{"code":"PF002","label":"AIX"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB57","label":"Power"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSPHKW","label":"PowerVM Virtual I\/O Server"},"ARM Category":[{"code":"a8mKe000000TN3DIAW","label":"MPIO"}],"Platform":[{"code":"PF002","label":"AIX"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
21 March 2025
UID
ibm17184977