The gmlinktolerance feature monitors the response times for Global Mirror relationships in
noncycling mode. You can use the chsystem command or the management GUI to set the gmlinktolerance feature.
The gmlinktolerance feature represents the number of seconds that the primary system tolerates slow
response times from the secondary system.
If the poor response extends past the specified tolerance, a 1920 error is logged. Also,
one or more Global Mirror relationships are automatically stopped to protect the application hosts
at the primary site. During normal operation, application hosts see a minimal impact to response
times because the Global Mirror feature uses asynchronous replication. However, if
Global Mirror operations experience
degraded response times from the secondary system for an extended time, I/O operations queue at the
primary system. This situation results in an extended response time to application hosts. In this
case, the gmlinktolerance feature stops Global Mirror relationships and the application hosts
response time returns to normal. After a 1920 error occurs, the Global Mirror auxiliary volumes are
no longer in the consistent_synchronized state until you fix the cause of the error and restart your
Global Mirror relationships. For this reason, ensure that you monitor the system to track when this
error occurs.
You can disable the gmlinktolerance feature by setting the gmlinktolerance value to
0 (zero). However, the gmlinktolerance feature cannot protect applications from extended response
times if it is unavailable. It might be appropriate to disable the gmlinktolerance feature in the
following circumstances:
- During periods when application hosts can tolerate extended response times, where it is expected
that the gmlinktolerance feature might stop the Global Mirror relationships. For example, if you are
testing by using an I/O generator that is configured to stress the backend storage, the
gmlinktolerance feature might detect the high latency and stop the Global Mirror relationships.
Disabling gmlinktolerance prevents this at the risk of exposing the test host to extended response
times.
Diagnosing and fixing 1920 errors
This error can be temporary (for example, a result of maintenance activity) or permanent
(for example, a result of a hardware failure or unexpected host I/O workload).
If the 1920 error was preceded by informational event 985004, Maximum replication delay
has been exceeded, the system might not find a path to the disk in the remote system within
the maximum replication delay timeout value. Investigate the remote system to find, and repair, any
degraded paths. You can also use the lssystem command to view the
maxreplicationdelay value. If the value is too low, use the
chsystem command to specify a new maxreplicationdelay
value.
For a
IBM® Spectrum Control system, the minimum interval is 5
minutes. If several 1920 errors occur, diagnose the cause of the earliest error first. The following
questions can help you determine the cause of the
error:
- Was maintenance occurring at the time of the error?
Maintenance might include replacing a storage system physical disk, updating the
firmware of the storage system, or
completing a code update on one of the
systems. Before you restart
the Global Mirror relationships
in noncycling mode, you must wait until the maintenance procedure is complete. Otherwise, another
1920 error is issued because the system has not yet returned to a stable state with good
performance.
- Were there any unfixed errors on either the source or target system?
If yes, analyze them to
determine whether they are the reason for the error. In particular, determine whether the errors
relate to the volume or MDisks that are being used in the relationship or if the errors reduced the
performance of the target system. Ensure that the errors are fixed before you restart the Global
Mirror relationship.
- Is the long-distance link overloaded?
If your link is not capable of sustaining the
short-term peak Global Mirror workload, a 1920 error can occur. Complete the following checks to
determine whether the long-distance link is overloaded:
- Look at the total Global Mirror auxiliary volume write throughput before the Global Mirror
relationships were stopped. If this volume is approximately equal to your link bandwidth, your link
might be overloaded. This issue might be due to application host I/O operations or a combination of
host I/O and background (synchronization) copy activities.
- Look at the total Global Mirror source volume write throughput before the Global Mirror
relationships were stopped. This value represents the I/O operations that are being completed by the
application hosts. If these operations are approaching the link's bandwidth,
reduce the I/O operations that the application is attempting to complete, or use Global Mirror to
copy fewer volumes. If the auxiliary disks show significantly more write I/O operations than the
source volumes, there is a high level of background copy. Decrease the Global Mirror
partnership's background copy rate parameter to bring the total application I/O bandwidth and
background copy rate within the link's capabilities.
- Look at the total Global Mirror source volume write throughput after the Global Mirror
relationships were stopped. If write throughput increases by 30% or more when the relationships are
stopped, the application hosts are attempting to complete more I/O operations than the link can
sustain. While the Global Mirror relationships are active, the overloaded link causes higher
response times to the application host, which decreases the throughput it can achieve. After the
Global Mirror relationships stop, the application host sees lower response times. In this case, the
link bandwidth must be increased, the application host I/O rate must be decreased, or fewer volumes
must be copied by using Global Mirror.
- Are the storage systems at the
secondary system overloaded?
If
application I/O operations cannot proceed at the rate that is needed by the application host because
one or more MDisks is providing poor service to the system, a 1920 error occurs.
If
the back-end
storage system requirements
were followed, the error might be due to a decrease in
storage system performance. A response time for an
individual MDisk that suddenly increased 50 ms or more or a response time above 100 ms indicates a
problem. Complete the following checks to determine whether the
storage systems are overloaded:
- Check the storage system for error
conditions such as media errors, a failed physical disk, or associated activity such as RAID
rebuilding. Fix any problems and then restart the Global Mirror relationships.
- If there is no error, determine whether the secondary storage system can process the required level
of application host I/O operations. It might be possible to improve the performance of the storage system by adding more physical disks
to an array, changing the RAID level of the array, changing the cache settings of the storage system, ensuring the cache battery is
operational, or changing other specific configuration parameters of the storage system.
- Are the storage systems at the
primary system overloaded?
Analyze the performance of the primary back-end storage by using the
same steps as for the secondary back-end storage. If performance is bad, limit the number of I/O
operations that can be completed by application hosts. Monitor the back-end storage at the primary
site even if the Global Mirror
relationships were not affected. If bad performance continues for a prolonged period, a 1920 error
occurs and the Global Mirror relationships are stopped.
- Is one of your systems overloaded?
If the total of these two statistics for either system is above 1 millisecond, the system might be
experiencing a high I/O load. Also, check the system node CPU utilization, as rates greater than 50%
can also contribute to the problem. In either case, contact your IBM service representative for further assistance.
- Do you have FlashCopy®
operations in the prepared state at the secondary system?
If the Global Mirror auxiliary volumes
are the sources of a FlashCopy mapping and that mapping
is in the prepared state for an extended time, performance to those volumes can be impacted because
the cache is disabled. Start the FlashCopy mapping to
enable the cache and improve performance for Global Mirror I/O operations.