The intention of VDisk copies is to protect servers against hardware failures. For this reason, it is important that the mirrors of the VDisk copies are stored on different storage systems. Here are some additional challenges in system administration and system monitoring.
This document describes how to ensure that all volumes are stored on different sites.
An wrong setup in this area can lead to an only 50% chance of surviving a fail over. If there is a mirrored pair on Site 1 and by an evil coincidence an another on Site 2 you have the 100% certainty that the fail over will not work as expected. So in worst cases nothing will work.
At the level of the server, the problem cannot be recognized at all and it is difficult to detect it using the SVC GUI.
In order to detect wrong configurations like this quickly I have developed a method in BVQ and stored it as a favorite.
click picture to enlarge
- (1) the alerts
- (2) the list of volumes that which are not compliant to the rules
- (3) The list of applications that are affected when the volumes can not fail over properly
- (4) The list of hosts that use at least one of these volumes, and can be affected of this in any way
We could extend this list with many volumes like capacity or performance
- (5) The list of rules in my installation as an overview
- (6) A Treemap, which shows in which applications these same hosts are used.
These applications are also on risk.
The red color of the application "Production" shows that this application owns a volume which fails
The red hosts in the other application just show that this application uses a host on risk.
- (7) This treemap represents the affected volumes graphically and helps in the orientation in the system.
These little red sports show that both volumes are located in two mirrored MDisk groups