Performance data files

The performance statistics of the IBM® Storage Virtualize are stored in the performance data files.

The IBM Storage Virtualize system collects performance statistics and stores them on the management node for a certain time, usually statistics are updated at 5 minutes interval which is configurable via the chsystem command and can be viewed using the lssystem command. The performance data files are accessible over the REST API endpoints. You can list and download the files by using a scripting language, for example Python or Linux® utility like cURL. The REST API accepts response type JSON (JavaScript Object Notation) which is a language-independent data format and has built-in support in various scripting languages. You can filter the JSON response from the REST API server on the client and enable the client end programs to remove any noise from the data.

Note: The following examples use Python programming language, and the code and variables that are declared in previous examples.

Authenticating

Acquire a token to complete operations on the system by using the REST API. To request an API token, make an HTTP POST call to /auth endpoint and provide a username and password in the request header. See the following example.

Request
import requests

url = "https://example.com:7443/rest/v1"
username = "myuser"
password = "mypassword"

headers = {
    "X-Auth-Username": username,
    "X-Auth-Password": password
}

response = requests.post(url + "/auth", headers=headers, verify=False)
token = response.json()['token']

response.json()
Response
{"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJpYXQiOjE3MDg1MjY4NzYsImV4cCI6MTcwODUzMDQ3NiwianRpIjoiZTEzMzQ4YmUzZGRkMWRjZTQ4YzhkYzE5Y2RhMTFjYzMiLCJzdiI6eyJ1c2VyIjoic3VwZXJ1c2VyIn19.I--KYnXUsFQ4wbtw_zmfkHrZ6V4bsWJgZA1j3WeHchw_kJVFRrPAhrjzJlYybIIDBYA05OSnmIZ9YWjpFY9uUQ"}

Listing files

The XML statistics files that contain the performance data are stored in /dumps/iostats/ directory on the config node. The /lsdumps API endpoint identifies the files that are accessible for download through the API. To list the contents of /dumps/iostats/, use the prefix parameter in the request body. See the following example.

Request
headers = {
    "X-Auth-Token": token,
}
data = {
    "prefix": "/dumps/iostats/"
}
response = requests.post(url + "/lsdumps", json=data, headers=headers, verify=False)
response.json()
Response
[{'id': '0', 'filename': 'Nm_stats_F304745-2_240221_151627'},
 {'id': '1', 'filename': 'Ng_stats_F304745-2_240221_151627'},
 {'id': '2', 'filename': 'Nn_stats_F304745-2_240221_151627'},
 {'id': '3', 'filename': 'Nv_stats_F304745-2_240221_151627'},
…
]

Filtering on file names

The system collects statistics over an interval and creates files that can be downloaded. For each collection interval, the system creates five statistics files:
  • Nm_stats for managed disks (MDisks)
  • Nv_stats for volumes and volume copies
  • Ng_stats for volume groups
  • Nn_stats for nodes
  • Nd_stats for drives

The file name consists of several parts, as shown in the following description:

<category>_<panel_name>_<yymmdd>_<hhmmss>

For example, Nv_stats_F304745-2_240221_151627

The naming convention enables a filtering capability that a client-side script might use to download:
  • Specific type of files
  • Files that were created at a specific time interval.
See the following example.
Request
import re
 
headers = {
    "X-Auth-Token": token,
}
data = {
    "prefix": "/dumps/iostats/"
}
response = requests.post(url + "/lsdumps", json=data, headers=headers, verify=False)
files = [row['filename'] for row in response.json()]

# Filter on category
nm_files = [name for name in files if re.match("Nm_stats", name)]
nv_files = [name for name in files if re.match("Nv_stats", name)]
ng_files = [name for name in files if re.match("Ng_stats", name)]
nn_files = [name for name in files if re.match("Nn_stats", name)]
nd_files = [name for name in files if re.match("Nd_stats", name)]
# Filter on time interval
tm_files = [name for name in files if re.match(".*_142127$", name)]
# Filter on panel name
pn_files = [name for name in files if re.match(".*F304745-2.*", name)]
# Make a selection
target_file = nm_files[0]
# Print
pn_files[:2]
Response
['Nm_stats_F304745-2_240222_143127', 'Nn_stats_F304745-2_240222_143127']

Downloading files

You can download the files that can be listed by using the /lsdumps API endpoint, through the /download endpoint. The /download interface accepts two parameters:
  • prefix: Name of the target directory
  • filename: Name of the target file

The contents of the downloaded file can be processed either in memory or stored on the host for post processing.

Note: The performance data files are constantly updated on the system. If an error occurs during the file download that means the file is rotated. Usually, you can fetch the latest list of files, followed by another download request. If not, reevaluate statistics intervals and collection intervals to ensure that the list and download is completed within the network and system limits.

See the following example.

Request
headers = {
    "X-Auth-Token": token,
}
data = {
    "prefix": "/dumps/iostats",
    "filename": target_file
}
response = requests.post(url + "/download", json=data, headers=headers, verify=False)

if response.status_code != 200:
    raise Exception("File not found")

response.content[:100]
Response
b'<?xml version="1.0" encoding="utf-8" ?>\n<diskStatsColl\nxmlns="http://ibm.com/storage/management/perf'