Vdisk set definition sizing and preview
Defining vdisk sets before actually creating the member vdisk NSDs permits the IBM Storage Scale administrator to preview the declustered array size and server memory requirements of the vdisk sets.
The mmvdisk command refuses to define a vdisk set that exceeds declustered array disk capacity in a recovery group or server memory capacity in the recovery group server node class.
Newly created recovery groups have no vdisk sets. To see what capacity the newly created recovery groups have for defining vdisk sets, use mmvdisk vdiskset list --recovery-group all. This command shows all recovery group declustered array capacities where vdisk sets can be defined, and the server memory capacities for the recovery group server node classes.
In this example, the two recovery groups, which are newly created, have 100% of their raw
declustered array capacity free. Their servers are using a baseline 386 MiB of 30 GiB available
memory for internal server vdisk set
maps.
# mmvdisk recoverygroup list
needs user
recovery group active current or master server service vdisks remarks
-------------- ------- -------------------------------- ------- ------ -------
BB01L yes server01.gpfs.net no 0
BB01R yes server02.gpfs.net no 0
# mmvdisk vdiskset list --recovery-group all
declustered capacity all vdisk sets defined
recovery group array total raw free raw free% in the declustered array
-------------- ----------- --------- -------- ----- ------------------------
BB01L DA1 36 TiB 36 TiB 100% -
BB01R DA1 36 TiB 36 TiB 100% -
vdisk set map memory per server
node class available required required per vdisk set
---------- --------- -------- ----------------------
BB01 30 GiB 386 MiB -
#As vdisk sets are defined in these recovery groups, mmvdisk shows the cumulative claim made by the vdisk sets upon the declustered array capacity and server memory resources. It does so without actually having to create the vdisk NSDs.
The next example shows the accumulating declustered array space and memory claims associated with
the definition of three vdisk sets in the BB01L and BB01R recovery groups and the BB01 server node
class.
# mmvdisk vdiskset define --vdisk-set fs1data --recovery-group BB01L,BB01R --set-size 45% \
--code 8+3p --block-size 2m --nsd-usage dataOnly --storage-pool data
mmvdisk: Vdisk set 'fs1data' has been defined.
mmvdisk: Recovery group 'BB01L' has been defined in vdisk set 'fs1data'.
mmvdisk: Recovery group 'BB01R' has been defined in vdisk set 'fs1data'.
member vdisks
vdisk set count size raw size created file system and attributes
-------------- ----- -------- -------- ------- --------------------------
fs1data 2 11 TiB 16 TiB no -, DA1, 8+3p, 2 MiB, dataOnly, data
declustered capacity all vdisk sets defined
recovery group array total raw free raw free% in the declustered array
-------------- ----------- --------- -------- ----- ------------------------
BB01L DA1 36 TiB 20 TiB 55% fs1data
BB01R DA1 36 TiB 20 TiB 55% fs1data
vdisk set map memory per server
node class available required required per vdisk set
---------- --------- -------- ----------------------
BB01 30 GiB 869 MiB fs1data (483 MiB)
# mmvdisk vdiskset define --vdisk-set fs1meta --recovery-group BB01L,BB01R --set-size 5% \
--code 4wayReplication --block-size 512k --nsd-usage metadataOnly
mmvdisk: Vdisk set 'fs1meta' has been defined.
mmvdisk: Recovery group 'BB01L' has been defined in vdisk set 'fs1meta'.
mmvdisk: Recovery group 'BB01R' has been defined in vdisk set 'fs1meta'.
member vdisks
vdisk set count size raw size created file system and attributes
-------------- ----- -------- -------- ------- --------------------------
fs1meta 2 463 GiB 1872 GiB no -, DA1, 4WayReplication, 512 KiB, metadataOnly, system
declustered capacity all vdisk sets defined
recovery group array total raw free raw free% in the declustered array
-------------- ----------- --------- -------- ----- ------------------------
BB01L DA1 36 TiB 18 TiB 50% fs1data, fs1meta
BB01R DA1 36 TiB 18 TiB 50% fs1data, fs1meta
vdisk set map memory per server
node class available required required per vdisk set
---------- --------- -------- ----------------------
BB01 30 GiB 908 MiB fs1data (483 MiB), fs1meta (38 MiB)
# mmvdisk vdiskset define --vdisk-set fs2 --recovery-group BB01L,BB01R --set-size 50% \
--code 8+3p --block-size 2m
mmvdisk: Vdisk set 'fs2' has been defined.
mmvdisk: Recovery group 'BB01L' has been defined in vdisk set 'fs2'.
mmvdisk: Recovery group 'BB01R' has been defined in vdisk set 'fs2'.
member vdisks
vdisk set count size raw size created file system and attributes
-------------- ----- -------- -------- ------- --------------------------
fs2 2 13 TiB 18 TiB no -, DA1, 8+3p, 2 MiB, dataAndMetadata, system
declustered capacity all vdisk sets defined
recovery group array total raw free raw free% in the declustered array
-------------- ----------- --------- -------- ----- ------------------------
BB01L DA1 36 TiB 4096 MiB 0% fs1data, fs1meta, fs2
BB01R DA1 36 TiB 4096 MiB 0% fs1data, fs1meta, fs2
vdisk set map memory per server
node class available required required per vdisk set
---------- --------- -------- ----------------------
BB01 30 GiB 1445 MiB fs1data (483 MiB), fs1meta (38 MiB), fs2 (537 MiB)
At this point, 100% of the disk capacity in DA1 has been claimed by these vdisk set definitions.
The server memory demand is well under the available limit. None of the vdisk sets have been created
yet. A vdisk set is only created when all of the expected vdisk NSDs in all the vdisk set's recovery
groups exist in reality and not merely as a claim upon resource. If none, or only some but not all,
of the expected vdisk NSDs actually exist, the value in the created column for a
vdisk set is no.
Assuming these definitions are what is desired, the next example shows actually creating these
three vdisk
sets.
# mmvdisk vdiskset create --vdisk-set all
mmvdisk: 2 vdisks and 2 NSDs will be created in vdisk set 'fs2'.
mmvdisk: 2 vdisks and 2 NSDs will be created in vdisk set 'fs1data'.
mmvdisk: 2 vdisks and 2 NSDs will be created in vdisk set 'fs1meta'.
mmvdisk: (mmcrvdisk) [I] Processing vdisk RG001VS003
mmvdisk: (mmcrvdisk) [I] Processing vdisk RG002VS003
mmvdisk: (mmcrvdisk) [I] Processing vdisk RG002VS001
mmvdisk: (mmcrvdisk) [I] Processing vdisk RG001VS001
mmvdisk: (mmcrvdisk) [I] Processing vdisk RG002VS002
mmvdisk: (mmcrvdisk) [I] Processing vdisk RG001VS002
mmvdisk: Created all vdisks in vdisk set 'fs2'.
mmvdisk: Created all vdisks in vdisk set 'fs1data'.
mmvdisk: Created all vdisks in vdisk set 'fs1meta'.
mmvdisk: (mmcrnsd) Processing disk RG001VS003
mmvdisk: (mmcrnsd) Processing disk RG002VS003
mmvdisk: (mmcrnsd) Processing disk RG001VS001
mmvdisk: (mmcrnsd) Processing disk RG002VS001
mmvdisk: (mmcrnsd) Processing disk RG001VS002
mmvdisk: (mmcrnsd) Processing disk RG002VS002
mmvdisk: Created all NSDs in vdisk set 'fs2'.
mmvdisk: Created all NSDs in vdisk set 'fs1data'.
mmvdisk: Created all NSDs in vdisk set 'fs1meta'.
# mmvdisk vdiskset list --vdisk-set all
member vdisks
vdisk set count size raw size created file system and attributes
-------------- ----- -------- -------- ------- --------------------------
fs1data 2 11 TiB 16 TiB yes -, DA1, 8+3p, 2 MiB, dataOnly, data
fs1meta 2 463 GiB 1872 GiB yes -, DA1, 4WayReplication, 512 KiB, metadataOnly, system
fs2 2 13 TiB 18 TiB yes -, DA1, 8+3p, 2 MiB, dataAndMetadata, system
declustered capacity all vdisk sets defined
recovery group array total raw free raw free% in the declustered array
-------------- ----------- --------- -------- ----- ------------------------
BB01L DA1 36 TiB 4096 MiB 0% fs1data, fs1meta, fs2
BB01R DA1 36 TiB 4096 MiB 0% fs1data, fs1meta, fs2
vdisk set map memory per server
node class available required required per vdisk set
---------- --------- -------- ----------------------
BB01 30 GiB 1445 MiB fs1data (483 MiB), fs1meta (38 MiB), fs2 (537 MiB)
#The mmvdisk vdiskset create command shows the creation of the individual
member vdisk NSDs in which the automatically generated names of the individual member vdisks and
NSDs are seen. The names of individual vdisk NSDs are generally unimportant since normal
mmvdisk usage manages them collectively as vdisk set members.
Note: If it is ever
required to know the details of individual member vdisk NSDs, the mmvdisk vdisk
list and mmvdisk recoverygroup list --vdisk commands can be
used.
After vdisk creation, when the vdisk sets are listed again, the
created column
shows yes for all three vdisk sets. The actual member vdisk NSDs have
been created and are ready to be used to create file
systems.
# mmvdisk filesystem create --file-system fs1 --vdisk-set fs1meta,fs1data
# mmvdisk filesystem create --file-system fs2 --vdisk-set fs2After file system creation, listing the vdisk sets shows the file system device name with the
vdisk set
attributes.
# mmvdisk vdiskset list --vdisk-set all
member vdisks
vdisk set count size raw size created file system and attributes
-------------- ----- -------- -------- ------- --------------------------
fs1data 2 11 TiB 16 TiB yes fs1, DA1, 8+3p, 2 MiB, dataOnly, data
fs1meta 2 463 GiB 1872 GiB yes fs1, DA1, 4WayReplication, 512 KiB, metadataOnly, system
fs2 2 13 TiB 18 TiB yes fs2, DA1, 8+3p, 2 MiB, dataAndMetadata, system
...