Configuring a mixed cluster and file system

Use these instructions for setting up IBM Storage Scale System 3000 and IBM Storage Scale System 5000 mixed cluster and file system.

The following high-level tasks need to be done for setting up IBM Storage Scale System 3000 and IBM Storage Scale System 5000 mixed cluster:

Deploy an IBM Storage Scale System 3000 system (including cluster, file system, GUI).
Deploy an IBM Storage Scale System 5000 system (adding to cluster, create recovery groups, and so on).
Create the IBM Storage Scale System 5000 vdisks and add to the existing file system.
Create a policy file.
Adjust sensors.
Add IBM Storage Scale System 5000 nodes to the GUI.

Note: These instructions contain summarized steps and references to documents that cover the items in more detail. The goal is to give an example scenario to help clients understand aspects of this procedure. At the end of this procedure, if you have Power9 protocol nodes, for guidance in implementing them into your environment, see Creating a shared root file system for CES.

Prerequisites

SSR has completed Code 20 on both the IBM Storage Scale System 3000 and IBM Storage Scale System 5000 nodes (including management server)
SSR works on Power® nodes and the management server node first, then the IBM Storage Scale System 3000 system.
Public connection setup on C11-T3 (f3 connection on management server)
IBM Storage Scale System 3000 and IBM Storage Scale System 5000 nodes are added to the /etc/hosts file.
- Low-speed names FQDNs , short names, and IP addresses
- High-speed names FQDNs, short names, and IP addresses (add suffix of low-speed names)
Host name and domain set on management server.
Latest code for IBM Storage Scale System 3000 and IBM Storage Scale System 5000 stored in the /home/deploy on the management server.

For information about how to deploy the IBM Storage Scale System system, see IBM Storage Scale System: Quick Deployment Guide.
For information about how to use the mmvdisk command, see mmvdisk in IBM Storage Scale System documentation.

Adding IBM Storage Scale System 3000 building blocks

Extract the IBM Storage Scale System 3000 installation package.
```
tar zxvf ESS3000InstallationPackage
```
Accept the license and deploy the container.
```
./ess_7.0.1.x_0604-21_dme_ppc64le
```

After logging in to the container, complete the following steps:

Run the config load command.

essrun -N ESS3000Node1,ESS3000Node2,EMSNode config load -p

Note: Use the low-speed names.

If required, update the management server node.
```
essrun -N EMSNode update --offline
```

Update the IBM Storage Scale System 3000 nodes.

essrun -N ESS3000Node1,ESS3000Node2 update --offline

Create network bonds.

essrun -N ESS3000Node1,ESS3000Node2,EMSNode network --suffix=Suffix

Create a cluster.
```
essrun -N ESS3000NodeGroup cluster --suffix=Suffix
```
Note: To obtain the group name, use -t group.

Add the management server node to the cluster.

essrun -N ESS3000Node1 cluster --suffix=Suffix --add-ems EMSNode

Create a file system.
```
essrun -N ESS3000NodeGroup filesystem --suffix=Suffix
```
Note: This command creates a combined data and metadata vdisk in the system pool. The file system name must be fs3k.

Type exit and press Enter to exit the container. Proceed with the instructions on how to setup the collector, sensors, and run the GUI wizard.

The status of the current IBM Storage Scale System 3000 container should be exited. To confirm, use the podman ps -a command. For example: Start of change

# podman container attach cems0
ESS 3000 CONTAINER root@cems0:/ # exit
exit
# podman ps -a
CONTAINER ID  IMAGE                                       COMMAND               CREATED     STATUS                    PORTS  NAMES
bfc2980bdfc9  localhost/ess_7.0.1.x_dme_xcat:0604-21  /myStartupScript....  7 days ago  Exited (0) 2 seconds ago         cems0

If the IBM Storage Scale System 3000 container is not in the stopped state, use the podman stop ContainerName command.

Adding IBM Storage Scale System 5000

Extract the IBM Storage Scale System 5000 installation package.
```
tar xvf ESS5000InstallationPackage
```
Verify the installation package.
```
sha256sum -c Extractedsha256sumFile
```
Accept the license and deploy the container
```
./ess_7.0.1.x_0604-21_dme_ppc64le7.0.1.x
```

After you have logged into the container, complete the following steps:

Run the config load command.
```
essrun -N ESS5000Node1,ESS5000Node2,ESS3000Node1,ESS3000Node2,EMSNode config load -p
```
Note: If you plan to add protocol nodes in the cluster, include them in the list of nodes that you are specifying in this command.

Update the nodes.

essrun -N ESS5000Node1,ESS5000Node2 update --offline

Create network bonds.

essrun -N ESS5000Node1,ESS5000Node2 network --suffix=Suffix

Add the IBM Storage Scale System 5000 nodes to an existing cluster.
1. Use SSH to connect to one of the IBM Storage Scale System 5000 I/O server nodes. For example:
```
ssh ESS5000Node1
```
2. Accept the license. Use high-speed names.
```
essaddnode -N ESS5000Node1-hs,ESS5000Node2-hs --cluster-node ESS3000Node-hs --nodetype ess5k --accept-license
```
  Where:
  ESS5000Node1-hs,ESS5000Node2-hs, ESS3000Node-hs
  
  Are high-speed names.
  
  --nodetype
  You can add the following nodes to any environment:
  
  For management server node, use ems.
  
  For IBM Storage Scale System 3000, use ess3k.
  
  For IBM Storage Scale System 5000, use ess5k.
  
  For IBM Storage Scale System 3200, use ess3200.
  
  Default is ems.
  If there is an error, you might need to log in to each IBM Storage Scale System 5000 node and start GPFS.
```
mmbuildgpl mmstartup
```
  Type exit and press Enter to exit the container. Running these commands, takes you to the IBM Storage Scale System 5000 node.
Create mmvdisk artifacts.
1. Create a node class.
```
mmvdisk nc create --node-class ess5k_ppc64le_mmvdisk -N  ListOfESS5000Nodes_highspeedsuffix
```
2. Configure the node class.
```
mmvdisk server configure --nc ess5k_ppc64le_mmvdisk --recycle one
```
3. Create recovery groups.
```
mmvdisk rg create --rg ess5k_rg1,ess5k_rg2 --nc ess5k_ppc64le_mmvdisk
```
4. Define vdiskset.
```
mmvdisk vs define --vs vs_fs5k_1 --rg ess5k_rg1,ess5k_rg2 code 8+2p --bs 16M --ss 80% --nsd-usage dataOnly --sp data
```
5. Create a vdiskset.
```
mmvdisk vs create --vs vs_fs5k_1
```
6. Add the vdiskset to the file system.
```
mmvdisk fs add --file-system fs3k --vdisk-set vs_fs5k_1
```
7. Add the policy file.
  Define your policy file. This can be used to move data from the system pool to the data pool when thresholds reached. For more information, see Overview of policies.
  
  You can also use the GUI to define policies. For more information, see Creating and applying ILM policy by using GUI.
  The following example rule ingests the writes on the IBM Storage Scale System 3000 and moves the data to IBM Storage Scale System 5000 when it reaches 75% capacity on the IBM Storage Scale System 3000:
  - Add callback for automatic movement of data between pools:
```
mmaddcallback MIGRATION --command /usr/lpp/mmfs/bin/mmstartpolicy --event lowDiskSpace,noDiskSpace --parms "%eventName %fsName"
```
  - Write the policy into a file with the following content:
```
RULE 'clean_system' MIGRATE FROM POOL 'system' THRESHOLD(75,25) WEIGHT(KB_ALLOCATED) TO POOL 'data'
```
    Note: You need to understand the implications of this rule before applying it in your system. When capacity on IBM Storage Scale System 3000 reaches 75%, it migrates files (larger ones first) out of the system pool to the data pool until the capacity reaches 25%.
8. On the management server node, run the following command.
```
mmaddcompspec default --replace
```
9. Add the IBM Storage Scale System 5000 nodes to the pmsensors list and use the Edit rack components option in the GUI to slot the new nodes into the frame.
10. If you want to add protocol nodes, see Creating a shared root file system for CES.