# IBM Storage Protect - Client Workload Simulation Tool

The Client Workload Simulation Tool (`sp_client_load_gen.pl`) can be used to test scalability of IBM Storage Protect server sessions.  It uses the `fakeload` to send data to the IBM Storage Protect server using the IBM Storage Protect API.  The `Client Workload Simulation Tool` uses in-memory data from the client system - to evaluate the network and server performance, without the I/O bottleneck (reads from the client disk).  In addition, the `Client Workload Simulation Tool` can be run on the IBM Storage Protect server node to measures CPU usage for the system, without the network delays between the client and server. 

## Usage

1. Generate [test data](./data/README.md) for load testing, in the `/benchdata` folder.
1. Install a `backup-archive client` and API on your test system
1. Update the `API options` files  
   * Note: Change the deduplication and compression options, if you prefer to test client-side deduplication and compression rather than server-side.
   * For UNIX: </br>
      `/opt/tivoli/tsm/client/api/bin64/dsm.sys`
      ``` 
        servername tapsrv03
          usereplicationfailover no
          tcps tapsrv03
          tcpp 1500
          deduplication no
          compression no
      ```
      `/opt/tivoli/tsm/client/api/bin64/dsm.opt`
      ``` 
        servername tapsrv03  
      ```

   * For Windows: </br> 
      Note: Create a `opt` file, where you extracted `Client Workload Simulation Tool` </br>
      set DSMI_CONFIG=c:\tools\sp-load-generator\dsm.opt
      ```
        C:\tools\sp-load-generator\dsm.opt 
          tcps tapsrv04
          tcpp 1500
          usereplicationfailover no
          deduplication no
          compression no
      ```
1. Register `nodenames`, for use by the `Client Workload Simulation Tool`.  
   * At least one node is needed for every session.
     * Example of registering a single node:
       ```
       dsmadmc -id=admin -pa=passw0rd reg node INLINE1 passw0rd do=tapsrv04_file  
       ```
   * Use the `regnodes.mac` macro, to automate the creation of 128 nodes.  
     * Note: The domain name for the test nodes must pre-exist, and should reference the test storage-pool as destination.
     * Example of using `regnodes.mac` macro:
       ```
        dsmadmc -id=admin -pa=passw0rd macro regnodes.mac MYDOMAIN
       ```  
     * Windows example:
       ```
        dsmadmc -id=admin -pa=passw0rd macro c:\tools\sp-load-generator\regnodes.mac tapsrv04_file
       ```

1. Run `Client Workload Simulation Tool` (or `sp_client_load_gen.pl`). 
   * Option 1: iterate from 1 to n sessions; for each step, record the _throughput_ and _CPU usage_ in a .csv file.  
   * Option 2: Use the `-sessmin` option to force a single iteration at a specific session count.
   * Examples to test scalability:
     1. Test with 10 parallel sessions, 20% dedup, 128 MB objects, 50 objects per session.  </br> 
        Total 10 sessions will backup 62.5 GB.  </br>
        Node name used: NODETEST1, NODETEST2, ... NODETEST10.
        ```
         perl sp_client_load_gen.pl -sesspeak=10 -sessmin=1 -objsize=128m -objcount=50 -dedup=20 -node=nodetest  
        ```
     1. Test with 10 parallel sessions, 20% dedup, 128 MB objects, 50 objects per session.  </br> 
        Node name used: NODETEST1, NODETEST2, ... NODETEST10. </br>
        Note: _Use different filespace, for each session._
        ```
         perl sp_client_load_gen.pl -sesspeak=10 -sessmin=1 -objsize=128m -objcount=50 -dedup=20 -node=nodetest -onenode  
         ```
     1. Test scaling from 1 session up to 20 sessions iteratively.  </br>
        Results are captured in a .csv file.
        ```
        perl sp_client_load_gen.pl -sesspeak=20 -objsize=128m -objcount=50 -dedup=20 -node=nodetest 
        ```
     1. Test with 40 sessions, where 20 sessions run on two different client systems (you need to set up the two clients). </br> 
        By using two different `-sessmin` value, you avoid using the same nodenames on the two systems. </br>
        On client1:
        ```
        perl sp_client_load_gen.pl -sesspeak=20 -sessmin=1 -objsize=128m -objcount=50 -dedup=20 -node=nodetest 
        ```
        On client2:
        ```
        perl sp_client_load_gen.pl -sesspeak=20 -sessmin=21 -objsize=128m -objcount=50 -dedup=20 -node=nodetest 
        ```
     1. Test with multiple nodes, each node running with multiple session to different filespaces.  </br>
        Backups will run to the nodes NODETEST2, NODETEST3, and NODETEST4 each with 5 sessions, for a total of 15 sessions.
        ```
        perl sp_client_load_gen.pl -sesspeak=3 -sessmin=2 -fscount=5 -objsize=128m -objcount=100 -node=nodetest 
        ``` 
   * Examples to test 'restore'
     1. It's recommended to take the backup with a very low deduplication savings to avoid reading the same data repeatedly from cache. </br>
        Note: Prior to restoring you must take a backup with the same parameters. </br>
        Backup:
        ```
        perl sp_client_load_gen.pl -sesspeak=20 -sessmin=1 -objsize=1g -objcount=100 -dedup=1 -node=nodetest  
        ```        
        Restore: 
        ```
        perl sp_client_load_gen.pl -sesspeak=20 -sessmin=1 -objsize=1g -objcount=100 -rest -node=nodetest    
        ``` 

## Troubleshooting

The output of each fakeload process is recorded in files named fake1.out, fake2,out, etc.  These files are cleaned up
on successful completion, but are left behind for any session which does not complete successfully.  Typically these files
will include an explanation for the failure.  Below are some common failures and recommendations for how to resolve the
problem.

1. > Error in dsmInitEx rc=-362. rcMsg=ANS1592E (RC-362) Failed to initialize SSL protocol.

   This is usually a result of the IBM Storage Protect SSL certificates not existing in the client's certificate database. </br>
   One approach to resolve this is to temporarily put the NODETEST nodes back into transitional authentication mode
   so that the certificate will be automatically transferred on the next attempt.  
   ```
   dsmadmc:  update node nodetest* sessionsec=transitional
   ```
   Alternatively, the server's public certificate can be copied over and applied with the dsmcert utility.

1. > Error in dsmSetUp for multi-threaded, rc=406 The DSMI_CONFIG is set to an invalid dsm.opt

   On Windows, it is necessary to prepare a dsm.opt file and set the DSMI_CONFIG environment variable to point to it.  </br>
   On UNIX, the DSMI_DIR may be set to a location without a valid dsm.sys file.

1. > Couldn't open the file <benchdata/bench1>.  Exiting.

   One or more of the required test files is missing from the benchdata folder.  
   </br> Did you run the create_data_files.pl script, or attempt to run more sessions than you created test files?

1. > Error in dsmInitEx rc=137. rcMsg=ANS1025E (RC137)  Session rejected: Authentication failure

   You have specified an incorrect nodename, the nodeprefix is not set correctly with `-node=`, or the node password is not correctly
   set with `-nodepw=`.

---

