Configuring a Liberty collective

You can organize Liberty servers into collectives to support clustering, administration, and other operations that act on multiple Liberty servers. By using collectives, you can efficiently and accurately deliver application services to your application.

1. Create and configure your controller.
   1. Create a server manually to act as the collective controller.

      In this procedure, the collective controller is refereed to as controller1.

      Place the controller in its own Liberty user directory, separate from the member servers for the controller. Specify the user directory on the WLP_USER_DIR environment variable. If necessary, customize the server.env file to contain the correct values for the JAVA_HOME environment variable and other settings.

   2. Create the collective controller configuration.
      The collective controller configuration consists primarily of the administrative domain security configuration that is used for secure communication between controllers and members. To create a collective that uses self-signed certificate keystores, run the following command. The --keystorePassword parameter must be specified, but the value for it is ignored on the z/OS operating system.

      wlp/bin/collective create controller1 --keystorePassword=controllerKSPassword

      This command creates XML statements that are added to the server.xml file of the controller. By default, these XML statements are written to the console.

      You can alternatively redirect the controller1 controller into an .xml file by using the --createConfigfile parameter:

      wlp/bin/collective create controller1 --keystorePassword=controllerKSPassword --createConfigFile=$WLP_USER_DIR/servers/controller1/controller.xml

      Put this .xml file in the server directory of the controller.

      By default, this collective command generates public and private SSH keys for collective-wide use. For more information on collective-wide SSH, including how to use existing keys, see Collective-wide SSH key configuration.

      If you set up your own keystore and truststore files, you can add parameters to the collective create command to specify the keystores and truststores to use:

      wlp/bin/collective create controller1 --keystorePassword=controllerKSPassword --createConfigFile=$WLP_USER_DIR/servers/controller1/controller.xml --serverIdentityKeystore="keystore_filespec" --serverIdentityKeystoreAlias="certificate_alias" --collectiveTrustKeystore="truststore_filespec" [–certificateRdn="xxx"]

   3. Update the server.xml file of the collective controller.
      If the collective create command wrote the configuration output to the console, copy this output and paste it into the controller server.xml file. If you instead used the --createConfigFile parameter to write the configuration to a file, use an include statement to specify this file in the server.xml file. The following code is an example of an include statement:

      <include location="c:\wlp\usr\servers\controller1\controller.xml" />

      To provide an administrative user name and password for the collective, edit the following statement in the server.xml file or the included file:

      <quickStartSecurity userName="" userPassword="" />

      Change this statement by specifying an administrative user name and password like the following example:

      <quickStartSecurity userName="adminUser" userPassword="adminPassword" />

      Alternatively, add registry information to the server.xml file for the controller. Specify your own administrative ID and password through the registry.
   4. Start the collective controller server.
      Run a command like the following one:

      wlp/bin/server start controller1

   5. Verify that the collective controller server started correctly and is ready to receive members.
      1. Open an editor on the collective controller messages log, $WLP_USER_DIR/servers/controller1/logs/messages.log.
      2. Look for the following message:

         CWWKX9003I: CollectiveRegistration MBean is available.

2. Add the server to the collective as a member.

   The controller and member servers can be on separate hosts but they must be in the same data center. In this example, the controller and member server are on the same host. The member server in the example is server1.

   1. If the server does not exist, create it.
      Run the following command:

      wlp/bin/server create server1

   2. Add the server to the collective as a member.

      The collective join command adds the server to the collective configuration, and also creates server-side XML to define the communication path to the collective controller.

      From the host server, run the collective join command to add the server.

      Before you run the command, ensure that you have these items:

      • A network connection to the collective controller.
      • An administrative user ID and password to perform MBean operations on the controller.
      • Values for the --host, --port, -user, and --password parameters. Obtain these parameters from the server.xml file of the collective controller.

      Run the following command:

      wlp/bin/collective join server1 --host=controllerHostname --port=9443 --user=adminUser --password=adminPassword --keystorePassword=memberKSPassword [--hostname=memberHostname]

      On the --keystorePassword parameter, set a value for the member keystore password, such as memberKSPassword. You can specify different --keystorePassword values for each server that you join to the collective.

      Set the optional --hostname parameter only if the system for the member server has multiple host names or does not have a host name configured. If you set the parameter, the value must match the defaultHostName variable that is defined in the server.xml file.

      To reduce the number of parameters needed, use the --controller parameter instead of the --user, --password, --host, and --port parameters:

      wlp/bin/collective join server1 --controller=user[:password]@host:httpsPort --keystorePassword=memberKSPassword [--hostname=memberHostname]

      By default, self-signed certificates are created. For more information on collective-wide SSH, including how to use existing keys, see Collective-wide SSH key configuration.

      If you already set up your own keystore and truststore files, you can add more parameters to the collective join command to specify the keystore and truststore files to use:

      wlp/bin/collective join server1 --controller=user[:password]@host:httpsPort --keystorePassword=controllerKSPassword --serverIdentityKeystore="keystore_filespec" --serverIdentityKeystoreAlias="certificate_alias" --collectiveTrustKeystore="truststore_filespec" [–certificateRdn="xxx"]

      For example, the JVM java.security file might specify the following two cryptographic providers.

      • security.provider.1=com.ibm.crypto.hdwrCCA.provider.IBMJCECCA
      • security.provider.2=com.ibm.crypto.provider.IBMJCE

      In this case, include the --safKeystoreType and --safKeystoreProvider parameters to specify the IBMJCECCA cryptographic hardware provider, as shown in the following example.

      wlp/bin/collective join memberName --host=controllerHostname --port=9443 --user=adminUser --password=adminPassword  
      --keystorePassword=memberKSPassword --serverIdentityKeystore="safkeyringhw:///MEMBER.KEY" 
      --serverIdentityKeystoreAlias="MEMBER" --collectiveTrustKeystore="safkeyringhw:///MEMBER.KEY" 
      --safKeystoreType=JCECCARACFKS --safKeystoreProvider=IBMJCECCA

      In a hybrid scenario, the JVM java.security file might specify the following three cryptographic providers.

      • security.provider.1=com.ibm.crypto.ibmjcehybrid.provider.IBMJCEHYBRID
      • security.provider.2=com.ibm.crypto.hdwrCCA.provider.IBMJCECCA
      • security.provider.3=com.ibm.crypto.provider.IBMJCE

      In this case, the --safKeystoreType and --safKeystoreProvider parameters can specify the IBMJCEHYBRID cryptographic hybrid provider, as shown in the following example.

      wlp/bin/collective join memberName --host=controllerHostname  --port=443 --user=adminUser --password=adminPassword
      --keystorePassword=memberKSPassword --serverIdentityKeystore="safkeyringhybrid:///MEMBER.KEY" 
      --serverIdentityKeystoreAlias="MEMBER" --collectiveTrustKeystore="safkeyringhybrid:///MEMBER.KEY"
       --safKeystoreType=JCEHYBRIDRACFKS --safKeystoreProvider=IBMJCEHYBRID

      In version 22.0.0.10 and later, the collective join --help command provides information about the --safKeystoreType and --safKeystoreProvider parameters.

      To write the output of this collective command to a file, instead of to a console screen, specify an optional --createConfigFile parameter. Then, include the outputted file in the collective configuration by adding an include statement to the member server.xml file:

      <include location=outputFilePath />

      By default, the join operation leaves remote procedure call (RPC) credentials undefined. You must specify values for the rpcUser parameter, the rpcUserPassword parameter. You must also specify values for the operating system login user ID and password for the host on which the member server exists. If the member host is registered with the collective controller and the member host is not enabled for SSH, specify an optional --useHostCredentials parameter. When you specify this parameter, the member inherits the RPC credentials from its host registration on the controller. Typically, Linux® hosts are enabled for SSH and Windows hosts are not enabled for SSH; thus, the --useHostCredentials parameter is useful for Windows member hosts. Specifying --useHostCredentials adds <hostAuthInfo useHostCredentials="true" /> to the member server.xml file. You then can run collective member server commands such as start or stop without specifying RPC credentials because the member inherits credentials from its host. See Overriding Liberty server host information for information about the hostAuthInfo element, the --useHostCredentials parameter, and connecting the collective controller to the server. See Setting up RXA for Liberty collective operations for information about enabling SSH on your host computer.

      For information about these required parameters and other optional parameters, run the collective help join command at a command line.

   3. If you are prompted to accept the certificate chain, enter y (yes).
   4. Update the member server.xml file.
      • If the collective join command wrote the configuration output to the console, copy this output and paste it into the server.xml file of the member server.
      • If you specified the --createConfigFile parameter to put the member configuration in a file, add an include statement to the server.xml file of the member server. The following example is of an include statement:

        <include location="c:\wlp\usr\servers\server1\member.xml" />

   5. If you did not specify the --useHostCredentials parameter on the collective join command and the member host is not enabled for SSH, set RPC credentials for the hostAuthInfo element. Set these credentials in the member server.xml file or in the outputted file.
      You can set RPC credentials for the member server in either of two ways:

      • Set the RPC user and password values on the hostAuthInfo element.

        Set the rpcUser option to an operating system login user ID for the host on which the member server exists.

        Set the rpcUserPassword option to the operating system login password for the user ID.

        For example, if you log in to the member computer with the test1 user ID and the test1pwd password, then change the hostAuthInfo element:

        <hostAuthInfo rpcUser="test1" rpcUserPassword="test1pwd" />

      • If the member host is registered with the collective controller, set the useHostCredentials option on the hostAuthInfo element to a value of true for the member server to inherit RPC credentials from its host:

        <hostAuthInfo useHostCredentials="true" />

      See Overriding Liberty server host information for information about the hostAuthInfo element and for an example that shows how to register a member host and run the join command with the --useHostCredentials parameter.

   6. Start the member server.

      wlp/bin/server start member1

   7. Verify that the member server started correctly and is publishing information to the controller.
      1. Open an editor on the member messages log, $WLP_USER_DIR/servers/member1/logs/messages.log.
      2. Look for the following messages in any order:

         CWWKX8112I: The server's host information was successfully published to the collective repository.
         CWWKX8114I: The server's paths were successfully published to the collective repository.
         CWWKX8116I: The server STARTED state was successfully published to the collective repository.

   8. Optional: Use the testConnection command to validate connectivity.
      The command validates RXA connectivity between the controller and the host where the member exists. It also validates JMX secure connectivity between the collective controller and the collective member.

      wlp/bin/collective testConnection hostName,usrDirPath,serverName  --host=controllerHost
      --port=controllerHTTPSPort --user=controllerAdmin 
      --password=controllerAdminPassword--autoAcceptCertificates

      Where the hostName,usrDirPath,serverName tuple describes the member server just joined: The hostName variable is the name of the host where the collective member exists. The userDirPath variable is the user directory of the collective member. serverName is the name of the collective member.

      Alternatively, use the simplified --controller parameter to provide the controller-specific information.

      wlp/bin/collective testConnection hostName,usrDirPath,serverName 
      --controller=user[:password]@host:HttpsPort 
      --autoAcceptCertificates

3. Add more servers to the collective as members.

   Repeat the previous step to add more servers to the collective.