Configuring multiple file system mount point access

Currently, the IBM Spectrum® Scale service GUI can support only up to two file systems.

Note: The gpfs.storage.type has to be configured during the initial deployment of the IBM Spectrum Scale service and it cannot be changed later.

During the IBM Spectrum Scale Ambari deployment, the following fields are required for setting up the multiple file system access:

Fields Description
gpfs.storage.type

Type of Storage. Comma-delimited string.

The first value will be treated as the primary file system and the values after that will be treated as the secondary file systems.

Only the following combination of file system values is supported:

gpfs.storage.type=local,remote

gpfs.storage.type=remote,remote

gpfs.storage.type=shared,shared
gpfs.mnt.dir Mount point directories for the file systems. Comma-delimited string. The first entry is for the primary file system. The second entry is the secondary file system.
gpfs.replica.enforced Replication type for each file system (dfs or gpfs). Comma-delimited string. The first entry is for the primary file system. The second entry is the secondary file system.
gpfs.data.dir Only one value must be specified. Null is a valid value. The data directory is created only for the primary file system.
GPFS file system name Names of the file systems. Comma-delimited string. The first entry is for the primary file system. The second entry is the secondary file system.
Note:
  1. If gpfs.storage.type has a local value, a pre-existing IBM Spectrum Scale cluster is required. If an FPO file system is not created, it can be created if the NSD stanza files are specified. If an FPO file system is created, the information is propagated in Ambari.
  2. If gpfs.storage.type has remote value, the pre-existing IBM Spectrum Scale remote mounted file system is required. For information on how to configure remote mount file system, see Configure remote mount access.
Follow the instructions based on the type of deployment model that you have:
  1. Add remote mount file systems access to existing HDP and an FPO file system that was deployed by IBM Spectrum Scale Ambari service.

    Prerequisites:

    • Deployed HDP.
    • Deployed FPO file system via IBM Spectrum Scale service through Ambari. The Ambari server requires to be on the GPFS master node.
    • Pre-existing remote mount file system.

    Use the gpfs.storage.type=local,remote configuration setting.

    On the Ambari server node on the local FPO file system:
    • Stop All services.

      On the Ambari UI, click Actions > Stop All1 to stop all the services.

    • On the owning IBM Spectrum Scale cluster, run the /usr/lpp/mmfs/bin/mmlsmlount all command to ensure that the file system is mounted.

      This step is needed for the IBM Spectrum Scale deploy wizard to automatically detect the existing file systems.

    • Update the IBM Spectrum Scale configuration:
      Click Ambari GUI > Spectrum Scale > Configs tab and update the following fields:
      • gpfs.storage.type
      • gpfs.mnt.dir
      • gpfs.replica.enforced
      • gpfs.data.dir
      • GPFS FileSystem Name

      In this example, the primary file system mount point is /localfs and the secondary file system mount point is /remotefs.

      Setting of the fields would be as follows:
      gpfs.storage.type=local,remote
gpfs.mnt.dir=/localfs,/remotefs
gpfs.replica.enforced=dfs,dfs
gpfs.data.dir=myDataDir OR gpfs.data.dir=
GPFS FileSystem Name=localfs,remotefs
    • Restart the IBM Spectrum Scale service.
    • Restart any service with Restart Required icon.
    • Click Ambari > Actions > Start All to start all the services.
  2. Add remote mount file systems access to existing HDP and an IBM Spectrum Scale FPO file system that was deployed manually.
    Prerequisites:
    • An FPO file system that is manually created.
    • Deployed HDP on the manually created FPO file system. The Ambari server requires to be on the GPFS master node.
    • Pre-existing remote mount file system.

    Use the gpfs.storage.type=local,remote configuration setting.

    On the Ambari server node on the local FPO file system, perform the following:

    • Stop All services.

      On the Ambari UI, click Actions > Stop All1 to stop all the services.

    • Start the IBM Spectrum Scale service cluster.

      On the local IBM Spectrum Scale cluster, run the /usr/lpp/mmfs/bin/mmstartup -a command.

    • Ensure all the remote mount file system is active and mounted.
    • On each IBM Spectrum Scale cluster, run the /usr/lpp/mmfs/bin/mmgetstate -a command to ensure it is started.

      This step is needed for the IBM Spectrum Scale deploy wizard to automatically detect the existing file systems.

    • Deploy the IBM Spectrum Scale service on the pre-existing file system.

      During deployment, the wizard would detect both the file systems and would populate the IBM Spectrum Scale config UI with recommended values for gpfs.storage.type, gpfs.mnt.dir gpfs.replica.enforced, gpfs.data.dir and GPFS FileSystem Name fields. Review the recommendations and correct them as needed before you continue to deploy the service.

      In this example, the primary file system mount point is /localfs and the secondary file system mount point is /remotefs.

      Setting of the fields would be as follows:
      gpfs.storage.type=local,remote
gpfs.mnt.dir=/localfs,/remotefs
gpfs.replica.enforced=dfs,dfs
gpfs.data.dir=myDataDir OR gpfs.data.dir=
GPFS FileSystem Name=localgpfs,remotegpfs
    • Click Ambari > Actions > Start All to start all the services.
  3. Add remote mount file systems access to existing HDP and a manually created IBM Spectrum Scale cluster.

    Create the FPO file system onto the local IBM Spectrum Scale cluster.

    Prerequisites:
    • A manual IBM Spectrum Scale cluster is created.
    • No FPO file system was created.
    • Deployed HDP onto the manual IBM Spectrum Scale cluster. The Ambari server requires to be on the GPFS master node.
    • Pre-existing remote mount file system.

    Use gpfs.storage.type=local,remote configuration setting.

    On the Ambari server node on the local cluster:

    • Stop All services.

      On the Ambari UI, click Actions > Stop All1 to stop all the services.

    • Start IBM Spectrum Scale service cluster.

      On the local IBM Spectrum Scale cluster, run the /usr/lpp/mmfs/bin/mmstartup -a command.

    • Ensure all the remote mount file system is active and mounted.
    • On each IBM Spectrum Scale cluster, run the /usr/lpp/mmfs/bin/mmgetstate -a command to ensure it is started.

      This step is needed for the IBM Spectrum Scale deploy wizard to automatically detect the existing file systems.

    • Deploy the IBM Spectrum Scale service.

      During deployment, the wizard would detect both the file systems and would populate the IBM Spectrum Scale config UI with recommended values for gpfs.storage.type, gpfs.mnt.dir, gpfs.replica.enforced, gpfs.data.dir and GPFS FileSystem Name fields. Review the recommendations and correct them as needed before you continue to deploy the service.

      In this example, the primary file system mount point is /localfs and the secondary file system mount point is /remotefs.

      • Configure fields for FPO cluster:
        • Update the NSD stanza file.

          If this is a standard stanza file, update the policy file field.

        • Review the replication fields. Default is set to 3.

      In this example, the primary file system mount point is /localfs and the secondary file system mount point is /remotefs.

      Setting of the fields would be as follows:
      gpfs.storage.type=local,remote
gpfs.mnt.dir=/localfs,/remotefs
gpfs.replica.enforced=dfs,dfs
gpfs.data.dir=myDataDir OR gpfs.data.dir=
GPFS FileSystem Name=localfs,remotefs
      Note: The newly created FPO cluster is set as the primary file system. The remote mounted file system is set as the secondary file system.
    • Restart IBM Spectrum Scale service.
    • Restart any service with the Restart Required icon.
    • On the Ambari UI, click Actions > Start All to start all the services.
  4. Add only the remote mount file systems access to existing HDP and a manually created IBM Spectrum Scale cluster.
    Prerequisites:
    • A manual IBM Spectrum Scale cluster is created.
    • Deployed HDP onto the manual IBM Spectrum Scale cluster. The Ambari server node requires to be on the GPFS master node.
    • Pre-existing remote mount file systems.

    Use gpfs.storage.type=remote,remote configuration setting.

    On the Ambari server node, on the local cluster:

    • Stop All services.

      On the Ambari UI, click Actions > Stop All1 to stop all the services.

    • Start the IBM Spectrum Scale cluster.

      On the local IBM Spectrum Scale cluster, run the /usr/lpp/mmfs/bin/mmstartup -a command.

    • Ensure all the remote mount file system is active and mounted.
    • On each IBM Spectrum Scale cluster, run the /usr/lpp/mmfs/bin/mmgetstate -a command to ensure it is started.

      This step is needed for the IBM Spectrum Scale deploy wizard to automatically detect the existing file systems.

    • Deploy the IBM Spectrum Scale service.

      During deployment, the wizard would detect both the file systems and would populate the IBM Spectrum Scale config UI with recommended values for gpfs.storage.type, gpfs.mnt.dir gpfs.replica.enforced, gpfs.data.dir and GPFS FileSystem Name fields. Review the recommendations and correct them as needed before you continue to deploy the service.

      In this example, the primary file system mount point is /remotefs1 and the secondary file system mount point is /remotefs2.

      Setting of the fields would be as follows:
      gpfs.storage.type=remote,remote
gpfs.mnt.dir=/remotefs1,/remotefs2
gpfs.replica.enforced=dfs,dfs
gpfs.data.dir=myDataDir OR gpfs.data.dir=
GPFS FileSystem Name=remotefs1,remotefs2
    • On the Ambari UI, click Actions > Start All to start all the services.
  5. Add the shared file system access to an existing HDP Scale cluster from an ESS or IBM Spectrum Scale cluster. Shared file system mode is a single GPFS cluster where the Hadoop Scale cluster is part of the existing ESS or IBM Spectrum Scale cluster.
    Prerequisites:
    • Deployed HDP cluster.
    • Two pre-existing GPFS file system.

    Use the gpfs.storage.type=shared,shared configuration setting.

    On the Ambari server node, on the local cluster:
    • Stop All services.

      On the Ambari UI, click Actions > Stop All to stop all the services.

    • Ensure that each of the file system is active and mounted on the ESS or the IBM Spectrum Scale cluster.
    • Deploy the IBM Spectrum Scale service. During deployment, the wizard would detect both the file systems and would populate the IBM Spectrum Scale config UI with recommended values for gpfs.storage.type, gpfs.mnt.dir, gpfs.replica.enforced, gpfs.data.dir and GPFS file system Name fields. Review the recommendations and correct them as needed before you continue to deploy the service.

      In this example, the primary file system mount point is /essfs1 and the secondary file system mount point is /essfs2.

      Setting of the fields would be as follows:
      gpfs.storage.type=/shared,/shared
gpfs.mnt.dir=/essfs1,/essfs2
gpfs.replica.enforced=dfs,dfs
gpfs.data.dir=myDataDir OR gpfs.data.dir=
GPFS FileSystem Name=essfs1,essfs2
    • After the IBM Spectrum Scale service is deployed successfully, on the Ambari UI, click Actions > Start All to start all the services.

1For FPO cluster, do not run STOP ALL from the Ambari GUI. Refer to the Limitations > General section on how to properly stop IBM Spectrum Scale.