scalectl policy command

Availability

Available on all IBM Storage Scale editions.

Description

Use the scalectl policy command to get, apply and update the file system policy.

Parameters

get {FilesystemName}

Retrieves the file system policy.

update {FilesystemName}

Updates the file system policy.

-F or --file {File}

Specifies the policy file to install.

-test-only

Tests only the validity of the policy file without installing.

apply {FilesystemName}

Applies a file system policy to delete files, migrate files between storage pools, or perform file compression or decompression as defined by policy rules. The operation attribute for this command is LRO. To run this command, you must have the RBAC permission for the start action on the /scalemgmt/v3/filesystems/{filesystem}/policy:start resource.

For more information, Policies for automating file management.

Note: This command is supported only when scalectl connects locally through a Unix Domain Socket (UDS).

Remember: When there are many millions of files to scan and process, the scalectl policy apply command can be very demanding of memory, CPU, I/O, and storage resources. Among the many options, consider specifying values for -N or --target-nodes, -a or --iscan-threads, -m or --exec-threads, --sort-buffer-size, and --qos-class to control the resource usage of scalectl apply policy.

You can run the scalectl policy apply command from any API node in the cluster that has mounted the file system owned by the local cluster.

The scalectl policy apply command does not affect placement rules (for example, the SET POOL and RESTORE rule) that are installed for a file system by the scalectl policy update command.

The scalectl policy apply command requires a policy file to be provided as input.

A given file can match more than one list rule. However, it is included only once in a given list. ListName provides the binding to an EXTERNAL LIST rule that specifies the executable program that is uded to process the generated list.

The EXTERNAL POOL rule defines an external storage pool. This rule does not match files, but serves to define the binding between the policy language and the external storage manager that implements the external storage.

Any given file is a potential candidate for at most one MIGRATE or DELETE operation during one invocation of the scalectl policy apply command. That same file may also match the first applicable LIST rule.

A file that matches an EXCLUDE rule is not subject to any subsequent MIGRATE, DELETE, or LIST rules. You should carefully consider the order of rules within a policy to avoid unintended results.

For more information about the GPFS policies, see the IBM Storage Scale: Administration Guide.

This command cannot be run from a non-Linux node. The GPFS API, documented functions in gpfs.h are not available on Windows. However, the policy language does support the Windows file attributes, so GPFS Windows files can be managed by using this command on a Linux node.

To terminate a scalectl policy apply command, you can cancel by using the scalectl operation cancel command.

--allow-scan-on-remote

Enables a policy scan on a remotely mounted file system to do a LIST operation.

-I or --apply-action {yes | defer | test | prepare}

Specifies what action the apply policy command performs on files. The possible values are yes, defer, test, and prepare. The default value is yes.

yes

Indicates that all applicable policy rules are run, and the data movement between pools is done during the processing of the scalectl policy apply command. All defined external lists are executed.

defer

Indicates that all applicable policy rules are run, but actual data movement between pools is deferred until the next scalectl filesystem restripe command.

test

Indicates that all policy rules are evaluated, but the scalectl policy apply command only displays the actions without performing them. There is no actual deletion of files or data movement between pools. This option is intended for testing the effects of particular policy rules.

prepare

Indicates that all policy execution is deferred and that the scalectl policy apply command only prepares file lists that are suitable for execution with the -r option. Records are created for each selected files and are stored in one or more file lists. These file lists are written to default global work directory. The actual data movement occurs when the command is run again with the -r option.

--directory-path {Path}

Specifies the fully-qualified path name of a GPFS file system subtree to which policy rules are applied. If not specified, the root of the file system is used.

-m or --exec-threads {ThreadCount}

Specifies the number of threads that are created and dispatched within each apply policy process during the policy execution phase. The default value is 24.

-F or --file {PolicyFile}

Specifies the file containing policy rules to apply.

-r or --filelist-pathname {Path}

Specifies one or more file lists of files for policy execution. The file lists that were used as input for -r were created by issuing scalectl policy apply with the -I prepare flag. The file lists must be within the specified file system. You can specify several file lists by doing one of the following:

• Provide the path name of a directory of file lists.
• Specify the -r option several times, each time with the path name of a different file list.

Use this parameter to logically continue from a previous scalectl policy apply command that was run with the -I prepare option. To continue, run the scalectl policy apply command again with the same parameters and options, except the -I prepare option and add the -r option. Between the runs, you can process, reorder, filter, or edit the file lists that were created when you invoked -I prepare. Specify one or more of these resulting file lists with the -r option.

The format of the records in each file list can be expressed as:

iAggregate:WEIGHT:INODE:GENERATION:SIZE:iRule:resourceId:attr_flags:
path-length!PATH_NAME:pool-length!POOL_NAME
[;show-length!SHOW]end-of-record-character

For more information about the file list records, see IBM Storage Scale: Administration Guide.

-a or --iscan-threads {ThreadCount}

Specifies the number of threads and sort pipelines that each node runs during parallel inode scanning and policy evaluation. This parameter affects the high-performance protocol that is used when multiple nodes participate in the operation (-N).

The default value is 2. Increasing this value can improve performance, but might increase resource usage on each node. In some environments, setting a high value can cause the command to fail.

Note: Set this parameter to the number of CPU cores on a typical node in the IBM Storage Scale cluster.

-B or --max-files {MaxFiles}

Specifies how many files are passed for each invocation of the EXEC script. The default value is 100.

-L or --msg-level {Level}

Controls the level of information displayed by the scalectl policy apply command. Larger values indicate the display of more detailed information. These terms are used:.

candidate file

A file that matches a MIGRATE, DELETE, or LIST policy rule.

chosen file

A candidate file that has been scheduled for action.

These are the valid values for n:

0

Displays only serious errors.

1

Displays some information as the command runs, but not for each file. This is the default.

2

Displays each chosen file and the scheduled migration or deletion action.

3

Displays the same information as 2, plus each candidate file and the applicable rule.

4

Displays the same information as 3, plus each explicitly EXCLUDEed or LISTed file, and the applicable rule.

5

Displays the same information as 4, plus the attributes of candidate and EXCLUDEed or LISTed files.

6

Displays the same information as 5, plus non-candidate files and their attributes.

For examples and more information on this flag, see the section: The mmapplypolicy -L command in the IBM Storage Scale: Problem Determination Guide.

--qos-class QosClass

Specifies the Quality of Service (QoS) for I/O operations class to which the instance of the command is assigned. If you do not specify this parameter, the instance of the command is assigned by default to the maintenance QoS class. This parameter has no effect unless the QoS service is enabled. Specify one of the following QoS classes:

maintenance

This QoS class is typically configured to have a smaller share of file system IOPS. Use this class for I/O-intensive, potentially long-running IBM Storage Scale commands, so that they contribute less to reducing overall file system performance.

other

This QoS class is typically configured to have a larger share of file system IOPS. Use this class for administration commands that are not I/O-intensive. For more information, see --qos in the scalectl filesystem command.

--scope {filesystem | inode-space | fileset}

Indicates the levels of scope to be applied to policy scan. The possible values are filesystem, fileset, and inode-space. The default value is filesystem.

filesystem

The scan involves the objects in the entire file system subtree pointed to by the Directory parameter. This filesystem level scan is the default action.

fileset

The scope of the scan is limited to the objects in the same fileset as the directory pointed to by the Directory parameter.

inode-space

The scope is limited to objects in the same single inode space from which the directory pointed to by the Directory parameter is allocated. The scan may span more than one fileset, if those filesets share the same inode space.

-S or --snapshot-name

Specifies the name of a global snapshot for file system backup operations or for migrating snapshot data. The name appears as a subdirectory of the .snapshots directory in the file system root and can be found with the scalectl filesystem snapshot list command.

Note:

• GPFS snapshots are read-only. Do not use deletion rules with -S {SnapshotName}.
• Do not use the -S option to scan files in a fileset snapshot. Instead, specify the full path of a directory in the snapshot as in the following example:

  scalectl policy apply fs1 --directory-path <directory_path> -F policyfile ...

• If multiple snapshots for the same file system or fileset exist, start migrating data from the latest snapshot and continue the snapshot migration backward in time until you reach the earliest snapshot available.

--sort-buffer-size Size

Sets the sort-buffer size that is passed to the sort command. This parameter limits memory usage by the sort commands that the scalectl policy apply command calls to do sorts and merges.

The scalectl policy apply command must do multiple, potentially large sorts and merges of temporary files as part of its processing. It calls the operating-system sort command each time that it must do a sort. It can have multiple instances of the sort command running at the same time. If the numbers of items to be sorted are very large, the result can be excessive memory usage. The operating system can respond by terminating some of the sort processes, which causes the scalectl policy apply command to run for a longer time or to return with an error.

To prevent excessive memory consumption, you can set the --sort-buffer-size parameter to a lower value than its default. The --sort-buffer-size parameter is the value that the scalectl policy apply command passes to the sort command in the buffer-size parameter. The default value is 8%. If you need a lower value, you might set it to 5%.

You can specify the --sort-buffer-size parameter in any format that the buffer-size parameter of the sort program accepts values such as 5% or 1M.

In general, accept the default value of this parameter unless the system has excessive memory consumption that is attributable to large sort operations by the scalectl policy apply command.

-N or --target-nodes

Specifies a list of target IBM Storage Scale nodes to run parallel instances of policy code for better performance. The nodes must be in the same cluster as the node from which the scalectl policy apply command is issued. The value can include any combination of node numbers, node number ranges, node names, IP addresses, or node classes. If the -N option is not specified, then the command runs parallel instances of the policy code on the nodes that are specified by the defaultHelperNodes attribute of the scalectl config cluster command. If the defaultHelperNodes attribute is not set, then the helper nodes are the members of the node class managerNodes. Otherwise, the command runs only on the node where the apply policy command is issued.

Global flags

Exit status

0

Successful completion.

nonzero

A failure occurred.

Security

You must have the specific role-based access control (RBAC) permission to run the command. For more information, see Role-based access control.

See also

Location

/usr/lpp/mmfs/bin