Policy rules: Terms

The terms of policy rules specify conditions for selecting files and operations to perform on files.

The following terms are used in policy rules. Some terms appear in more than one rule:

ACTION (SqlExpression)

Specifies an SQL expression that is evaluated only if the other clauses of the rule are satisfied. The action of the SqlExpression is completed, and the resulting value of the SqlExpression is discarded. In the following example, the rule sets the extended attribute user.action to the value set pool s6 for files that begin with the characters sp. These files are assigned to the system pool:

rule 's6' set pool 'system' action(setxattr('user.action','set pool s6')) where name like 'sp%'

Note: Encryption policies do not support the ACTION clause.

COMPRESS ({'yes' | 'no})

Indicates that the file is to be compressed or decompressed. The following rule compresses the files in the pool datapool that begin with the string green%:

RULE 'COMPR1' MIGRATE FROM POOL 'datapool' COMPRESS('yes') WHERE NAME LIKE 'green%'

For more information about file compression and decompression, see File compression.

DIRECTORIES_PLUS

Indicates that non-regular file objects (directories, symbolic links, and other items) must be included in the list. If not specified, only ordinary data files are included in the candidate lists.

DELETE

Identifies a file deletion rule. A file that matches this rule becomes a candidate for deletion.

ESCAPE '%SpecialCharacters'

Specifies that path names and the SHOW('string') expressions within the associated file lists are encoded with a scheme based on RFC3986 URI percent encoding.

Compare the two following rules:

RULE 'xp' EXTERNAL POOL 'pool-name' EXEC 'script-name' ESCAPE '%'
RULE 'xl' EXTERNAL LIST 'list-name' EXEC 'script-name' ESCAPE '%/+@#'

Both rules specify that all characters except the unreserved characters in the set a-zA-Z0-9-_.~ are encoded as %XX, where XX comprises two hexadecimal digits.

However, the GPFS™ ESCAPE syntax adds to the set of unreserved characters. In the first rule, the syntax ESCAPE '%' specifies a rigorous RFC3986 encoding. Under this rule, a path name such as /root/directory/@abc+def#ghi.jkl appears in a file list in the following format:

%2Froot%2Fdirectory%2F%40abc%2Bdef%23ghi.jkl

In the second rule, the syntax ESCAPE '%/+@#' specifies that none of the characters in set /+@# are escaped. Under this rule, the same path name appears in a file list in the following format:

/root/directory/@abc+def#ghi.jkl

If you omit the ESCAPE clause, the newline character is escaped as '\n', and the backslash character is escaped as '\\'; all other characters are presented as is, without further encoding.

EXCLUDE

Identifies a file exclusion rule.

RULE ’x’ EXCLUDE: A file that matches this form of the rule is excluded from further consideration by any MIGRATE or DELETE rules that follow.
RULE 'rule-name' LIST ’listname-y’ EXCLUDE: A file that matches this form of the rule is excluded from further consideration by any LIST rules that name the same listname-y.

EXEC 'InterfaceScript'

Specifies an external program to be invoked to pass requests to an external storage management application. InterfaceScript must be a fully qualified path name to a user-provided script or program that supports the commands described in User-provided program for managing external pools.

EXTERNAL LIST ListName

Defines an external list. This rule does not match files. It provides the binding between the lists that are generated with regular LIST rules with a matching ListName and the external program that you want to run with these lists as input.

EXTERNAL POOL PoolName

Defines an external storage pool. This rule does not match files but defines the binding between the policy language and the external storage manager that implements the external storage.

FOR FILESET ('FilesetName'[,'FilesetName']...)

Specifies that the rule applies only to files within the specified filesets.

FROM POOL FromPoolName

Specifies the name of the source pool from which files are candidates for migration.

GROUP POOL PoolName

Defines a group pool. This rule supports the concept of distributing data files over several GPFS disk pools.

Optionally, a LIMIT, specified as an occupancy percentage, can be specified for each disk pool; if not specified, the limit defaults to 99%. The THEN keyword signifies that disk pools that are specified before a THEN keyword are preferred over disk pools that are specified after. When a pool that is defined by a GROUP POOL rule is the TO POOL target of a MIGRATE rule, the selected files are distributed among the disk pools that comprise the group pool. Files of highest weight are put into the most preferred disk pool up to the occupancy limit for that pool. If more files must be migrated, they are put into the second most preferred pool up to the occupancy limit for that pool. Again, files of highest weight are selected.

If you specify a file that is defined by a GROUP POOL rule in a FROM POOL clause, the clause matches any file in any of the disk pools in the group pool.

You can “repack” a group pool by WEIGHT. Migrate files of higher weight to preferred disk pools by specifying a group pool as both the source and the target of a MIGRATE rule.

rule 'grpdef' GROUP POOL 'gpool' IS 'ssd' LIMIT(90) THEN 'fast' LIMIT(85) THEN 'sata'
rule 'repack' MIGRATE FROM POOL 'gpool' TO POOL 'gpool' WEIGHT(FILE_HEAT)

See Tracking file access temperature within a storage pool.

LIMIT (OccupancyPercentage)

Limits the creation of data in a storage pool. GPFS does not migrate a file into a pool if doing so exceeds the occupancy percentage for the pool. If you do not specify an occupancy percentage for a pool, the default value is 99%. See Phase two: Choosing and scheduling files.

You can specify OccupancyPercentage as a floating point number, as in the following example:

RULE 'r' RESTORE to pool 'x' limit(8.9e1)

For testing or planning purposes, and when you use the mmapplypolicy command with the -I defer or -I test option, you can specify a LIMIT larger than 100%.

The limit clause does not apply when the target TO POOL is a GROUP POOL. The limits that are specified in the rule that defines the target GROUP POOL govern the action of the MIGRATE rule.

LIST ListName

Identifies a file list generation rule. A file can match more than one list rule but appears in a list only once. ListName provides the binding to an EXTERNAL LIST rule that specifies the executable program to call when the generated list is processed.

MIGRATE

Identifies a file migration rule. A file that matches this rule becomes a candidate for migration to the pool specified by the TO POOL clause.

OPTS 'OptionsString ...'

Specifies optional parameters to be passed to the external program defined with the EXEC clause. OptionsString is not interpreted by the GPFS policy engine.

REPLICATE (DataReplication)

Overrides the default data replication factor. This value must be specified as 1, 2, or 3.

RESTORE TO POOL PoolName

where Identifies a file restore rule. When you restore a file with the gpfs_fputattrswithpathname() subroutine, you can use this rule to match files against their saved attributes rather than the current file attributes. This rule also applies to a command that uses that subroutine, such as the IBM Spectrum Protect™ command dsmc restore.

RULE ['RuleName']

Initiates the rule statement. RuleName identifies the rule and is used in diagnostic messages.

SET POOL {PoolName | EXCLUDE}

Identifies an initial file placement rule.

PoolName: Specifies the name of the storage pool where all files that match the rule criteria are placed.
EXCLUDE: Specifies that files that match the rule criteria are excluded from further consideration and will not be stored in any pool. If you try to create a file that matches a SET POOL EXCLUDE rule, the file system API returns the error ENOSPC.

SET SNAP_POOL PoolName

Identifies an initial snapshot placement rule. PoolName specifies the name of the storage pool where all snapshot files that match the rule criteria are placed.

Note: The pool is only set when the file data is written to the snapshot, not when the snapshot is created.

SHOW (['String'] SqlExpression)

Inserts the requested information (the character representation of the evaluated SQL expression SqlExpression) into the candidate list that is created by the rule when it deals with external storage pools. String is a literal value that gets echoed back.

This clause has no effect in matching files but can be used to define other attributes to be exported with the candidate file lists.

SIZE (numeric-sql-expression)

Is an optional clause of any MIGRATE, DELETE, or LIST rules that are used for choosing candidate files. numeric-sql-expression specifies the size of the file to be used when in calculating the total amount of data to be passed to a user script. The default is KB_ALLOCATED.

SIZE sum-number

Is an optional clause of the EXTERNAL POOL and EXTERNAL LIST rules. sum-number limits the total number of bytes in all of the files named in each list of files passed to your EXEC 'script'. If a single file is larger than sum-number, it is passed to your EXEC 'script' as the only entry in a singleton file list.

Specify sum-number as a numeric constant or a floating-point value.

Note: The value of sum-number is in kilobytes.

THRESHOLD (HighPercentage[,LowPercentage[,PremigratePercentage]])

Controls migration and deletion based on the percent of assigned pool storage that is occupied.

HighPercentage

Indicates that the rule is to be applied only if the occupancy percentage of the current pool of the file is greater than or equal to the HighPercentage value. Specify a nonnegative integer in the range 0-100.

LowPercentage

Indicates that MIGRATE and DELETE rules are to be applied until the occupancy percentage of the current pool of the file is reduced to less than or equal to the LowPercentage value. Specify a nonnegative integer in the range 0-100. The default is 0%.

PremigratePercentage

Defines an occupancy percentage of a storage pool that is below the lower limit. Files that lie between the lower limit LowPercentage and the pre-migrate limit PremigratePercentage are copied and become dual-resident in both the internal GPFS storage pool and the designated external storage pool. This option allows the system to free up space quickly by deleting pre-migrated files if the pool becomes full. Specify a nonnegative integer in the range 0 to LowPercentage. The default is the same value as LowPercentage.

Notes:

Percentage values can be specified as numeric constants or floating-point values.
This option applies only when you migrate to the external storage pool.
This option does not apply when the current rule operates on one group pool.

THRESHOLD (ResourceClass)

Specifies the type of capacity-managed resources that are associated with ListName. The following values are valid:

FILESET_QUOTAS: Indicates that the LIST rule must use the occupancy percentage of the hard limit fileset quota per the mmlsquota and mmedquota commands.
FILESET_QUOTA_SOFT: Indicates that the LIST rule must use the occupancy percentage of the soft limit fileset quota per the mmlsquota and mmedquota commands.
GROUP_QUOTAS: Indicates that the LIST rule must use the occupancy percentage of the hard limit group quota per the mmlsquota and mmedquota commands.
GROUP_QUOTA_SOFT: Indicates that the LIST rule must use the occupancy percentage of the soft limit group quota per the mmlsquota and mmedquota commands.
POOL_CAPACITIES: Indicates that the LIST rule uses the occupancy percentage of the pool when it applies the threshold rule. This value is the default value. This value is used if the threshold is not specified in the EXTERNAL LIST rule but appears in the LIST rule.
USER_QUOTAS: Indicates that the LIST rule uses the occupancy percentage of the hard limit user quota per the mmlsquota and mmedquota commands.
USER_QUOTA_SOFT: Indicates that the LIST rule uses the occupancy percentage of the soft limit user quota per the mmlsquota and mmedquota commands.

Note: This option does not apply when the current rule operates on one group pool.

For more detail on how THRESHOLD can be used to control file migration and deletion, see Phase one: Selecting candidate files and Pre-migrating files with external storage pools.

TO POOL ToPoolName

Specifies the name of the storage pool to which all the files that match the rule criteria are migrated. This phrase is optional if the COMPRESS keyword is specified.

WEIGHT (WeightExpression)

Establishes an order on the matching files. Specifies an SQL expression with a numeric value that can be converted to a double-precision floating point number. The expression can refer to any of the file attributes and can include any constants and any of the available SQL operators or built-in functions.

WHEN (TimeBooleanExpression)

Specifies an SQL expression that evaluates to TRUE or FALSE, depending only on the SQL built-in variable CURRENT_TIMESTAMP. If the WHEN clause is present and TimeBooleanExpression evaluates to FALSE, the rule is skipped.

The mmapplypolicy command assigns the CURRENT_TIMESTAMP when it begins processing. It uses either the actual Coordinated Universal Time date and time or the date specified with the -D option.

WHERE SqlExpression

Specifies an SQL expression that can reference file attributes as SQL variables, functions, and operators. Some attributes are not available to all rules. Compares the file attributes specified in the rule with the attributes of the file that is created.

SqlExpression must be an expression that evaluates to TRUE or FALSE, but can be any combination of standard SQL syntax expressions, including built-in functions.

Omitting the WHERE clause entirely is equivalent to writing WHERE TRUE. The WHERE clause must be the last clause of the rule.