Using IBM Spectrum Scale Policies for automated Information Lifecycle Management
Ulf Troppens 2700003H05 Comments (3) Visits (9418)
By Markus Rohwedder, Przemyslaw Podfigurny, Stefan Roth and Ulf Troppens
IBM Spectrum Scale provides means to include a broad range of storage devices into the same file system and to automate their efficient use based on policies. Spectrum Scale allows to group storage devices based on their performance, cost, locality and reliability characteristics (for example SSD drives, spinning disk drives or tape storage) in so called storage pools. A Spectrum Scale file system comprises one or more storage pools. Spectrum Scale also includes a policy engine which quickly identifies files based on their attributes and manages them automatically via rules.
Spectrum Scale rules enable automation of file placement, migration, listing, compression, encryption and deletion. A set of rules is called policy. The active policy of a file system must contain a default placement rule. Additionally it can contain further rules to handle the placement of selected files based on criteria (e.g. file type) and migration, compression and deletion rules which start certain actions based on capacity usage thresholds.
Properly configured rules optimize the use of premium and less expensive storage resources. In this blog posting we describe basic policies for the placement and movement of files:
Spectrum Scale GUI supports the management of file placement and file management rules for the most common use cases. More sophisticated use cases can be configured with the Command Line Interface (CLI).
To open the GUI panel for policies and rules go to Files → Information Lifecycle.
The example shows a file system 'fs1' which is pre-configured with two storage pools (‘system’, ‘Archive’). Each file system includes the system storage pool and optional additional storage pools. Metadata is stored in the system storage pool only. In addition, the system storage pool can optionally contain data. For this example we assume that the system storage pool includes SSD drives to provide best performance for metadata and data. The ‘Archive’ storage pool uses near-line SAS disk drives. The active policy includes two placement rules (‘mp3Placement’, ‘default’) and one migration rule (‘th
The ‘mp3placement’ rule applies to all files which end with ‘.mp3’. The idea is that mp3 files never need to be stored on expensive SSD drives. They directly get stored on the near-line SAS disk drives in the Archive pool, when they are created.
Spectrum Scale stores rules in a SQL-like syntax. The GUI generates the code for the rules and activates it. Here is the syntax for the ‘mp3placement’ rule:
In our example the 'second' placement rule is named 'default'. It stores all new files in the ‘system’ storage pool. The rule ordering is essential, because Spectrum Scale searches for matching rules top down. The first matching rule will be applied and the default rule will be used only, if no other rule matches before. Therefore, the default placement rule needs to be placed after all other placement rules.
Here is the syntax for the ‘default’ rule:
In contrast to placement rules which are only evaluated during file creation, a migration rule relates to existing files. In our case the migration rule is evaluated only when a certain capacity usage threshold is reached. The ‘thr
Here is the syntax for the ‘thr
Spectrum Scale policies are a powerful tool for automated file management. This blog posting gives just a basic introduction. Check out Spectrum Scale knowledge center for more details or download the evaluation virtual machine and try it on your own.
IBM Spectrum Scale 4.2 Knowledge Center
Spectrum Scale Evaluation Virtual Appliance