File heat: Tracking file access temperature

A file's access temperature is an attribute for policy that provides a means of optimizing tiered storage.

File temperatures are a relative attribute, indicating whether a file is hotter or colder than the others in its pool. The policy can be used to migrate hotter files to faster tiers and colder files to slower tiers. The access temperature is an exponential moving average of the accesses to the file. As files are accessed the temperature increases; likewise when the access stops the file cools. File temperature is intended to optimize nonvolatile storage, not memory usage; therefore, cache hits are not counted. In a similar manner, only user accesses are counted.

The access counts to a file are tracked as an exponential moving average. An unaccessed file loses a percentage of its accesses each period. The loss percentage and tracking period are set by the following configuration values:

fileHeatPeriodMinutes: A nonzero value enables file access temperature tracking and specifies the frequency with which the file heat attributes are updated. A value of 0 disables FILE_HEAT tracking. The default value is 0.
fileHeatLossPercent: This attribute specifies the percent of file access heat that an unaccessed file loses at the end of each tracking period. The valid range is 0 - 100. The default value is 10. The tracking period is set by fileHeatPeriodMinutes.

For more information, see mmchconfig command.

File Heat can be configured on a per-cluster basis, not on a per–file system basis. Use WEIGHT(FILE_HEAT) with a policy MIGRATE rule to prioritize migration by file temperature. (You can use the GROUP POOL rule to define a group pool to be specified as the TO POOL target.) See Policies for automating file management.

Be aware of the following factors:

New values for fileHeatPeriodMinutes and fileHeatLossPercent are not effective until the GPFS daemon is stopped and restarted.
The file heat attribute of a file is accessible externally through the FILE_HEAT policy attribute. This attribute is not updated until the update inode of the file is written to the storage media. You can trigger the update by unmounting the file system.
After a file access, the access temperature of the file is increased when the file access time (atime) is set. If the updating of atime is suppressed or if relative atime semantics are in effect, proper calculation of the file access temperature may be adversely affected.

Object heatmap data tiering policies

For more information see the following topics:

Examples

The following command sets fileHeatPeriodMinutes to 1440 (24 hours) and fileHeatLossPercent to 10, meaning that unaccessed files lose 10% of their heat value every 24 hours, or approximately 0.4% every hour (because the loss is continuous and increases geometrically):
```
mmchconfig fileheatperiodminutes=1440,fileheatlosspercent=10
```
You can test file heat with the following steps:
1. Issue the mmchconfig command to set the file heat tracking period to 60 minutes. This action enables file access temperature tracking for the cluster if it is not already enabled. You can issue the mmlsconfig command to verify that the tracking period is set:
```
#mmchconfig fileHeatPeriodMinutes=60
mmchconfig: Command successfully completed
#mmlsconfig | grep -i heat
fileHeatPeriodMinutes 60
```
2. Restart the GPFS daemon to make the new fileHeatPeriodMinutes value effective.
```
#mmshutdown
#mmstartup
```
3. Mount a file system and verify that a file in it exists:
```
#mmmount c23
#ls -l /c23/10g
-rw-r--r--. 1 root root 10737418240 May 16 15:09 /c23/10g
```
4. Display the file access temperature (file heat) of the file:
```
#mmlsattr -d -X /c23/10g
file name:            /c23/10g
security.selinux
```
  In this example the file access temperature of the file has never been set, so the mmlsattr command does not report any file heat.
5. Run the dd command-line utility to read the file from the storage device:
```
dd if=/c23/10g bs=1M of=/dev/null
```
6. Wait a short while for the inode of the file to be written to the storage device. You can force this action by unmounting the file system.
7. Issue the mmlsattr command to verify that the file heat is updated:
```
#mmlsattr -d -X /c23/10g
file name:            /c23/10g
   ....
security.selinux
gpfs.FileHeat
```
8. Issue the mmlsattr command again to display the hexadecimal values of the cluster attributes:
```
#mmlsattr -d -X -L /c23/10g
file name:            /c23/10g
    ....
security.selinux:
   ....
gpfs.FileHeat:        0x000000EE42A40400
```
  In this example the file heat of the file is 0x000000EE42A40400. The FILE_HEAT policy attribute returns this value.

Below is an example of migration rules to rebalance data between the pools, moving the hottest data to the platinum pool until it is 70% full, then moving the next hottest data to the gold pool until it is 80% full, then the silver and finally all remaining data to the offline bronze pool.

/* Define an external pool for the off-line storage */
RULE EXTERNAL POOL 'bronze' EXEC ''

/* Define pool group using three on-line pools and the external off-line pool. */
RULE 'DefineTiers' GROUP POOL 'TIERS'
                   IS 'platinum' LIMIT(70)
                   THEN 'gold' LIMIT(80)
                   THEN 'silver' LIMIT(90)
                   THEN 'bronze'

RULE 'Rebalance' MIGRATE FROM POOL 'TIERS' TO POOL 'TIERS' WEIGHT(FILE_HEAT)
RULE 'Ingest' MIGRATE FROM POOL 'system' TO POOL 'TIERS' WEIGHT(FILE_HEAT)

See the mmapplypolicy-fileheat.sample script in /usr/lpp/mmfs/samples/ilm/README.