Topic
7 replies Latest Post - ‏2012-12-12T21:58:22Z by SystemAdmin
SystemAdmin
SystemAdmin
2092 Posts
ACCEPTED ANSWER

Pinned topic Migration based on GPFS fileset policies (quotas)

‏2012-12-10T16:46:12Z |
Hello,

This is my GPFS fileset policies (as set up by an IBM consultant.) HSM is TSM 6.3.0, running on server tsm1.

root@gpfs1 ~# mmlspolicy lb -L
rule 'default' SET POOL 'system'
rule 'f' external list 'f'
exec '/tmp/exec.hsm'
threshold ('fileset_quotas')
rule 'f2' list 'f' threshold(85,65,0)
for fileset('robot')
show(fileset_name)
weight(CURRENT_TIMESTAMP-ACCESS_TIME)
rule 'f3' list 'f' threshold(95,65,0)
for fileset('project')
show(fileset_name)
weight(CURRENT_TIMESTAMP-ACCESS_TIME)
RULE EXTERNAL POOL 'hsm' EXEC '/tmp/exec.hsm' OPTS '-server=tsm1'
RULE 'migrate' MIGRATE TO POOL 'hsm'
FOR FILESET('robot','project')

My understanding is that rule f2 specifies that if fileset "robot" reaches 85% full, files will be migrated until the fileset reaches 65% full status.

However, my fileset is nowhere near 85% full (and never has been):

robot FILESET 27 235 235 1 none | 1468193 0 0 0 none

But still files have been migrated. Is my GPFS policy correct?

Thanks,

Andras
Updated on 2012-12-12T21:58:22Z at 2012-12-12T21:58:22Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    2092 Posts
    ACCEPTED ANSWER

    Re: Migration based on GPFS fileset policies (quotas)

    ‏2012-12-10T16:49:28Z  in response to SystemAdmin
    The last line should read :

    root@gpfs1 ~# mmrepquota --block-size=1T lb | grep robot
    robot FILESET 27 235 235 1 none | 1468193 0 0 0 none

    So 27 TB used (on disk) in a fileset that is 235 TB in size.

    Andras
    • SystemAdmin
      SystemAdmin
      2092 Posts
      ACCEPTED ANSWER

      Re: Migration based on GPFS fileset policies (quotas)

      ‏2012-12-10T17:16:47Z  in response to SystemAdmin
      I believe the problem is the rule:

      RULE 'migrate' MIGRATE TO POOL 'hsm'

      Which works somewhat independently of your 'LIST' rules.

      In fact, that single rule will cause ALL FILES to be migrate to 'hsm', which is probably NOT what you want!

      You could eliminate that rule.
      OR, if you want a rule that is based on GPFS pool occupancy rather than fileset_quotas, you could amend the rule by adding a
      THRESHOLD(98,90,80) clause or similar.

      To avoid confusion, you should also amend the other rules to use THRESHOLD(85,65) without that last ,0. The third threshold number is meaningless for LIST rules,
      but for MIGRATE rules, controls pre-migration.

      For example

      RULE ... MIGRATE ... THRESHOLD(0,100,0) causes all files to be premigrated. RTFM and carefully!
  • SystemAdmin
    SystemAdmin
    2092 Posts
    ACCEPTED ANSWER

    Re: Migration based on GPFS fileset policies (quotas)

    ‏2012-12-10T18:35:50Z  in response to SystemAdmin
    Marc, you are right. I don't want to migrate everything, but I want to migrate based on fileset quotas. Here are my revised policies :

    
    rule 
    'default' SET POOL 
    'system' rule 
    'f' external list 
    'f' exec 
    '/tmp/exec.hsm' threshold (
    'fileset_quotas') rule 
    'f2' list 
    'f' threshold(85,65) 
    
    for fileset(
    'robot') show(fileset_name) weight(CURRENT_TIMESTAMP-ACCESS_TIME) rule 
    'f3' list 
    'f' threshold(95,65) 
    
    for fileset(
    'project') show(fileset_name) weight(CURRENT_TIMESTAMP-ACCESS_TIME) RULE EXTERNAL POOL 
    'hsm' EXEC 
    '/tmp/exec.hsm' OPTS 
    '-server=tsm1'
    


    However, my rule f2 is no longer linked to external pool 'hsm', and it no longer specifies the keyword 'MIGRATE' to the script /tmp/exec.hsm . Would this work?

    Thanks,

    Andras
    • SystemAdmin
      SystemAdmin
      2092 Posts
      ACCEPTED ANSWER

      Re: Migration based on GPFS fileset policies (quotas)

      ‏2012-12-10T20:28:42Z  in response to SystemAdmin
      The MIGRATE rule is special, it "understands" hsm/dmapi and the notions of migration and premigration, but because that's already a lot, it does not "understand" quotas, only pool utilization.

      The LIST rule is generic - you can use it and an associated EXEC 'program' for just about any purpose you like, including migration - which is what you are doing here, ASSUMING /tmp/exec.hsm does that.

      But two more things I notice are wrong:

      a) /tmp/exec.hsm: Putting an important script in /tmp is almost certainly a bad idea. We should really look at the code in /tmp/exec.hsm to see what it is doing. And then move it to some "permanent" location. Also, it should be either in file system that is accessible from each node in your gpfs cluster -- or -- you will need to copy it to each node in the cluster -- which is then a liability when the time comes to make some changes.

      b) The list rules you have will select files that have already been migrated. You probably want to add a clause like this
      WHERE MISC_ATTRIBUTES NOT LIKE '%V%'

      Consider other conditions, such as you want to avoid migrating small files...

      AND KB_ALLOCATED > 8

      And so forth, RTFM!
      • SystemAdmin
        SystemAdmin
        2092 Posts
        ACCEPTED ANSWER

        Re: Migration based on GPFS fileset policies (quotas)

        ‏2012-12-12T14:33:22Z  in response to SystemAdmin
        Thanks.

        a) Yes, /tmp is not a good place. I'll put it in /root instead, and run it from a single node (vi a cron : mmapplypolicy). exec.hsm script is the sample script /usr/lpp/mmfs/samples/ilm/mmpolicyExec-hsm.sample . (I'll include it in the attachment.)

        b) V stands from read-managed? It means it has already been migrated?

        I already set the stubsize to 4 MB in Tivoli, so it takes care about that.

        So my policy looks like this :

        rule 'default' SET POOL 'system'
        rule 'f' external list 'f'
        exec '/root/exec.hsm'
        threshold ('fileset_quotas')
        rule 'f2' list 'f' threshold(85,65)
        for fileset('robot')
        show(fileset_name)
        weight(CURRENT_TIMESTAMP-ACCESS_TIME)
        WHERE MISC_ATTRIBUTES NOT LIKE '%V%'
        rule 'f3' list 'f' threshold(95,65)
        for fileset('project')
        show(fileset_name)
        weight(CURRENT_TIMESTAMP-ACCESS_TIME)
        WHERE MISC_ATTRIBUTES NOT LIKE '%V%'
        RULE EXTERNAL POOL 'hsm' EXEC '/root/exec.hsm' OPTS '-server=tsm1'

        Would this work?

        Thanks,

        Andras

        Attachments

        • SystemAdmin
          SystemAdmin
          2092 Posts
          ACCEPTED ANSWER

          Re: Migration based on GPFS fileset policies (quotas)

          ‏2012-12-12T21:56:42Z  in response to SystemAdmin
          4MB stub size means Tivoli will not migrate any file <=4MB and moreover will leave the first 4MB "online" so an app can read the first 4MB without suffering recall delay.

          BUT the policy LIST rule knows nothing about that so your rules will uselessly select small files and ask TSM to migrate them.

          So I still think you will want to add AND KB_ALLOCATED > 4*1024*1024
        • SystemAdmin
          SystemAdmin
          2092 Posts
          ACCEPTED ANSWER

          Re: Migration based on GPFS fileset policies (quotas)

          ‏2012-12-12T21:58:22Z  in response to SystemAdmin
          Correct, "read-managed" (V) means HSM/TSM must intervene on read requests, so that the data can be retrieved and brought back online.