Topic
16 replies Latest Post - ‏2013-03-21T10:04:49Z by SystemAdmin
SystemAdmin
SystemAdmin
2092 Posts
ACCEPTED ANSWER

Pinned topic GPFS with TSM as external storage pool

‏2013-02-20T14:07:40Z |
Hello everybody,

Right now we have a GPFS file system that contains files that are migrated to tape (managed by TSM) by a content management system. We are looking however to migrate the ILM rules from the application level towards the file system by using TSM as an external storage pool.

I do read posts where HSM is used on filesets in a gpfs file system and it works with the following policy in a test environment:

RULE 'TM1' MIGRATE
FOR FILESET('hsm_poc')
FROM POOL 'system'
THRESHOLD(90,70,50)
WEIGHT (weight_expression)
TO POOL 'hsm'
WHERE NOT (exclude_list)
AND NOT (is_migrated)
However if I look at the current known TSM v6.3 HSM client known problems (https://www-304.ibm.com/support/docview.wss?uid=swg21508282), there they state that filesets are not supported in HSM.

So does this mean that you cannot integrate TSM as an external storage pool on a GPFS file system with multiple filesets?
Updated on 2013-03-21T10:04:49Z at 2013-03-21T10:04:49Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    2092 Posts
    ACCEPTED ANSWER

    Re: GPFS with TSM as external storage pool

    ‏2013-02-20T14:30:32Z  in response to SystemAdmin
    Hmm, I might have answered my own question by looking to a more recent list of known problems:

    http://www-01.ibm.com/support/docview.wss?uid=swg21608424#linux_hsmprobs
    • SystemAdmin
      SystemAdmin
      2092 Posts
      ACCEPTED ANSWER

      Re: GPFS with TSM as external storage pool

      ‏2013-02-20T16:32:04Z  in response to SystemAdmin
      AFAIK TSM/HSM works okay with GPFS and filesets.

      The article you cited just gives the warning that moving a file between two different filesets in the same GPFS filesystem is treated by TSM/HSM as if you were moving the file between two different file systems.

      That is because the 'mv' command in that circumstance likewise "thinks" the two locations are in different filesystems and does a complete data copy.

      "Doctor, doctor it hurts when I `mv`!", saith the patient. The doctor responds, "Don't do that!"
      • HajoEhlers
        HajoEhlers
        251 Posts
        ACCEPTED ANSWER

        Re: GPFS with TSM as external storage pool

        ‏2013-02-21T18:49:17Z  in response to SystemAdmin
        Q: Are inodes on GPFS with filesets are globaly unique ?
        • ezhong
          ezhong
          32 Posts
          ACCEPTED ANSWER

          Re: GPFS with TSM as external storage pool

          ‏2013-02-21T19:27:56Z  in response to HajoEhlers
          Q: Are inodes on GPFS with filesets are globaly unique ?

          A: yes
          • SystemAdmin
            SystemAdmin
            2092 Posts
            ACCEPTED ANSWER

            Re: GPFS with TSM as external storage pool

            ‏2013-02-21T20:50:34Z  in response to ezhong
            To elaborate slightly,

            If inodeA == inodeB in filesystemC then inodeA and inodeB are the same file in C, regardless of any fileset considerations.
            Moreover, if you have created independent filesets (I like to call these inode-spaces), then some of the bits of the inode number identify the independent fileset.

            HOWEVER, if two "dependent" filesets CX and CY are in the same inode-space C(independent fileset C), then you can determine if a particular inode number z is in the inode-space C,
            but you can NOT determine if z is in CX or CY by merely examining the value of the inode number z. You would have to read the inode record number z...

            Clear?!
        • dlmcnabb
          dlmcnabb
          1012 Posts
          ACCEPTED ANSWER

          Re: GPFS with TSM as external storage pool

          ‏2013-02-21T22:31:41Z  in response to HajoEhlers
          I a filesystem (may include many filesets), an inode number and a generation number identify a unique file. When an inode is deleted, the inode number may be reused for a new file, and that file will have a new generation number.

          If you use snapshots, the snapshot file has the same inode number as the active file. But then if the active file is deleted and the inode reused for a new file, the generation numbers will diverge.
          • HajoEhlers
            HajoEhlers
            251 Posts
            ACCEPTED ANSWER

            Re: GPFS with TSM as external storage pool

            ‏2013-02-22T14:03:17Z  in response to dlmcnabb
            > inode number and a generation number identify a unique file

            Is that the reason why no hardlinks are allowed between filesets ?

            tia and a nice weekend for all
            Hajo
            • SystemAdmin
              SystemAdmin
              2092 Posts
              ACCEPTED ANSWER

              Re: GPFS with TSM as external storage pool

              ‏2013-02-22T15:00:40Z  in response to HajoEhlers
              hard links between filesets? Logically, that might make sense and "doable" within the context of GPFS. EXCEPT that fileset can be mm-unlinked which would leave a (sometimes) invalid hardlink. So another way the filesets are "kind of" like traditional Unix filesystems/volume (think mm-linkfileset kind of like Unix mount.)
              • SystemAdmin
                SystemAdmin
                2092 Posts
                ACCEPTED ANSWER

                Re: GPFS with TSM as external storage pool

                ‏2013-02-22T15:08:31Z  in response to SystemAdmin
                In short GPFS filesets are odd ducks. They are great if you need/want some of their special characteristics, otherwise I would fahgeddabout them.

                In GPFS filesets can "do":

                • Quota space management.
                • Share space on the same disk volumes and pools.
                • Independent snapshots of Independent file sets.
                • Fileset name is an attribute of each file, can factor into policy decisions.
                • Can be linked and unlinked so they can be made inaccessible (seem to disappear!) and later re-appear. Personally I think this is not such a hot idea and recommend against using it. It causes terrible problems with backups. But they didn't ask me ;-) when they added this feature ;-(
                • HajoEhlers
                  HajoEhlers
                  251 Posts
                  ACCEPTED ANSWER

                  Re: GPFS with TSM as external storage pool

                  ‏2013-02-22T15:38:05Z  in response to SystemAdmin
                  > It causes terrible problems with backups. But they didn't ask me ;-) when they added this feature ;-(

                  Its good that they didn't asked you _
                  I think its a nice feature but we have only terrible backup software ;-(

                  TSM could be enhance to determine via an option if filesets should be expiered if they are not online.
                  - In this case TSM would need to check for each deleteted directory if is a fileset link or determine the fileset links during startup and exclude them for the expiring.

                  My own pdsmc ( mentioned on some old thread ) will not expires fileset links ( since i do a final dsmc incr -subdir=no ) . I think i will add an option how to handle filesets in anyway.

                  Have a nice weekend
                  And thx to all
                  Hajo
            • dlmcnabb
              dlmcnabb
              1012 Posts
              ACCEPTED ANSWER

              Re: GPFS with TSM as external storage pool

              ‏2013-02-22T15:45:46Z  in response to HajoEhlers
              The biggest reason for no hardlinks between filesets is quota. Which fileset gets charged for the quota on a common file?
              • HajoEhlers
                HajoEhlers
                251 Posts
                ACCEPTED ANSWER

                Re: GPFS with TSM as external storage pool

                ‏2013-02-26T11:03:00Z  in response to dlmcnabb
                Good question i have no answer for.

                But maybe hardlinks could be used for a future fileset merge feauture. _

                thx
                Hajo
            • SystemAdmin
              SystemAdmin
              2092 Posts
              ACCEPTED ANSWER

              Re: GPFS with TSM as external storage pool

              ‏2013-02-22T16:59:49Z  in response to HajoEhlers
              A fileset is a self-contained subtree. A given file belongs to exactly one fileset. So, no, hard links between filesets are not possible.

              The basic idea of a fileset is to allow a certain level of data management with finer granularity (i.e. less than a full file system). A GPFS file system typically pools a considerable amount of storage together, and it is desirable to use the same large pool of storage for multiple users/projects/data types without having to create a separate file system for each managed entity and thus fragment storage into lots of tiny islands. A fileset provides a basic sub-file system unit of management. One can use filesets for things like managing quotas, storage pool data placement, and snapshots. So a logical way to think about a fileset is as a mini-file system inside a real file system (we've actually originally considered exposing filesets as individual file systems to the OS, with individual mount points and everything, but that was running into some OS-level limits and complications). Still, a given file can't be in two filesets, just like it can't be in two file systems simultaneously. Thus no cross-fileset hardlinks.

              yuri
  • sberman
    sberman
    59 Posts
    ACCEPTED ANSWER

    Re: GPFS with TSM as external storage pool

    ‏2013-02-20T16:05:18Z  in response to SystemAdmin
    Kurt,
    Yes, you can utilize TSM HSM as an external storage pool for migration using mmapplypolicy as you illustrated. Using mmapplypolicy and a policy rule as you have shown restricts the set of files being examined by mmapplypolicy to those within the FILESET('hsm_poc'). In this manner the use of the "fileset" construct is limited only to the GPFS interpretation inside mmapplypolicy, and TSM HSM is not specifically even aware of the fact that the files being migrated happen to all reside inside a fileset called 'hsm_poc'.

    My thanks to colleague Dominic for these additional notes
    Please note that filesets can have an impact to the HSM and backup processing in the following cases:

    Typically the move of a migrated file inside the file system has no impact to the migration state of the file nor it will be recalled. If a migrated file is moved from one fileset to another the file will be recalled.

    An unlinked fileset doesn't have a functional impact to the HSM reconciliation processing. But, it requires additional steps in the compare algorithm and decreases the overall reconcile performance. Note: It is highly recommended to use the policy based reconciliation in GPFS TSM environments. See the TSM Information Center for details.

    An unlinked fileset has impact to the backup of a file system. Both backup methodologies GPFS mmbackup and TSM progressive incremental will expire all files on the TSM server that are stored in the unlinked fileset.

    The relink of a fileset to another location will lead to a full backup of all files stored in the fileset in case of the next backup run. This is the case for both GPFS mmbackup and TSM progressive incremental.

    For more information regarding the integration between GPFS and TSM HSM see the following Tivoli field guide:

    http://www-01.ibm.com/support/docview.wss?uid=swg27018848&aid=1
  • SystemAdmin
    SystemAdmin
    2092 Posts
    ACCEPTED ANSWER

    Re: GPFS with TSM as external storage pool

    ‏2013-03-11T12:10:13Z  in response to SystemAdmin
    Thanks everybody for the answers and additional questions that gave some new insight too!
  • SystemAdmin
    SystemAdmin
    2092 Posts
    ACCEPTED ANSWER

    Re: GPFS with TSM as external storage pool

    ‏2013-03-21T10:04:49Z  in response to SystemAdmin
    Hello Everybody,

    I also set up a TSM-for Spacemanagement (HSM)-Service integrating a gpfs-filesystem with about 100 different filesets.
    After testing manually with various dsmmigrate-, dsmrecall-, dsmreconcile-jobs
    and adjusting appropriate TSM-service-,-HSM- and gpfs-settings,
    the HSM-tools worked without problems.
    Than I migrated "gpfs-policy-driven"
    about 60 TB of gpfs-data (located in different storage pools and filesets)
    => no problems also with the recall of part of it through "user-access".

    Applying the
    dsmreconcile-tool (that worked for the small data-ammount before)
    every evening at 19:02 o'clock ran into an error:

    Reconciling '/gpfs/airsat' file system:
    Querying the TSM server for a list of migrated files...
    (=> at 19:02 independent from the start)

    ANS8049E A connection with the server cannot be established due to a communications time-out.

    ANS9082W dsmreconcile: error encountered while reconciling file system /gpfs/airsat.

    => The reconcilation stopped

    changing different parameters in the dsm.sys (RECOncileinterval t0 / 9999 etc.)
    than restarting everytimes the TSM-service
    => same behaviour.

    Is there any parameter to adjust (server or client) ?
    Can anybody give me a hint on this behaviour and how to prevent it ?

    Thanks previously