Topic
  • 4 replies
  • Latest Post - ‏2014-01-26T20:08:53Z by René Hamberg
JohnI_
JohnI_
1 Post

Pinned topic CM - TSM - slow performance

‏2012-09-06T11:28:06Z |
I have a migration policy in CM to move the files after 30 days to TSM server (on tapes).
In CM i have over 20 milion files, small files (under 200 KB).
Everything is working but is very slow, only 3000 files/hour.
The systems are used below 10%.

Help !
  • René Hamberg
    René Hamberg
    3 Posts

    Re: CM - TSM - slow performance

    ‏2014-01-13T10:48:42Z  

    Unfortunately I don't have an answer but exactly the same problem.

    Anyone with an idea what might be the problem? We're using a setup on AIX where both LPARs (the one for CM and the one for TSM) are running on the same physical machine and communicating via an internal network. But still only 1 object every 2.5 sec. is migrated. What draws attention is that for every object the migrator seems to do a complete connect/move/disconnect cycle. This obviously won't speed things up.

    Anyone with similar experiences and a solution to this problem?

  • MP2K_George_Silva
    MP2K_George_Silva
    1 Post

    Re: CM - TSM - slow performance

    ‏2014-01-21T20:00:59Z  

    Unfortunately I don't have an answer but exactly the same problem.

    Anyone with an idea what might be the problem? We're using a setup on AIX where both LPARs (the one for CM and the one for TSM) are running on the same physical machine and communicating via an internal network. But still only 1 object every 2.5 sec. is migrated. What draws attention is that for every object the migrator seems to do a complete connect/move/disconnect cycle. This obviously won't speed things up.

    Anyone with similar experiences and a solution to this problem?

    There are many factors which may affect migration performance, and little information presented here to about your specific CM/TSM releases, configuration, content, or migration scheme.

    Below are a few options to consider, but availability of some of these options is limited based on TSM or CM releases.

    * Consider use of the TSMPOOLED device manager can help reduce session processor usage for Tivoli Storage Manager connections. You can enable Tivoli Storage Manager connection pooling for resource managers that are not in z/OS. Content Manager 8.4.1 and later offers connection pooling feature in the Resource Manager.

    * Consider enabling aggregation for smaller objects. Aggregation can only be enabled and configured if the Tivoli Storage Manager server is retention-enabled. Starting in IBM Content Manager 8.3 fix pack 1, you may use the Resource Manager to aggregate objects when migrating storage devices to TSM  that are managed with archive retention-protection enabled.  Instead of sending objects or e-mail to TSM Archive as a single object, Content Manager bundles objects depending on the aggregate size forming one aggregate or object. This aggregate is then archived to TSM.

    *Tune Migrator threads: For increased performance when archiving data to TSM, increase the number of work manager threads available to do work. Consideration needs to be made for memory and processor utilization such that Resource Manager machine is not overloaded.  

    * Tune Migrator batch size: Each Resource Manager batch size option sets or controls the number of objects processed, increasing the batch size reduces the number of table scans that are needed by the Resource Manager for each cycle it runs

    Of course with any change, you would need to test and changes to your configuration thoroughly before applying to production systems. Ultimately, if performance is not satisfactory I would suggest you bring this concern to attention of IBM support.   
     

  • René Hamberg
    René Hamberg
    3 Posts

    Re: CM - TSM - slow performance

    ‏2014-01-22T08:57:04Z  

    There are many factors which may affect migration performance, and little information presented here to about your specific CM/TSM releases, configuration, content, or migration scheme.

    Below are a few options to consider, but availability of some of these options is limited based on TSM or CM releases.

    * Consider use of the TSMPOOLED device manager can help reduce session processor usage for Tivoli Storage Manager connections. You can enable Tivoli Storage Manager connection pooling for resource managers that are not in z/OS. Content Manager 8.4.1 and later offers connection pooling feature in the Resource Manager.

    * Consider enabling aggregation for smaller objects. Aggregation can only be enabled and configured if the Tivoli Storage Manager server is retention-enabled. Starting in IBM Content Manager 8.3 fix pack 1, you may use the Resource Manager to aggregate objects when migrating storage devices to TSM  that are managed with archive retention-protection enabled.  Instead of sending objects or e-mail to TSM Archive as a single object, Content Manager bundles objects depending on the aggregate size forming one aggregate or object. This aggregate is then archived to TSM.

    *Tune Migrator threads: For increased performance when archiving data to TSM, increase the number of work manager threads available to do work. Consideration needs to be made for memory and processor utilization such that Resource Manager machine is not overloaded.  

    * Tune Migrator batch size: Each Resource Manager batch size option sets or controls the number of objects processed, increasing the batch size reduces the number of table scans that are needed by the Resource Manager for each cycle it runs

    Of course with any change, you would need to test and changes to your configuration thoroughly before applying to production systems. Ultimately, if performance is not satisfactory I would suggest you bring this concern to attention of IBM support.   
     

    Thanks for the heads-up. I found an article describing lots of tuning parameters you're referring to and which we may want to play with (https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Storage%20Manager/page/IBM%20Content%20Manager%2C%20System%20Storage%20Archive%20Manager%2C%20and%20SnapLock%20Integrations%20--%20Retention%20and%20Expiration%20Management

    Also this (http://www-01.ibm.com/support/docview.wss?uid=swg21413542) article hints at reasons why in our case (CM 8.4.3.3 with TSM 6.3.4) performance might be sluggish. We'll start looking into the number of worker threads and the use of TSMSPOOLED (http://www-01.ibm.com/support/docview.wss?rs=86&uid=swg27016385).

  • René Hamberg
    René Hamberg
    3 Posts

    Re: CM - TSM - slow performance

    ‏2014-01-26T20:08:53Z  

    There are many factors which may affect migration performance, and little information presented here to about your specific CM/TSM releases, configuration, content, or migration scheme.

    Below are a few options to consider, but availability of some of these options is limited based on TSM or CM releases.

    * Consider use of the TSMPOOLED device manager can help reduce session processor usage for Tivoli Storage Manager connections. You can enable Tivoli Storage Manager connection pooling for resource managers that are not in z/OS. Content Manager 8.4.1 and later offers connection pooling feature in the Resource Manager.

    * Consider enabling aggregation for smaller objects. Aggregation can only be enabled and configured if the Tivoli Storage Manager server is retention-enabled. Starting in IBM Content Manager 8.3 fix pack 1, you may use the Resource Manager to aggregate objects when migrating storage devices to TSM  that are managed with archive retention-protection enabled.  Instead of sending objects or e-mail to TSM Archive as a single object, Content Manager bundles objects depending on the aggregate size forming one aggregate or object. This aggregate is then archived to TSM.

    *Tune Migrator threads: For increased performance when archiving data to TSM, increase the number of work manager threads available to do work. Consideration needs to be made for memory and processor utilization such that Resource Manager machine is not overloaded.  

    * Tune Migrator batch size: Each Resource Manager batch size option sets or controls the number of objects processed, increasing the batch size reduces the number of table scans that are needed by the Resource Manager for each cycle it runs

    Of course with any change, you would need to test and changes to your configuration thoroughly before applying to production systems. Ultimately, if performance is not satisfactory I would suggest you bring this concern to attention of IBM support.   
     

    Solved it! Using TSMPOOLED did the trick in our case.

    This seems to be one of the better hidden options in CM. Oh well, at least we hope to migrate our couple of terabytes in a month now as opposed to 3 years :-)

    Thanks anyway for pointing me in the right direction.