Topic
  • 4 replies
  • Latest Post - ‏2012-07-10T17:58:15Z by daleto
SystemAdmin
SystemAdmin
1188 Posts

Pinned topic TSM with MySQL Enterprise Backup (MEB 3.7.0)

‏2012-02-06T17:19:39Z |
A new security policy in our organization requires that backups no longer go to disk, with the preferred method now being directly to TSM. As a result, we have successfully migrated 200+ Oracle databases to use RMAN/TDPO and the next project phase is MySQL databases.

We have TSM (Version 5, release 5, level 2.7) installed on a MySQL 5 database server and have previously been using mysqldump to backup the databases first to disk and then to TSM. During a recent call with Oracle Sales & Support, we were informed that the 3.6.0 & 3.7.0 versions of MySQL Enterprise Backup (MEB) now provide Media Management Software support. While Oracle has not tested directly with TSM, the tech resources state that SBT functionality should be available.

My main question is whether anyone on the forum has integrated TSM with the latest version(s) of MEB?

Oracle Support has no configuration information available and have asked that we test a similar configuration as is used with RMAN/TDPO. My initial issue is that the libobk library is not installed on this database server, and after informing Oracle, they asked that I install the TDPO agent to get the library installed and reference it in a command similar to:

mysqlbackup --port=3307 --protocol=tcp --user=mebbackup --password --backup-image=sbt.backup-GAC-2012-02-06
--sbt-lib-path=/usr/bin/libobk.so --backup-dir=/backup backup-to-image

I'm not sure where the final location of the libobk library will be, but used /usr/lib for reference.

Even with the library installed, Oracle is unsure if we need to configure TSM opt files. We don't expect to get much feedback for this TSM-MySQL configuration and may simply have to start running backup tests, but we thought we would try the forums as they are always a source of great information.

Thanks,

Jeff
Updated on 2012-07-10T17:58:15Z at 2012-07-10T17:58:15Z by daleto
  • DavidMcClelland
    DavidMcClelland
    2 Posts

    Re: TSM with MySQL Enterprise Backup (MEB 3.7.0)

    ‏2012-02-07T14:08:50Z  
    Hi Jeff,

    Interesting post. I've a lot of experience working with Oracle backups and TSM and am doing some work with a customer right now to integrate some MySQL instances with TSM too.

    My first hunch is that what Oracle are suggesting to you won't work: I don't believe that trying to send MySQL backup data through the TDP for Oracle agent will be possible unless MEB is pretending to be an Oracle/RMAN source that will also replicate other RMAN-managed capabilities such as backup object expiration - from what I've seen of the MEB command syntax this doesn't appear to be the case.

    However, TSM does include a streaming input (i.e., it would go directly to the backup server/media without staging on disk) with its ADSMPipe interface using the TSM API client. You may need to roll your sleeves up and get stuck in to some code, but there are some examples online of how others have done this.

    For example, there's a great IBM RedPaper called "Backing Up Databases using ADSMPIPE
    and the TSM API: Examples Using Linux" at http://www.redbooks.ibm.com/redpapers/pdfs/redp3980.pdf which includes a section on backing up MySQL.

    Good luck - let us know how you get on, and what Oracle have to say (and whether they suggest trying Oracle Secure Backup...).

    Regards,
    ________________
    David Mc
    London, UK
    @davidmcclelland
  • SystemAdmin
    SystemAdmin
    1188 Posts

    Re: TSM with MySQL Enterprise Backup (MEB 3.7.0)

    ‏2012-02-07T16:21:01Z  
    Thank you for the prompt reply David. I have a copy of the ADSM Pipe Red Book on my desk, but with the hopes that we can still get TSM working as expected. We had originally purchased and tested the Repostor product, but were running into some issues with multiples instances on the same server.

    At about the same time, we were informed by Oracle Sales that MEB 3.7.0 provided "direct to TSM" functionality, no license fees and free Oracle Consultative Services support. I'm a little irked that Oracle/MySQL has stated they provide "support" for TSM, but have not tested against TSM Enterprise level infrastructure and their support teams seem uncertain as to how it should be configured. The Oracle resource I'm working with is great, but I believe she has not been provided the required details by the engineering teams, and that's assuming the product will work at all as expected. I've included the last three Oracle SR responses, and as you can see, it's a bit of a guessing game.

    My initial hope from installing the TDPO agent was to get the libobk.so library and required files installed. I'm no TSM expect and another team member has been involved with the Oracle/TSM migrations so I'm not really familiar with the MySQL-TSM interface. I was hoping that we were simply missing required libraries and the TDPO install was just the means to get them on the server. I understand the TDPO agent has RMAN-specific functionality, but I did not think we would be getting to that level. I though the mysqlbackup command line reference --sbt-lib-path=/.../libobk.so would suffice from the DBA side and then the TSM team would take over and configure what they required. I have so much to learn...

    Thanks again,

    Jeff

    =================================================

    MySQL Enterprise Backup (MEB) is supposed to work with TSM using SBT interface.

    Command line should look like: $MYSQLBACKUP --backup-image=sbt:<name> --sbt-lib-path=<path to sbt library> --sbt-database-name=<storage_locator> --backup-dir=<backup_dir> backup-to-image

    where sbt-database-name is the name of the database given to the SBT for it's database management/backup policies and sbt-lib-path is path to sbt library.

    We are actively testing it with Oracle software. TSM, afaik, is IBM Tivoli Storage Manager. Please confirm or reject.

    Since version 3.7 MEB also includes option sbt-environment which allows to specify variables needed by third-party software. If you could not run backup with command above you probably need to setup this option too.

    =================================================

    I just checked TSM user manual. I think you can use this page describing integration with "Data Protection for Oracle": http://publib.boulder.ibm.com/infocenter/tsminfo/v6r3/index.jsp?topic=%2Fcom.ibm.itsm.db.orc.doc%2Ft_dporc_quickcfg.html as start for setting TSM for MEB.

    Library seems to be /usr/lib/libobk64.a and you would probably need to set environment: --sbt-environment="TDPO_OPTFILE=/usr/tivoli/tsm/client/oracle/bin64/tdpo.opt"

    =================================================

    That TSM demo I talked about was not exactly product you use, but FastBackup. I could not find proper trial or demo version, so can not test it right now. We only can test this integration in your environment it seems.

    I also talked with MEB developers and they told library which should be provided for --sbt_lib_path option must contain sbtinit symbol. It must exist as a T (uppercase - text) symbol. You can check if one of libraries has this symbol if run "nm -p libname.so |less" or just binary grep all libraries provided with TSM for this symbol.

    I think would be better if you test with libobk from RMAN support first, then, if this does not work, try this solution.

    =================================================
  • DavidMcClelland
    DavidMcClelland
    2 Posts

    Re: TSM with MySQL Enterprise Backup (MEB 3.7.0)

    ‏2012-02-07T17:36:39Z  
    Thank you for the prompt reply David. I have a copy of the ADSM Pipe Red Book on my desk, but with the hopes that we can still get TSM working as expected. We had originally purchased and tested the Repostor product, but were running into some issues with multiples instances on the same server.

    At about the same time, we were informed by Oracle Sales that MEB 3.7.0 provided "direct to TSM" functionality, no license fees and free Oracle Consultative Services support. I'm a little irked that Oracle/MySQL has stated they provide "support" for TSM, but have not tested against TSM Enterprise level infrastructure and their support teams seem uncertain as to how it should be configured. The Oracle resource I'm working with is great, but I believe she has not been provided the required details by the engineering teams, and that's assuming the product will work at all as expected. I've included the last three Oracle SR responses, and as you can see, it's a bit of a guessing game.

    My initial hope from installing the TDPO agent was to get the libobk.so library and required files installed. I'm no TSM expect and another team member has been involved with the Oracle/TSM migrations so I'm not really familiar with the MySQL-TSM interface. I was hoping that we were simply missing required libraries and the TDPO install was just the means to get them on the server. I understand the TDPO agent has RMAN-specific functionality, but I did not think we would be getting to that level. I though the mysqlbackup command line reference --sbt-lib-path=/.../libobk.so would suffice from the DBA side and then the TSM team would take over and configure what they required. I have so much to learn...

    Thanks again,

    Jeff

    =================================================

    MySQL Enterprise Backup (MEB) is supposed to work with TSM using SBT interface.

    Command line should look like: $MYSQLBACKUP --backup-image=sbt:<name> --sbt-lib-path=<path to sbt library> --sbt-database-name=<storage_locator> --backup-dir=<backup_dir> backup-to-image

    where sbt-database-name is the name of the database given to the SBT for it's database management/backup policies and sbt-lib-path is path to sbt library.

    We are actively testing it with Oracle software. TSM, afaik, is IBM Tivoli Storage Manager. Please confirm or reject.

    Since version 3.7 MEB also includes option sbt-environment which allows to specify variables needed by third-party software. If you could not run backup with command above you probably need to setup this option too.

    =================================================

    I just checked TSM user manual. I think you can use this page describing integration with "Data Protection for Oracle": http://publib.boulder.ibm.com/infocenter/tsminfo/v6r3/index.jsp?topic=%2Fcom.ibm.itsm.db.orc.doc%2Ft_dporc_quickcfg.html as start for setting TSM for MEB.

    Library seems to be /usr/lib/libobk64.a and you would probably need to set environment: --sbt-environment="TDPO_OPTFILE=/usr/tivoli/tsm/client/oracle/bin64/tdpo.opt"

    =================================================

    That TSM demo I talked about was not exactly product you use, but FastBackup. I could not find proper trial or demo version, so can not test it right now. We only can test this integration in your environment it seems.

    I also talked with MEB developers and they told library which should be provided for --sbt_lib_path option must contain sbtinit symbol. It must exist as a T (uppercase - text) symbol. You can check if one of libraries has this symbol if run "nm -p libname.so |less" or just binary grep all libraries provided with TSM for this symbol.

    I think would be better if you test with libobk from RMAN support first, then, if this does not work, try this solution.

    =================================================
    Hi Jeff,

    If Oracle really is claiming support for MySQL backups to TSM with MEB and recommending using the TDP for Oracle agent then I'd like to understand how they maintain there should be no licensing involved (not to mention why IBM doesn't anywhere list MySQL as a supported agent backup, instead pointing customers at 3rd party solutions).

    Unfortunately, while there are TSM application agents/interfaces for other databases such as Oracle, MS SQL and DB2, there really is no MySQL/TSM integration at present, which is why there has been a rich market for 3rd party products from the likes of Repostor and Zmanda. As I'm sure you'll have seen from your research, other customers have simply used one of the common MySQL backup tools like AutoMySQLBackup and dumped their backup data to disk before sweeping up with TSM.

    I'd push Oracle for more information if the guys there insist integration with TSM this is supported. SBT is, I believe, Oracle's own specification, which is likely why they'd push you towards using it - it is then up to the 3rd party software supplier (e.g., IBM, Symantec etc) to interface its backup product with SBT. Interestingly, Symantec's NetBackup doesn't yet support MySQL in this way either - its 'NetBackup Agent for MySQL' is (not-so-very) under the covers the very same Zmanda ZRM product that is also available for TSM.

    Good luck, let us know how you get on

    _________________
    David Mc
    London, UK
    @davidmcclelland
  • daleto
    daleto
    1 Post

    Re: TSM with MySQL Enterprise Backup (MEB 3.7.0)

    ‏2012-07-10T17:58:15Z  
    Interesting article.
    But how do you deal with the transaction logs (bin-logs)?
    repostor's Data Protector for MySQL protect both databases, and all the transaction logs.
    And the restore is just one "click", or one command:
    (restore as far as possible)

    1. mysqlrestore -s yourdb -l

    or

    (restore to point in time)

    1. mysqlrestore -s yourdb -l -t YYYY-MM-DD.HH:MM:SS