Continuous availibility is the concept that enables 24/7 access to IT-enabled business functions, processes and applications.
And minimizing planned downtime is what business and IT departments consider seriously nowadays.
Especially, minimizing and setting the appropriate downtime is a bit challenging decision in case of DB2 version upgrade or migration to another system, because we also need to consider the time for getting back to original system in preparation for any unexpected problem after the planned change works.
I would like to give an idea in this blog for those kinds of cases which had been already proven and done from a very clever customer’s real migration project.
Typically, we use 'db2 upgrade' command or ‘export/import’ for the database migration. And once the database starts to operate in new system and version, it’s really hard and tricky to make a decision for reverting to previous system when there are some problems in new environment.
By using InfoSphere Replications Server (IRS), we can consider to try following steps and ideas in terms of reducing the downtime.
< While current systems is operating>
1. Configure the uni-directional mapping of all tables for both current system to new system and new system to current system.
New system to current system mapping will be used just in case that new system has some problems after the migration.
2. About 7 days before the new system open, start the IRS capture processes with cold start mode and stop quickly just to let DB2 identifies the transaction log timestamp and sequence number that IRS needs to catch up.
3. Export all table data and load to new systems.
At this step, we need to monitor the current system load and control the concurrent export works.
4. Start IRS to catch up the data up to current time.
As we marked the start time point in step 2, IRS will replicate the data by ignoring the data, which had been already moved to new system by export and load works.
If you worry about possible impact with IRS capturing to current system, we can choose to start the IRS only in the nighttime.
<Downtime for new system open>
5. For system open on new system, we just need to stop the business database systems and configure the applications servers to use new database system.
If you feel like checking the row count of all tables, you may need to take additional time depending on the elapsed time for counting the rows of the biggest tables.
<After new system is launched for real business application>
6. And the next step is just starting up the new database.
In preparation for possible significant problems, we can enable the IRS table mapping from new database system to old systems.
As old system catches up data transactions in new system by IRS, we can quickly revert to old system exactly same way in steps 5 and 6.
As we can see, this way provides the effective way to save the typical downtime with DB2 migration and easy way to get back to old system as well. Of course, you can continue to use the new system and remove the old system once all relevant departments give the confirmation that the new system runs normally.
If you want to consider this way for your upcoming migration project case, you can refer to the ‘Q Replication’ sections of DB2 Information Center for getting more details about IRS.
Infosphere Guardium support will end September 30, 2014, but customers may purchase a service extension.
Customers interested in continued support for these versions after the EOS date may purchase a Service Extension (SE) up to 2 years. The fee-based Service Extension provides technical support (e.g. defect corrections, restrictions, and bypasses) for new problems discovered after a product has been withdrawn from support. Fixes are provided at the "then current" corrective service level (Fix Pack, PTF, CSD, etc.). Conditions apply. Customers with current Subscription & Support contracts are eligible for an upgrade prior to the EOS date.
Customers interested in obtaining a Service Extension quote can contact their local sales person/team. We recommend requesting a quote early to avoid unnecessary delays in receiving support. Click this link for more information: https://ibm.biz/BdxPuN
Other Helpful links:
IBM Product Lifecycle: http://www.ibm.com/software/support/lifecycle/
IBM InfoSphere Guardium v9 Announcement:
IBM InfoSphere Guardium V8 to V9 upgrade:
Recorded tech talks for InfoSphere Guardium V9: http://bit.ly/guardwik
Serviceability improvements reduce the amount of time and effort that it takes to identify and resolve issues. It IOD next week, Support expert Jim Tsimis will demonstrate the serviceability features in the latest versions of InfoSphere CDC, Information Server & MDM products.
Jim will show you how serviceability tools, such as ISA Lite / ISA DC, can help you document and assess the health of your installation and simplify the task of collecting diagnostic information. You will also learn how to subscribe to notifications, search for known issues, manage service requests, and download Fix Packs and recommended patches that are applicable to your installation. Jim will also point you to product specific communities, developerWorks resources and Redbooks.
Demos are available in the InfoSphere Demo Room, Pedestal 20.
Known problems for InfoSphere Information Server, Version 9.1.2, are documented in APARs and technotes, which are available on the IBM Support Portal.
The following table provides links for customized queries for APARs and technotes for Version 9.1.2.
IBM InfoSphere Information Analyzer evaluates the content and structure of your data for consistency and quality. The following links contain answers to the frequently asked questions about installing, configuring, and running InfoSphere Information Analyzer.
Find the answers to frequently asked questions about installing IBM® InfoSphere® Information Analyzer such as how to correct an error about insufficient space and how to access the log files.
Where are the installation log files?
What should I name the IBM InfoSphere DataStage project during installation?
After installation, why am I unable to log into the IBM InfoSphere DataStage Director client?
Find the answers to frequently asked questions about configuring IBM® InfoSphere® Information Analyzer such as which manual steps you must complete after installation and how to set up an ODBC DSN.
How do I start InfoSphere Information Server on a UNIX or Linux platform?
The installer finished. Do I need to perform any manual steps to complete the installation?
How do I set up an ODBC DSN on UNIX or Linux?
How do I start or restart the ASBAgent?
How do I see if DB2 is running?
How do I stop or start DB2 on UNIX or Linux?
How do I change the locale of a UNIX or Linux server?
If the IBM InfoSphere DataStage server is installed on the same machine as that of the InfoSphere Information Server, how do I test the DSN that was created on the UNIX or Linux server?
Find the answers to frequently asked questions about uninstalling IBM® InfoSphere® Information Server, such as how to proceed if uninstallation fails.
How do I uninstall InfoSphere Information Server?
Uninstallation failed. What should I do now?
IBM InfoSphere Information Server 9.1 has been updated to version 9.1.2. This link describes the changes and enhancements in the 9.1.2 release, including updates to documentation. The software updates include enhancements to InfoSphere DataStage and QualityStage, InfoSphere Metadata Asset Manager, InfoSphere Business Glossary, and connectors. Documentation changes include improvements to information about InfoSphere DataStage, connectors, and stages.
Summary of fixes included in IBM InfoSphere Information Server, Version 9.1.2, with fixes grouped by components
InfoSphere Information Server, Version 9.1.2 includes fixes for product modules and components: ASB Packs, Business Glossary, Connectivity, Balance Optimizer, DataStage, DataStage Engine, DataStage Mainframe, DataStage Parallel Engine, Fast Track, Information Analyzer, Install, Information Services Director, Information Server Framework, Information Server Tools, Metadata Asset Manager and Metadata Integration Bridges, Metadata Workbench, Metadata Common Services, Quality Stage, XML Pack, DataClick and ODBC drivers
InfoSphere Information Server Version 9.1.2 is now available for Windows, AIX, Solaris, Linux for System Z, HP-UX Itanium and Redhat and SUSE Linux
To troubleshoot issues with IBM® InfoSphere® Information Server, you can consult log files and temporary files
Installation log files
Log files are created during the installation and uninstallation process. You can use the log files to troubleshoot installation errors.
Operating system Directory
AIX® HP-UX Solaris /var/tmp/ibm_is_logs
Temporary files generated by installation program
Temporary files are created during the installation process. The temporary files are not used after the installation is completed.
During the installation process, temporary files are saved to the following installation directories
Operating system Directory
AIX® HP-UX Solaris /var/tmp
IBM WebSphere Application Server system log files
The WebSphere® Application Server log files contain information that you can use to monitor WebSphere Application Server startup and diagnose errors.
The following log files are useful for diagnosing problems with IBM® InfoSphere® Information Server:
WebSphere Application Server messages to STDOUT are redirected to this file.
WebSphere Application Server messages to STDERR are redirected to this file.
These files are located in the following directories on each node in your WebSphere Application Server installation:
Operating system Directory
AIX® HP-UX Solaris path/profiles/profile/logs/server1
Path refers to WebSphere Application Server installation path. By default, path is one of the following paths:
Operating system Directory
AIX HP-UX Solaris opt/IBM/WebSphere/AppServer
The IBM InfoSphere DataStage and QualityStage Operations Console is a web application that allows the DataStage engine components of an InfoSphere Information Server installation to be monitored in real time. This provides a complete view of all DataStage job runs on that system, both present time and historic. It also includes the monitoring of key operating system metrics such as CPU usage, free memory and disk space.
We have gathered few published documents concerning this topic below for you :
Explore the IBM InfoCenter documentation for operations database schema.
Explore the IBM InfoCenter documentation for Operations Console.
Read the developerWorks article "Get started with the IBM InfoSphere DataStage and QualityStage Operations Console Database, Part 1: An introduction" (developerWorks, May 2012) to get a deep dive into the schema of the IBM InfoSphere DataStage and QualityStage Operations Database, and the tables and columns that make up its key relationships.
The developerWorks article "Configuring and using IBM InfoSphere DataStage and QualityStage Operations Console in a multiple-node or grid environment" (developerWorks, August 2012) describes how system information can be selected and displayed in the Operations Console UI.
The developerWorks article "Configuration and tuning guidelines for IBM InfoSphere DataStage Operations Console" (developerWorks, July 2012) provides configuration and tuning guidance on how to minimize the performance effect of the Operations Console on the system.
Watch the "InfoSphere DataStage Operations Console" demo of the Operations Console.
The IBM Information Management best practices present advice on the optimal way to use a variety of IBM Information Management products. They include technical papers, presentations, and articles authored by leading experts in IBM's development and consulting teams and are designed to give you practical advice to improve the value of your IBM data servers and data integration products.
Best practices for IBM InfoSphere Information Server
These best practices papers present advice on ways you can leverage InfoSphere Information Server to satisfy key business requirements for your information integration solutions. These articles are authored by leading experts in IBM's InfoSphere development and services teams. Each best practice paper is designed to provide practical guidance for common InfoSphere implementation scenarios. By applying these recommendations, you may improve the value of your InfoSphere solution and align yourself with IBM's technical direction for InfoSphere.
Best practices for DB2 for Linux, UNIX, and Windows and InfoSphere Warehouse
Discover the best practices for IBM® DB2® for Linux®, UNIX®, and Windows® and IBM InfoSphere Warehouse products. Get practical guidance for the most common DB2 product configurations and use this knowledge to improve the value of your DB2 data servers
Best practices for IBM® DB2® for z/OS
Discover the best practices for IBM® DB2® for z/OS®. Get practical guidance for the most common DB2 product configurations and use this knowledge to improve the value of your DB2 data servers
Discover the best practices for MDM
These best practices papers present advice on ways you can leverage IBM® InfoSphere™ MDM to satisfy key business requirements of master data management solutions. These articles are authored by leading experts in IBM's MDM development and services teams
Discover the best practices for IMS
These best practice papers present advice on the most optimal ways you can use IMS to satisfy key business data processing needs. Each best practice paper is designed to provide practical guidance for the most common IMS product configurations. By applying these recommendations, you may improve the value of your IMS data servers and align yourself with IBM's technical direction for IMS
IBM offers a number of information management related free ebooks that were written and published with the cooperation of subject matter experts and the community
Please see the some of published free ebooks :
Understanding Big Data :
In this book, the three defining characteristics of Big Data -- volume, variety, and velocity, are discussed. You'll get a primer on Hadoop and how IBM is 'hardening' it for the enterprise, and learn when to leverage IBM InfoSphere BigInsights (Big Data at rest) and IBM InfoSphere Streams (Big Data in motion) technologies. Deployment and scaling strategies plus industry use cases are also included in this practical guide. Review this book and get started with big data!
Getting Started with DB2 Express-C
This ebook is ideal for developers, consultants, ISVs, DBAs, students, or anyone who wants to get started with DB2. While this ebook focuses on DB2 Express-C , the free database edition of DB2, the concepts and content are equally applicable to other DB2 editions on Linux, UNIX, and Windows.
Getting Started with InfoSphere Data Architect :
InfoSphere Data Architect (IDA), formerly known as Rational Data Architect, is the premier tool from IBM for database design and more!. Learn how to use it in conjunction with DB2 Express-C, the no-charge edition of DB2. Since IDA is built on top of Eclipse, you can integrate it easily with other tools from IBM such as Data Studio also built using Eclipse technology. This book is for beginners in IDA and also in data modeling.
Getting Started with DB2 Application Development
DB2 Express-C from IBM is the no-charge edition of DB2 data server for managing relational and XML data with ease. DB2 Express-C runs on Windows®, Linux®, Solaris, and Mac OS X systems, and provides application drivers for a variety of programming languages and frameworks including C/C++, Java, .NET, Ruby on Rails, PHP, Perl, and Python. Review this book and get started with DB2 application development!
Data is one of the most valuable assets a company has. Relational and hierarchical databases have been used in the industry for decades. This book teaches you the fundamentals of databases, including relational database theory, logical and physical database design, and the SQL language. Advanced topics include using functions, stored procedures and XML. Use this book in conjunction with InfoSphere Data Architect software for database design, and with DB2 Express-C , the no-charge edition of the DB2 database server, and put your new knowledge into practice
While your business is growing, you might face some performance issues, that weren't there when you started using DataStage, or even worse, DataStage, has never been as fast as you thought it would be.
As follows, I'm sharing with you some tips to check in general and specific environments, maybe it's matter of changing a parameter or maybe you have to install a product Fix, but remember, it's always useful to test in a controlled environment, stress your environment, and review if modifications helped or not.
How to improve performance of IDocs outbound from SAP
Improving DataStage job performance on Solaris
InfoSphere DataStage: Parallel Job Performance Issue on Redhat Linux 6.1 System
Parallel Engine in IS 8.7 has poor performance creating lookup tables.
How can I improve performance during an import into Datastage?
Please, if you have any other performance question or specific environment, let us know and we'll help you to find a techdoc that might help you.
So, someone asked you to install InfoSphere DataStage Server for a new environment in your enterprise, or, to install the new DS version.
You, obviously, ask for the media (CD/DVD/zip) to install it, but, then you realize that you have to obtain it from wherever it's legal to obtain. Question now is... where? how?
Here are some links to download installation packages of InfoSphere DataStage Server 8.7 for different operating systems from IBM Passport Advantage.
So start downloading and install IS DataStage!!
InfoSphere DataStage Server, Version 8.7, for AIX
Download InfoSphere DataStage Server, Version 8.7, for HP-UX
InfoSphere DataStage Server, Version 8.7, for Linux
Download InfoSphere DataStage Server, Version 8.7, for Solaris
InfoSphere DataStage Server, Version 8.7, for Windows
Remember that in www.ibm.com, you can find the Information Center that include documentation for this and other versions.
There is a number of artifacts listed below that are requested by InfoSphere Information Server Support Team. Providing them during initial PMR creation facilitate issue investigation and troubleshooting of the reported problems
The requested items are:
1. Version.xml files for InfoSphere Information Server and Client machines
2. Operating systems Information of InfoSphere Information Server and Client machines
3. The detailed description of the problem and/or screenshots if relevant
4. If Datastage job related issue, the export of the job
To export the job:
Log in to Datastage Designer
Choose Export > Datastage Components on the top menu
Add items to export in Repository Export window
Choose directory to export to a file
Select preferred type (dsx) of the export
5. If Datastage related issue, export of Datastage job log
To produce the detail log file:
Log in to Datastage Director
Select the job-log on the menu
Select menu item View-Show All
Select menu item: Project-Print
Select options - All-Entries -- Full Detail- print a file.
6. In certain cases, InfoSphere Information Server Support Team might request ISALite diagnostic tools logs. For its usage,please refer to the link
Information Management Bootcamps are technical workshops in a classroom setting. They are held throughout the year in different regions worldwide. These bootcamps contain detailed lectures and labs for deep-dive learning of DB2® 10, Informix®, InfoSphere® products, Optim™, Guardium®, Big Data® and PureData®. Each bootcamp includes a free Technical Mastery Test and/or IBM Certification Exam
All available bootcamps listed in here
InfoSphere Informatiion Server specific Bootcamps are listed also below :
- InfoSphere Business Information Exchange Bootcamp info is available here
- InfoSphere Information Server for Data Integration Fundamentals Bootcamp info is available here
- InfoSphere Information Server for Data Integration Advanced Bootcamp info is here
- InfoSphere Information Server for Data Quality Fundamentals Bootcamp info is here
- InfoSphere Information Server for Data Quality Advanced Bootcamp info is here
There are few recent published technotes that address to different InfoSphere Information Server installation related issues :
- During InfoSphere Information Server 9.1 installation, users might get the message "The image properties file was not found...." In order to avoid this message,You need to download the proper IIS_V9.1_BUNDLE*.zip which contains the license key from the Passport Advantage site for your
purchased InfoSphere Information Server 9.1 product edition and unpack
it into your product media directory,e.g. 'is-suite' directory, please visit the related technote
- The Data Masking stage requires Optim Data Privacy Providers, which is a library to mask privacy data. To use the Data Masking stage, you must install Optim Data Privacy Providers on the InfoSphere Information Server engine tier. Refer to the technote that contains information about installing the IBM InfoSphere DataStage Data Masking stage and installing and configuring Optim Data Privacy Providers, version 9.1.
- Go to the technote to get to know about infoSphere Information Server, Version 9.1 installation issue on Solaris
Do you want to develop highly efficient and scalable information integration applications using IBM InfoSphere DataStage parallel jobs?
Then this Redbook from IBM expert practitioners with combined proven experience is a must read! InfoSphere DataStage Parallel Framework Standard Practices
This Redbook includes a comprehensive detailed description of the components available and descriptions on how to use them to obtain scalable and efficient solutions for both batch and real-time scenarios. The Redbook is intended for IT architects, Information Management specialists, and Information Integration specialists responsible for delivering cost-effective IBM InfoSphere DataStage performance on all platforms.
IBM InfoSphere Information Server, Version 8.7 Fix Pack 2 is available at IBM Support FixCentral.
IBM InfoSphere Information Server, Version 8.7 Fix Pack 2, includes updates to the following product modules and components: Business Glossary, DataStage, FastTrack, Information Analyzer, Information Services Director, Metadata Workbench, QualityStage, Connectivity, Information Server Framework, Balance Optimizer, Metadata Asset Manager, Metadata Integration Bridges, Metadata Common Services, Operational Metadata, XML Pack, and the DataDirect ODBC drivers.
IBM InfoSphere Information Server Unified Update Installer, Version 220.127.116.11 is available at IBM Support FixCentral
The Information Server Unified Update Installer is used to install fixes, fix packs, and updates to InfoSphere Information Server and its product modules. There is a separate download available for each Operating Systems
- Follow the online instructions to install and use the Update Installer. This Update Installer package is for versions 8.7 and 9.1 installed on Windows platforms.
- Follow the online instructions to install and use the Update Installer. This Update Installer package is for versions 8.7 and 9.1 installed on the AIX platform.
- Follow the online instructions to install and use the Update Installer. This Update Installer package is for versions 8.7 and 9.1 installed on the Linux x86 (32-bit) platforms.
- Follow the online instructions to install and use the Update Installer. This Update Installer package is for versions 8.7 and 9.1 installed on Linux x86_64 (64-bit) platforms.
- Follow the online instructions to install and use the Update Installer. This Update Installer package is for versions 8.7 and 9.1 installed on the Solaris platform.
- Follow the online instructions to install and use the Update Installer. This Update Installer package is for versions 8.7 and 9.1 installed on the zLinux platform.
- Follow the online instructions to install and use the Update Installer. This Update Installer package is for versions 8.7 and 9.1 installed on the HP (64-bit) Itanium platform.
IBM InfoSphere Data Explorer, Version 8.2-2 is available to download.
InfoSphere Data Explorer Version 8.2-2 includes the following updates:
• Data Explorer Engine uses Installation Manager 1.6.1, which provides a 64-bit version of Data Explorer Engine. It is therefore no longer necessary to install 32-bit libraries to use Installation manager on Linux systems.
• Data Explorer Engine now supports optionally displaying query results in original document formats, known as rich preview. Displaying rich preview results in a browser is not supported in Microsoft Internet Explorer (IE) 6.0, though all other Data Explorer Engine features are IE6-compatible. Using Rich Preview is described in the Data Explorer Engine Users Guide.
• The capabilities of the Data Explorer Application Builder administration tool have been substantially improved for this fix pack. New features and capabilities include:
◦ Added support for modular, graphical configuration of the schema that defines the resources, entities, and other fundamental information that is used by an Application Builder application
◦ Added support for setting the default theme used in applications developed using the Application Builder
◦ Added the ability to import and export the configuration information that is used by an Application Builder instance
◦ Added search configuration options including relevancy formula configuration
The release notes are available here.
Are you looking for new ways to grow your skills and knowledge around InfoSphere Guardium in 2013? Look no further than InfoSphere Guardium Tech Talks, available for a variety of topics throughout the year.
Tech Talks enable you to interact directly with product experts from the labs in a low key, informal setting. Experts will vary with each talk, and will include product management, support analysts, development, QA, and lab services.
The next Tech Talk will take place on March 14 at 11:30am Eastern, 8:30am Pacific
. The topic for this talk is "Roadmap to a successful V9 upgrade"
and speakers are Vlad Langman
, L3 Support Manager, and Abdiel Santos
, L3 Support Specialist. These support experts will provide you with a range of tips and advice when planning your upgrade to InfoSphere Guardium V9.
Register for the March 14 talk and listen to recordings of past talks in the InfoSphere Guardium Community on developerWorks
IBM InfoSphere Foundation Tools Components, Version 8.1.2 Fix Pack 5 is available to download at the link
InfoSphere Foundation Tools components, Version 8.1.2 Fix Pack 5 includes updates to the following Version 8.1.2 installations of:
- IBM® InfoSphere™ Business Glossary
- IBM InfoSphere FastTrack
- IBM InfoSphere Information Analyzer
- IBM InfoSphere Metadata Workbench
Version 8.1.2 Fix Pack 5 also includes updates for IBM InfoSphere Business Glossary Anywhere and IBM InfoSphere Business Glossary Client for Eclipse
There are two recently published InfoSphere Information Server related Education Assistant modules
The first module
points to an IBM Education Assistant module about starting node agents as a non-root user
- Common Issues
The second module
points to an IBM Education Assistant module about how to import Infosphere Data Architect Models using IBM InfoSphere Metadata Asset Manager in Information Server version 8.7
- Logical and Physical Data Models
- Configuring InfoSphere Metadata Asset Manager
- Importing InfoSphere Data Architect Model in InfoSphere Metadata Asset Manager
- Re-importing InfoSphere Data Model
- Command Line Import of InfoSphere Data Model
Importing large business glossary files, such as the XML files provided with IBM Industry Models, might fail due to exceeding the timeout values set in WebSphere Application Server and InfoSphere Information Server. You can change these values so that the import succeeds. Please refer to the technote
for more details
IBM InfoSphere Information Server, Version 9.1 fix list can be accessed at the link
That shows the summary of fixes included in the IBM InfoSphere Information Server, Version 9.1 release
There are also InfoSphere Information Server Version 9.1 fixes that require post-installation steps at the link
That shows certain APAR fixes require post-installation steps to use the fixes
The osh command is the main program of the InfoSphere Parallel Engine.
This command is used by DataStage to perform several different tasks
including parallel job execution and dataset management. Normally, there
is no need to run this command directly but sometimes it is useful to
use it for troubleshooting purpose
You can access to that technote
that describes how to run the osh command of the InfoSphere Parallel Engine in InfoSphere Information Server product
Check out very nice technote
that explains how to create a Rule in Information Analyzer that joins a table to itself by creating a virtual table
Information Analyzer roll-up patch 4 for Information Server Version 8.7, Fix Pack 1 has been released
This technote describes how the number of jobs that a DataStage project can have may be impacted by Operating Systems' limitations. When you reach the maximum number of folders the OS allows then you won't be able to create new jobs. Please refer to the technote
that lists some suggestions about how to address this problem.
This a great resource to understand how to use APT_DUMP_SCORE and how to read scores in parallel jobs. This document written by the Development team uses a example to show the sections of a typical score and includes descriptions for every section, partition and operator notation, and even suggested self-training excercises
Do you want to learn more about InfoSphere Streams and Big Data? Take a look at Roch Achambualt’s IBM InfoSphere Streams 2.0 presentation. The presentation introduces Streams. It provides Streams Application examples along with an overview Streams Processing Language and Streams Toolkits. The presentation provides links to the following:
Streams Wiki an online information hub for InfoSphere Streams users
Streams Exchange a place for Streams application developers to share ideas, source code, design patterns, experience reports and rules of thumb
Click here Roch Achambualt's presentation.