Basic Initial Load installation

Step 1. Prepare the file systems

File systems should be allocated following established best practice methodology for InfoSphere Information Server (source files, target files, datasets, and scratch/sort work areas).

Move the contents of the Java/lib directory to $DSHOME/java/lib, where $DSHOME is the home directory of your DataStage installation.

The Java/lib directory holds specific Java™ library (JAR) assets used in the generation of phonetic values. An example of the $DSHOME/java/lib directory is /opt/IBM/InformationServer/Server/DSEngine/java/lib.

Create the following directories specifically for use with BIL. Place them in the directory structure /home/dsadm/<your BIL DataStage project name>/.

Job parameter values will be set later to point to these directories.

Ensure that the directory $APT_ORCHHOME/user_lib exists. If not, create it with proper security.

Step 2. Create the InfoSphere DataStage project for BIL

Create an InfoSphere DataStage® project using DataStage Administrator, following any preferred or required naming standard. In the absence of a standard, a suggested name is MDMIS_BIL_R10.

Step 3. Verifying path requirements

Ensure that the project LIBPATH or LD_LIBRARY_PATH include the following paths:

• $APT_ORCHHOME/lib—Required to run the orchadmin utility from a Job Sequence
• $APT_ORCHHOME/user_lib—Required to run the custom operators to be installed.

Ensure that the project PATH includes the following paths

• $APT_ORCHHOME/bin—Required to run the orchadmin utility from Job Sequence.

Step 4. Install the BIL custom routine and Java libraries

Operators are located in the asset tarball in the following directories:

• AIX®: Information Server 8.5 – MDMIS-aix/CAB/PXEngine/user_lib64/IIS8.5

This release contains one routine:

• UTCCvt.o

The routine should be installed in the following location: $APT_ORCHHOME/user_lib (such as /opt/IBM/InformationServer/Server/PXEngine/user_lib)

Append the contents of CAB/PXEngine/etc/append_to_operator.apt to $APT_ORCHHOME/etc/operator.apt (make a backup and then edit operator.apt).

Verify that LIBPATH or LD_LIBRARY_PATH contain $APT_ORCHHOME/user_lib.

The contents of the Java/lib folder (see the BIL Manifest description) should be copied to -. The resulting folder will consist of 5 jar files and 2 property files: common-codec-1.3, icu4j, Linguistic, Logging, PhoneticGen, JDKLog.properties, and Log4J.properties.

Step 5. Install the clean up script into the ARCHIVE directory

The clean up script (DSClean.sh) is located in the asset tarball in the SCRIPT directory. Copy the DSClean.sh script into the ARCHIVE directory. Verify that the execution permissions are set. Verify that the system supports bash shell scripting. If the system utilizes korn shell scripting (i.e. Solaris) it is will be necessary to change the shell scripting declaration (#!/bin/sh) to appropriately reflect korn shell (#!/bin/ksh). Once scripting has been verified insure that the Cleanup_Env execute command activity stage in BL_000_BASIC_LOAD refers appropriately to DSClean.sh.

Step 6. Import the BIL InfoSphere DataStage jobs to the new project

Using the InfoSphere DataStage client, import the BIL jobs into the newly-created InfoSphere DataStage project. The BIL jobs are contained in ZIP files within the BIL tarball, which will unzip into *.dsx (InfoSphere DataStage eXport) files.

The MDMIS-BIL.zip file contains the main *.dsx file that should be installed.

BIL supports four options regarding database loading that are dependent on the values set by the following job parameters:

• MDMIS_DB_DATABASE_TYPE
• DB_DATABASE_VARIANT

Depending on customer requirements, any one of the following database options can be chosen:

• ODBC – default
• DB2® – can be configured for bulk loading
• Oracle

Step 7. Set up the ODBC/DB2/Oracle configuration for connection to the database

Setup and configure the appropriate connection type for the InfoSphere MDM database on the InfoSphere Information Server.

For ODBC connections the files $DSHOME/uvodbc.config and $DSHOME/.odbc.ini will need to be modified to include definitions for the appropriate databases. The ODBCINI environment variable should point to $DSHOME/.odbc.ini.

For Oracle, appropriate Oracle client software must be installed on the InfoSphere Information Server server and the ORACLE_HOME environment variable must point to the Oracle installation directory. PATH should include $ORACLE_HOME/bin, LIBPATH, or LD_LIBRARY_PATH should point to the appropriate 32-bit or 64-bit Oracle libraries. A typical ORACLE_HOME value might be /u01/app/oracle/product/11.2.0. If you are using TNSNAMES for connection information, TNS_ADMIN should point to the directory containing the file tnsnames.ora. The ORACLE_HOME, PATH, and load library path variables must be configured in the dsenv file.

For DB2, appropriate DB2 client software must be installed on the InfoSphere Information Server machine. The DB2DIR environment variable should point to the DB2 installation directory, DB2INSTANCE should contain the appropriate instance name, PATH and LIBPATH/LD_LIBRARY_PATH should point to the appropriate DB2 bin and lib folders.

If the database is a DB2 for z/OS® instance, you must configure the DB2 Connector stage in the BLDBINHISTORY and BLDBINTABLE database shared containers so that the Partitioning tab Partition Type setting is set to "(AUTO)".

If you wish to use the bulk loading settings for either DB2 or Oracle, then you must make corresponding changes in the related database shared containers BLDBINTABLE and BLDBINHISTORY to the DRS stage.

Step 8. Test the database connection

A temporary InfoSphere DataStage job can be created to confirm that the server can connect to the InfoSphere MDM database, or other methods such as importing table definitions can be employed. Command-line tools such as sqlplus for Oracle or db2 for DB2 can also be used to verify connectivity from the server.

Step 9. Create the InfoSphere MDM default configuration files

MDM_1X1.apt and MDM_Default.apt are provided in the MDMIS tarball (see the earlier manifest for their exact locations by operating system). Copy these files to the $APT_ORCHHOME/../Configurations directory on the InfoSphere Information Server machine and modify as required for the specific server installation.

The distributed configuration files must be edited to reflect not only the existing environment but also the availability of system resources. While the default configuration file presents a typical two node configuration, you must consider the existing system resources. The default configuration is not appropriate for all environments.

The following sample configuration file includes placeholder values that should be changed. Additional configuration files can be created as desired. Additional configuration options may be added to support a particular system configuration, such as separate sort and scratch space allocations.

MDM_1X1.apt:
{
	node "node1"
	{
	    fastname "<servername>"
	    pools ""
	    resource disk "<resource disk path>" {pools ""}
	    resource scratchdisk "<resource scratchdisk path>" {pools ""}
	}
}
MDM_Default.apt
{
	node "node1"
	{
	   fastname "[servername]"
	   pools ""
	   resource disk "<resource disk path>" {pools ""}
	   resource scratchdisk "<resource scratchdisk path>" {pools ""}
	}
	
	node "node2"
	{
	   fastname "[servername]"
	   pools ""
	   resource disk "<resource disk path>" {pools ""}
	   resource scratchdisk "<resource scratchdisk path>" {pools ""}
	}
}

Step 10. Update the MDMIS parameter set

Set the default values for the MDMIS set’s parameters. The parameters in MDMIS are broken down into several categories, identified by the parameter name prefix:

• DB – Database-related parameters, such as connection information
• DS – DataStage-related parameters, such as job control and date format
• FS – File system-related parameters, such as SIF file location
• SK – Surrogate Key-related parameters, such as file location and format

The following parameters should be reviewed for advanced configuration:

Step 11. (OPTIONAL) Enable BIL for NLS usage

By default, BIL does not support the use of NLS data (ustring datatype). Therefore, multi-byte characters sets are not recognized as valid character data during processing. In order to support NLS data, BIL jobs must be modified, patches applied to InfoSphere Information Server and InfoSphere MDM table sizes must be increased.

NLSTool for BIL, available through the secure FTP site where BIL assets are accessible, can be used to "NLS-Enable" the BIL InfoSphere DataStage assets. For details about how to configure BIL for supporting NLS data, please consult the NLS documentation that comes with the NLSTool.

For information on how to obtain the NLSTool, please contact the Services District Practice Manager.

Step 12. Compile the Parallel and Sequencer jobs

After the configurations have been completed, the BIL parallel and sequencer jobs can be compiled. While jobs can be compiled individually, due to the number of jobs contained in BIL, it is recommended that the Multiple Job Compile tool be used. This is available within the DataStage Designer Tools menu: Tools > Multiple Job Compile.

Step 13. Create the Surrogate Key state file

The BIL jobs make use of the InfoSphere DataStage Surrogate Key stage to generate both temporary and persistent keys utilized within BIL and InfoSphere MDM processing. The Surrogate Key stage uses a “state” file to regulate the assignment of these keys both during a job and from one job run to another.

Users must create the required surrogate key state files at least once prior to running the BIL for the first time.

In order to do that, the Job Sequencer BL_061__AI_SK_State_File_Control should be run. Default values for parameters can be used at submission time assuming that they have been configured correctly.

Once the imports have completed, a folder will appear in the InfoSphere DataStage Repository named MDMIS BIL R1.0. This folder will contain all BIL assets and can be exported in its entirety for backup or replication purposes, such as deployments from Quality Assurance to Production.