InfoSphere MDM Collaboration Server V10.0 design strategy and implementation, Part 2: A guide to designing and implementing solutions using IBM InfoSphere MDM Collaboration Server v10.0

Referencing a sample business case

In Part 1 of this series, a sample business case scenario illustrated the best approach for designing and creating a technical specification of an application using IBM® InfoSphere® Master Data Management Collaboration Server v10.0. Part 2 examines the implementation strategy and shows step-by-step how to build a robust application using InfoSphere MDM Collaboration Server. Read this article to gain an understanding of the basic development considerations for implementing an application using MDM Collaboration Server.

Share:

Supal Chowdhury (supal.chowdhury@in.ibm.com), IBM Accredited IT Architect, IBM

Photo of author Supal ChowdhurySupal is IBM Advisory IT architect in application discipline and has more than 13 years of relevant experiences in the field of Information Technology with a sound understanding and knowledge of software development process. His focused technical domains are Portal, CDM and CDI, EI. He has designed, developed, tested and advised client on various elements of a technical solution required by the client. He has applied specialized knowledge to analyze, design, construct, test and implement solutions which addressed complex business or technical requirements.



Divakar Maddikera (divakar.maddikera@in.ibm.com), IT Specialist, IBM

Author photo of Divakar MaddikeraDivakar is a IT specialist with more than six years of well-honed experience in designing, developing, and programming in IBM MDM (Master Data Management - Customer and Product) technologies. He brings with him software development experience in the Retail Industry, and sound knowledge in FMCG (Fast Moving Consumer Goods) business.



24 April 2013

Introduction

This two-part series explains how IBM InfoSphere Master Data Management Collaboration Server (MDM CS) helps technical practitioners to build a product information management solution using a sample business scenario, introduced in Part 1. Part 1 introduced the product and laid out the scope, the system requirements, and a sample business case scenario, followed by technical design specification using best practices.

Part 2 includes the following:

  • Establishing a common application development methodology in MDM CS, following industry standards
  • Application programming techniques in MDM CS
  • Step-by-step development procedures for a design specification of a sample business scenario that was described in Part 1
  • Troubleshooting strategies to use during application development with MDM CS

Development strategy and best practices

This section provides guidance to developers in establishing a common methodology and common tactics for writing scripts (Java API, MDM Collaboration Server script from extension point) for your MDM CS implementation.  

J2EE knowledge is required, since MDM Collaboration Server is based on standard J2EE technology. Refer to the Sun Java or IBM Java standard to develop applications in this environment.

Setting up the development environment

First you need to set up the integrated development environment (IDE). Either Eclipse or IBM Rational Software Architect (IBM RSA) is required to install the script editor and to support the following programming interfaces:

  • MDM Collaboration Server scripting
  • Java API

You can obtain the Eclipse IDE from the Eclipse web site. Install Eclipse using the Eclipse documentation.

You can obtain an evaluation copy of IBM RSA through the developerWorks web site. Install IBM RSA using the RSA installation procedure.

Once you have the IDE installed, download and install the Script Workbench for InfoSphere MDM Collaboration Server code into your IDE system as described in the Information Center.

Development naming conventions

It is important to follow development guidelines when you are working with the components of InfoSphere MDM Collaboration Server to build a high quality, seamless solution for product management.

Table 1 provides guidelines for naming conventions that you need to follow:

Table 1. Collaboration Server object naming convention.
PIM object Naming style guideline Example Remarks
CatalogMust end with the word "Catalog" "FictXYZ Product Catalog"
SpecificationMust end with the word "Spec""FictXYZ Product Ctg Spec"
"FictXYZ Product Hier Spec"
HierarchyMust end with the word "Hierarchy""FictXYZ Product Hierarchy"
Access Control GroupShould contain basic words that are self-explanatory, ending with suffix "_ACG".  All words should be  interconnected with "_" (underscore). Base_Catalog_ACGThis example indicates "Access Control Group for base catalog"
RoleShould be self-explanatory and end with the word "Role". VP Approval RoleThis example indicates that VP Approval role has the privileges to approve or reject any introduction and enrichment of the product item.
Lookup tableEnds with word "Lookup"Item Nutrition LookupContains the key value pair of Food properties
WorkflowEnds with "WF"New FictXYZ Product WF
Collaboration AreaEnds with "Collab Area"Product Enrichment Collab Area
Attribute Collection Ends with suffix "AC" Product Enrich AC
ReportEnds with "Rep""Create FictXYZ Users Rep"This example indicates that this report is run to create users for FictXYZ organization.
ImportEnds with word "Import"FictXYZ Product Hierarchy Import
ExportEnds with word "Export"FictXYZ Product Ctg Export
Attribute Must contain only alphabetic characters. Cannot have the following characters: []{}:/"#@<>,*|’
The name of the object should not be included in the attribute names contained within that object. (That is, the attribute name should not contain the spec name)
Item Code
Staging AreaPrefixed with "Sa_""Sa_Abc_Pim" Indicates the staging table in MDM Collaboration Server where the data comes from
Mapping Ends with with "Mapping_<mapping kind>"
There are 5 kinds of mapping:
1) Catalog Spec to Destination Spec (CS-DS)
2) Catalog to Catalog (C-C)
3) Catalog to Destination Spec (C-DS)
4) File Spec to Catalog (FS-C)
5) File Spec to Catalog Spec (FS-CS)
"Print File Output Mapping_C-DS" Indicates that this mapping is created to map from FictXYZ_Product Catalog to Print output destination spec.

Best approaches to scripting in MDM Collaboration Server

Note:

The scripting standards given here are not rules of thumb. Rather an attempt is being made here to provide a baseline for the normal conventions to assist developers working on MDM CS-based projects. You don't have to strictly adhere to these guidelines, but may choose to use other available guidelines.

Scripting guidelines provide the following benefits:

  • They ensure consistency in how scripts are written.
  • They provide a consistent visual presentation of all scripts.
  • They simplify scripts maintenance by reducing dependence on the original author.
  • They make it easier to transition the script maintenance process to new team members.

Follow these guidelines for the best approaches to scripting for MDM CS:

Script naming convention
Use this basic structure:
  • COMPONENT_TYPE: The component type should represent the component type which is executing the script, for example, import, export, report, and so on.
  • CONTAINER_TYPE: If applicable, the container type should be used if the script is executing code for a container type in MDM for PIM. For example, CTG, CTR, LKP, WF, and so on.
  • FUNCTIONAL_DESCRIPTION: The description should not be too long but should have some relevant meaning to what the purpose of the script is. You should separate each word using a period, making it relevant to the naming of the extension points class which is being invoked by the script.
Stript sctions
All scripts should contain the following sections:
  • A descriptive header
  • Private functions (those you don't expect to be called from the outside)
  • Public functions (for these, you'd probably want to initialize the "class" using private functions)
  • Contextual comments throughout the body of the script
Stript sctions
A script header should contain:
  • The script title
  • The script author
  • The author's company
  • Creation date
  • A short description of what the script does
  • A list of assumptions, inputs and outputs
  • A description of how the script should be used, including a sample
  • An audit trail of modifications to the scripts
Script file extensions
Use .wpcs and .wsp.  Alternatively, until WPCW has code reformatting, use .java and .jsp so that you can open in the Java perspective occasionally to reformat the code.
Conditional statements
For testing conditions MDM Collaboration Server provides the following constant values for comparison:

true, false, null

checkString()
You can never use it enough. Remember that when concatenating strings, (null+validString) is always null.
null checks
For non-string objects always perform null checks.
catchError
Use this around blocks of code where you need specific error handling.

Never use a catchError without a matching throwError. (You can leave the MDM Collaboration Server database in an inconsistent state — the database transactions prior to the catchError are rolled back, and any database transactions after the catchError succeed)

Secure Invoker Pages
Always use "Secure Trigger" script type. Use secure_invoker.jsp as the target to post to.
Invoker script
Always access the variables from the request object.
"ASP/JSP Like" scripts
Always end an "ASP/JSP like" script with <% %> tags, even if it is empty.
Arrays and hashmaps
The script console may seem misleading. There is no difference between the two objects.
Hashmaps
You can access elements of a hashmap using the "." notation, in addition to the "[…]" notation. You can NOT initialize Hashmaps in the declaration using the var foo = [ [… , …], … ] notation anymore in MDM CS.
Avoid hard-coding
The scripts should not contain any values hard-coded that may require editing if the script is moved from one environment to another, for example, IP address, queue manager, and email address. Instead use a shared lookup table for such values.
Constants
Any constants hard-coded in to scripts should be moved to a common library script, or into a lookup table.
Script logging
Use log4j for logging and debugging.

The following functions are available:

  • Logger getLogger(String) and Logger::loggerDebug, Logger::loggerInfo, Logger::loggerWarn, Logger::loggerError, Logger::loggerFatal
  • Define routes in $TOP/etc/default/log.xml to route the logging and debugging messages to various log files. (Note that all user log statements are prefixed with "com.ibm.ccd.wpc_user_scripting".
beginPerf() and endPerf()
Use beginPerf() and endPerf() to monitor a block of code to see if it is optimized or not. And also to optimize the code for performance.
Avoid re-instantiations
Instantiating large objects like catalogs or hierarchies are heavy on the system. If you are re-instantiating the same object multiple times within a script, review your script.
Avoid multiple saves
If you are setting several values to an object, perform a single save after you have updated all the values.
Create libraries
If you see any piece of code that may be reused, go ahead and place it in a shared library (trigger script). Include signatures of all the functions as part of the header comments. This saves users for the library from scrolling through the entire code to find a required function.
Use case for libraries
All the Trigger Scripts that are set as libraries should include a simple usage for each library function.
XML tag attributes
Use the @ notation to get at an attribute of an XML tag.
Always validate XMLs
If a DTD is available, consider validating the XML against it before starting parsing.

Note: validateXML() returns a string value "Success" upon validating an XML. However, if the XML document was an empty file, the method still does return "Success". Therefore, use this method in conjunction with Doc::getDocLength().

Doc. getDocLength()
Notice that by default the method returns the size round down to the nearest KB value. Therefore, any file smaller than 1KB will return 0 (zero) value. It is probably always better to call the method with option to return value in Bytes.
Tail MDM Collaboration Server application logs
If you have system level access during development, it is a good idea to open a window that tails the relevant logs, for example scheduler logs for imports/exports; appsvr logs for trigger scripts, xandbox scripts, and so on.
useTransaction
If you're doing a huge commit on a big import or report, then you should break it into smaller commits by using useTransaction.
forEachHmElement()
Use forEachHmElement() instead of for(;;) when looping through a hash map.
getCategoryCache
Use this function to get all categories.
sendEmail
Avoid using sendEmail() to debug the code.
Variable names
Variable names should start with the following lowercase letter(s):

s              -->            String

i               -->            Integer

d              -->            Date

b              -->            Boolean

hm          -->            HashMap

l              -->            List (a.k.a. vector/array, for example an integer indexed hashmap)

itm          -->            Item

cat           -->            Category

ctr           -->            Category Tree

ctg           -->            Catalog

spc          -->            Spec

is             -->            Item Set

cs            -->            Category Set

o              -->            Object (generic object, if applicable)

en           -->            Entry Node (also known as) Item Node

doc         -->            Document Store (IDoc) Object

rdr          -->            Reader Object

wtr          -->            Writer Object

lkp          -->            Lookup table

General guidelines
  • Control the changes to scripts using a configuration tool (such as CVS, Perforce, Rational, and so on).
  • Use the right character set when loading a data file.
  • Use selections for extracting selective product information.
  • Test for nulls compulsively.
  • Use checkString(), checkInt(), and so on to test all input.
  • Use MVC design paradigm when building applications with trigger scripts.
  • Check services repository for details.
  • Use EntryNode::throwValidationError() in pre/post processing scripts to mark errors on individual attributes.
  • Use script libraries in pre/post processing scripts (or the validation framework) to do validations.
  • See the MDM Collaboration Server Services Repository.

Best approaches for Java API scripting with MDM Collaboration Server

Requirements

Java APIs can be invoked from any of the following areas of MDM CS:

  • Product hosted web services, directly using the Java APIs
  • Product hosted web services, using reflection support in scripting API
  • Custom-hosted web services, directly hosted on an application server such as WebSphere Application Server
  • Scripting API using reflection support
  • Custom tools, using reflection support of scripting API
  • Extension points

MDM CS should be running when you execute the Java API code. Ensure that any Java API code that you develop is:

  • Re-entrant
  • Thread safe

Best practices

Following are some best practices to use wherever you need to use the Java API for MDM Collaboration Server implementations:

  • Create and reuse the Java object class whenever possible.
  • Watch specific product behaviors.

    It is likely that Java API will inherit the same behaviour as the product since the Java API is a wrapper on the core product functionality.

  • Use JUnit to test the Java API code.
  • Use the Java naming standard (as stated earlier).
  • Catch and handle exceptions properly.
  • Use logging.
  • Configure custom log files for every component. Context helps to create log files while programming using log4j API supported by MDM.
  • Optionally create log files in the doc store for imports or integrations, as shown in Listing 1. (The doc store should be cleaned up periodically.)
    Listing 1. Creating log files in the doc store
    StringBuffer logStatements
    = new StringBuffer();
    String docPath ="/imports/sam_import_0444444.txt";
    DocstoreManager docMngr =
    PIMContextFactory.getCurrentContext().getDocstoreManager(); Document  importLogFile
    = docMngr.createAndPersistDocument(docPath);
    importLogFile.setContent(logStatements.toString);
  • Use the following common properties parameters which are defined in the system level properties for MDM CS:
    • DB connection parameters
    • Performance parameters (such as queue size, max space in cache, max lookup for cache, and max roles in cache

API Call restrictions

There are a few restrictions for using the Java API:

  • Do not call the Java API methods from a stand-alone Java class. Only the invocation points listed in the requirements sections are supported, for example, extensionpoints.
  • Do not use the Java API code in a multi-threaded program.
  • Do not mix scripting and Java API extension point redirection URL scripts. You can either run the script or re-direct the execution to Java API extensionpoint’s class.
  • Do not call Script API scripts from Java API code. You can access Java API from Scripts API by using a reflection mechanism. Note: Passing parameters from scripts to Java API is not supported.
  • The Java API does not support several of the wrapper operations that were supported by Script API, including currency operations, JDBC operations, and Excel operations.

API usage strategy

The following listing and figure depict an example showing how important and significant APIs are internally linked in a class diagram and how they are used.

Listing 2. Code example: Working with items
Context ctx = PIMContextFactory.getContext
("myUserName","myPassword","My Company Code")'

CatalogManager catalogManager = ctx.getCatalogManager();

Catalog ctlg = catalogManager.getCatalog("MyCatalogName");

Item itm = ctlg.getItemByPrimaryKey("101");
Figure 1. PIM API class diagram
Java class PIMContextFactory, connected to Java interface Context, connect to Java interfaces JierarchyManager and CatalogManager, connected to more Java interfaces.

Extension points

Extension points represent the area in the MDM Collaboration Server application where a script is called. (Note: This could be a logic or customization code.) The script can be a preview script, post save script, or validation rule script. There is a redirection script that is used by the Java-enabled extension point. It is normally pointing to an implementation in Java. Following are sample redirection code snippets used from UI:

Listing 3. Doc store deploy mechanism
//script_execution_mode=java_api="japi://<doc-store-path>:<fully-qualified-class-path>"
Listing 4. User Jar deployment mechanism
//script_execution_mode=java_api="japi:// <fully-qualified-class-path>"

If you want to represent a unique interface, an extension point can act on its behalf. There is a specific implementation process in order to develop an extension point:

  1. Create an implementation of the appropriate interface. For example, public  ProductProcessingImpl implements PrePostProcessingFunction.
  2. Declare implementing class in the redirection script. (You can mention this in the scriptlet area.) For example: script_execution_mode=java_api="japi://com.mycompany.mdmpim.ProductProcessingImpl".

Transactions

MDM Collaboration Server provides an excellent way to map transactions using the Java API. Here is an example showing how to use transactions to group multiple updates together:

Listing 5. Grouping updates into a single transaction
pimcontext.startTransaction(); 
try{ 
 //perform some updates 
 .... ..... .... 
 //commit
 transaction pimContext.commit() 
} 
catch(Exception e) { 
 //rollback changes incase of 
 error pimContect.rollback(); 
}

Using the sandbox

The sandbox is an area wherein you can update exiting code, running the MDM CS script to test prior being implemented in actual  MDM CS development scopes. For example, you can test import, export, workflow, and so on. The sandbox consists of:

  • Expression builder
  • Script input pane
  • Script pan sections

Significant development steps for FictXYZ smarter merchandise system implementation.

This section describes the primary development steps for our example scenario, FictXYZ Smarter Merchandise System. We'll focus ons development and running processes, based on the business requirements described in Part 1. These steps are:

  • Creating the company
  • Creating the data model
  • Creating the sample CSV files of upstream systems for import
  • Implementing custom import for CSV files
  • Developing the business workflow
  • Exporting of MDM CS data in CSV files for downstream systems

InfoSphere  MDM CS has a very rich user interface that helps you build the basic data model constructs: catalog, hierarchy, category, and product.

Creating the company

To start with your development you need to start by creating the company name using the out-of-box command line shell scripts as below:

Listing 6. Creating the company
$>$<install dir>/bin/db/create_cmp.sh –code =FictXYZ –name=FictXYZ

The command line script for deleting company is shown below:

Listing 7. Deleting the company
$>$<install
dir>/bin/db/cleanup_cmp.sh –code=FictXYZ --dropindexes=Y

Creating the data model

The first step to start your development using MDM Collaboration Server UI is to invoke the URL  http://mdmpim:7507 from any browser and provide the  default  Admin user credentials:

  • Username: Admin
  • Password: trinitron
  • Company: FictXYZ
Figure 2. PIM login page
screen cap: PIM Login Page with user name, password, and company

Note: It is assumed that the MDM Collaboration Server is fully installed, and in running state. The DNS of ‘MDM CS’ is defined with proper IP address in the hosts file.

For example:

192.168.142.128 MDM Collaboration Server

Now, based on the design specification for FictXYZ smarter merchandise system described in Part -1, you need to start implementing the data model using the following sequence of tasks:

  1. Creating specifications
  2. Creating hierarchies
  3. Creating cataogs
  4. Creating lookup tables

The following sections show how to implement these tasks.

Creating specifications

  1. Go to Data Model Manager -> Specs/Mappings -> Specs Console and click on New as shown in the figure below:
    Figure 3. Specs creation page
    Screen cap: shows SpecsMappings -> Specs Console
  2. In our scenario, FictXYZ Product Ctg Spec is the primary spec for the FictXYZ Product Catalog. When you click on the New button in above screen, you are prompted for the Spec Name. Enter FictXYZ Product Ctg Spec.
  3. After entering the Spec name, click on the Next button.
  4. In the subsequent screens, follow these steps:
    1. Click the (+) icon button to add attributes
    2. Enter the attribute name and click on the (+) icon which is parallel to attribute name.
    3. Change the attribute properties as per your requirement.
    4. If attribute is a primary key attribute then select Primary Key check box. In our scenario, Item Id would be the primary key.
  5. Once all the attributes are created, the specification appears as shown in the figure below:
    Figure 4. Product category spec
    Screen cap shows product category spec in explorer format

Creating FictXYZ product hierarchy specification

"FictXYZ Product Hier Spec" is primary hierarchy specification for the hierarchy "FictXYZ Product Hierarchy."

Similar to FictXYZ Product Ctg Spec, you can create the hierarchy specification as follows:

  1. On the Spec Tree, Spec Type will be Primary Spec.
  2. Enter Spec name as "FictXYZ Product Hier Spec," and click Next.
  3. You will be prompted to enter the attribute name, similar to creating the FictXYZ Product Ctg Spec.
  4. Once you have created all the required attributes, the spec looks as shown in the figure below:
    Figure 5. Product hierarchy spec
    Hierarchical view of the product spec.

Creating hierarchies

To create a hierarchy in MDM Collaboration Server, go to Product Manager ->Hierarchy ->New Hierarchy, as shown below:

Figure 6. New hiearchy creation
This figure shows new hierarchy creation process.

For our scenario, FictXYZ Product Hierarchy is created to categorize the FictXYZ products. This hierarchy contains 4 levels:

  • Business units: Fresh food and processed food
  • Food categories: Vegetable and non-vegetable
  • Business categories: Fruits, floral, meat, and poultry
  • Business sub-categories: Apple, banana, cut flowers, and beef

Here are the steps for creating the FictXYZ product hierarchy:

  1. Select Product Manager -> Hierarchies -> New Hierarchy. Here are the input fields required to create the new hierarchy:
    • Hierarchy Name: FictXYZ Product Hierarchy
    • Primary Specification: FictXYZ Product Hier Spec
    • Display Attribute: Code
    • Path Attribute: Code
    • Hierarchy Type: Category Hierarchy
    • Access Control Group: Default

Creating catalogs

To create a catalog in MDM Collaboration Server, go to Product Manager -> Catalogs -> New Catalog.

Enter the following values to create the FictXYZ Product Catalog:

  • Name of catalog: FictXYZ Product catalog
  • Name of Specification: FictXYZ Product Ctg Spec
  • Primary Hierarchy: FictXYZ Product Hierarchy
  • Secondary Hierarchy: NONE
  • Access Control Group: Default
  • Views: All

Creating lookup tables

Start by creating the item nutrition lookup table.

  1. First, go to Product Manager -> Lookup Tables -> Lookup Tables Console. Provide the information below:
    • Selected type: Single String Key
    • Selected Spec: Key Value Lookup Spec
    • Lookup Table Name: Item Nutrition Lookup
  2. After you have provided this information on the Create Lookup Table screen, click the Next button to create the Item Nutrition Lookup lookup table. Then click the Back button to return to the Lookup Table Console to view the lookup table. In the Lookup Table Console, the lookup table Item Nutrition Lookup appears under the Name column.
  3. Click the magnifying glass icon to the right to add all the nutrition food properties (key/value) to the lookup table. After you have added the properties, the loaded lookup table looks like this:
    Figure 7. Item nutrition lookup table
    Table includes key numbers and values, such as protein, fats, vitamins and minerals.
  4. View the data model import spec by clicking Data Model Manager -> Specs/Mapping ->Specs Console. Click the Lookup table tab at the top of the console. Click the View button to the right of Lookup Table Import Spec to view the specification as shown below:
    Figure 8. Lookup table spec view
    Shows key value lookup spec

Now, you are finished with creating the data model.

Creating the sample CSV files for importing from upstream systems

As mentioned in the FictXYZ smarter Merchandise System solution requirement (in Part 1), food product information will be imported from different FictXYZ legacy systems (known as upstream systems) using a CSV file to MDM Collaboration Server repository. In order to perform the import, you need to create the two CSV files: one for the hierarchy data load and another for the catalog data load.

For FictXYZ product hierarchy data import, CSV file will be defined as follows:

Table 2. CSV data format for hierarchy.
Column1Column2Column3Column4Column5Column6Column7Column8
1000Fresh Food1101Veg1201 Fruits1301Apples
1000 Fresh Food1101Veg1202Floral1302Banana
1000Fresh Food1101Veg1203Vegetable Salad1303Dry Fruit
1000Fresh Food1102Non-Veg1204Poultry1304Turkey
1000Fresh Food1102Non-Veg1205Meat 1305Beef
1001Processed food1103Veg1206Cheese1306Cheddar cheese
1001Processed food 1103Veg 1207Fruits 1307Apples
1001Processed food1103Veg1208Vegetables1308 Mushrooms
1001Processed food1104 Non-Veg1209 Meat1309Lamb
1001Processed food1104Non-Veg1209Poultry1310Duck
1001Processed food1104Non-Veg1210Fish1311Fish fillets

The significance of the columns is as follows:

  • Column1 data is for item code in level 1 of the hierarchy.
  • Column2 data is for item description in level 1 of the hierarchy.
  • Column3 data is for item code in level 2 of the hierarchy.
  • Column4 data is for item description in level 2 of the hierarchy.
  • Column5 data is for item code in level 3 of the hierarchy.
  • Column6 data is for item description in level 3 of the hierarchy.
  • Column7 data is for item code in level 4 of the hierarchy.
  • Column8 data is for item description in level 4 of the hierarchy.

For FicXYZ Product Catalog data import, the CSV file will be as follows:

Table 3. CSV data format for item.
Item Code Item DescItem Status Expiry Date Vendor Id Vendor Desc Department Code
1992AU Clustered apple Approved 10/10/2012 382929 ABC apples inc 21121
1992AI Clustered appleApproved10/11/2012 382929 ABC apples inc21121
1992ACClustered appleApproved10/12/2012 382929 ABC apples inc 21121
1992AD Clustered appleRework 10/13/2012382929 ABC apples inc21121
1992AESimla appleRework 10/14/2012382929 ABC apples inc21121
1992Af Dry FruitApproved 10/15/2012382911MNO Fruit inc3232
1992AGDry FruitRework11/10/2014 382929 MNO Fruit inc3232
200018Fresh ChickenApproved 11/11/2014392939 XYZ Supplier82929
332113Fish Filets Approved11/12/2014932932 LM Fish Supplier 323

…..continuing with more table colomns:

Table 4. Continuation of Table 4, with more columns
Subclass Code(Level 4 Code of hierarchy code)Net WeightWarehouse CodeNutritionIdCreated OnLast Modified OnLast Modified UserLast Modified User Name
130122319/1/20121/26/2012Mac AdMac
1301423 39/2/2012 1/27/2012SdasBuySDas
130152329/3/20121/28/2012VPCVPChouhan
1301 12339/4/20121/29/2012VPCVPChouhan
130122329/5/20121/30/2012MacAdMac
130383059/6/20121/31/2012SdasBuySDas
1303 23029/7/20122/1/2012VPCSiva
1304 1001129/8/20122/2/2012 VPCVPChouhan
1311502189/9/2012 2/3/2012PsamProdSam

The two tables above constitute 15 columns, actually one table. Column1 data is the Item Code, the primary key of the item.

So, this completes the sample text CSV file.

Implementing custom import for CSV files

Next, you need to implement first the hierarchy data and then the catalog data from the two CSV files.

Follow these steps to create and run the hierarchy import:

  1. Go to Collaboration Manager -> Imports -> Imports Console>>New Import.
  2. On the New Import screen, provide these input values:
    • Import Name: FictXYZ Product Hierarchy Import
    • Access Control Group: Default
    • Data import type: Hierarchy feed
    • Import semantics: Update
    • Data Source: Upload Via Browser (Upload via web browser)
    • Charset: Cp1252:Windows Latin-1
    • Hierarchy: FictXYZ Product Hierarchy
    • Hierarchy Import Script: FictXYZ Product Hierarchy Import script
    • Authorized User: None Required
  3. Next, you need to run the import. (Note: detailed steps can be found in the InfoSphere MDM CS Information Center. The link is in the Resources section.) On the Import Console, clck the right arrow, shown in the figure below:
    Figure 9. Run import
    Screen cap shows green right arrow in the Retrieve column of the Import Console
  4. On the following screen, click the Browse button to load the CSV file you are importing.
    Figure 10. Browse to upload file
    Screen cap shows input field for file
  5. After file is loaded, it appears as shown below in the import console. Click on the right arrow.
    Figure 11. Run import
    Screen cap: click on right arrow in Load column

Follow these steps to create the product items import:

  1. Go to Collaboration Manager -> Imports -> Imports Console and enter the following values:
    • Import Name: FictXYZ Product Items Import
    • Access Control Group: Default
    • Data import type: Item feed
    • Import semantics: Update
    • Data Source: Upload Via Browser (Upload via web browser)
    • Charset:Cp1252:Windows Latin-1
    • File Specification: Dummy
    • Catalog: FictXYZ Product catalog
    • File to Catalog Mapping: Sample
    • Catalog Import Script: FictXYZ Product Item import script
  2. Once above two imports are created successfully, you can see the imports on the Import Console screen with these headers:
    • Name: FictXYZ Product Item Import
    • Retrieved Files: Data content files that have been retrieved using the Retrieve button
    • Retrieve: This option lets you retrieve the file.
    • Semantics: Update or delete
    • Source: Method for uploading files
    • File Spec/Import Script: Script containing the logic for implementation
    • Map: Maps the file from source to destination
    • Ctlg/Cat Tree: Name of catalog or hierarchy associated with the import
    • ACG: Access control, giving permissions to the catalog and hierarchy
  3. Now you can run the import job from the import console by clicking on the second arrow mark (->) of each import job.
    Figure 12. Import console
    Screen cap: Import Console with columns described above
  4. Click the View button to browse the contents of the imported file. This file contains the input data which can map to catalog or hierarchy.
    Figure 13. Product data CSV view
    Screen cap shows product item import with the option to view

If you want to open the file, click the Open button. The file to be imported will open in a new window. In the window, the first row contains headings for the columns of data. Close the window when you have finished viewing the file.

After you have completed the import of product hierarchy data and the product item from CSV files into the MDM Collaboration Server repository, the left pane of the product catalog UI will appear as shown below:

Figure 14. Product hierarchy view
Hierarchical view of the FictXYZ product catalog

Congratulations! You have successfully imported the FictXYZ legacy data in to the MDM Collaboration Server repository of the FictXYZ smarter merchandise system.

Developing the business workflows

To build the workflows, first create the following entities in MDM CS:

  • Users and roles
  • Attribute collections (the product attributes)
  • Workflows containing workflow steps
  • Collaboration areas

Follow these steps to create users:

  1. Go to DataModelManager->Security->User Console as shown below:
    Figure 15. User console
    Screen cap shows Security -> User Console selected
  2. Create a user called VPChouhan, filling in these values in the user profile:
    • Last Name: Chowhan
    • First Name: VP
    • User Name: VPChowhan
    • email: VPChowhan@VPChowhan.com
  3. For current user role, click the check box for Marketing People Role.
  4. In the same way, create BuySDas user and assign the Product Specialist user role, and then create the AdMac user and assign the VP Approval role.

Follow these steps to create roles:

  1. Go to Data Model Manager->Security->Role Console, as shown below:
    Figure 16. Going to the role console
    Screen cap: Selecting Security -> Role Console
  2. Create the VP Approval role by giving the appropriate access privileges.
    Figure 17. Access privilege :VP Approval Role
    Shows privileges for list, edit catalog views, view items, add items, clone items, and modify items
  3. In the same way, create the Product Specialist user role and the Marketing People role, with appropriate access privileges.

Next, you need to create attribute collections. Follow these steps to create attribute collections (the product attributes):

  1. For the attribute collection name, enter "FictXYZ Product item AC" as the name.
  2. For Show Locale Restrictions, select "FictXYZ Product CTG Spec".
    Figure 18. Attribute collection for item
    screen cap: shows attribute collection FictXYZ Product Item AC

Here are the steps to create the workflows:

  1. Go to Data Model Manager->Workflows->New Workflow as shown below:
    Figure 19. Goto new workflow
    Screen cap: shows selecting workflows -> New Workflow
  2. On the following screen, you provide information about the steps, such as name, description, and access control group.
    Figure 20. New workflow view
    Screen cap: New workflow view lets you enter information about the workflow
  3. FictXYZ Create Product workflow consists of the following steps, as shown in figure below:
    1. Initial
    2. Create item
    3. Classify item
    4. Enrich item
    5. Merge item
    6. VP approve
    7. Success, Failure, and Fixit
    Figure 21. Create product workflow step
    Screen cap: Shows the workflow steps listed above

FictXYZ Product Enrich WF consists of these steps:

  1. Initial
  2. Modify Item
  3. VP Approve
  4. Success, Failure, and Fixit
    Figure 22. Modify product workflow step
    This figure shows modify workflow step.

In order to see all the workflows in MDM CS, go to Data Model Manager>> Workflows>>Workflow Console, as shown below:

Figure 23. Workflow console
Screen cap: workflow console lists the workflows that have been defined

Defining collaboration areas

In our scenario, we create two collaboration areas: New FictXYZ Product Collab Area for new items, and Modify FictXYZ Product Collab Area for modifying existing items.

To create New FictXYZ Product Collab Area, go to the new collaboration area view and enter these values, as shown in Figure 23:

  • Name: New FictXYZ Product Collab Area
  • Description: New FictXYZ Product Collab Area
  • Workflow: FictXYZ Product Enrich WF
  • Container: FictXYZ Product Catalog (Catalog)
  • Access Control Group: Default
  • Administrators: AdMac, BuySDAS, VPChouhan, Marketing People Role, and Product Specialist Role
  • Timeout: None
Figure 24. New Collab area view
Screen cap: shows the entries listed above

Save this information to produce the collaboration areas for creating and modifying the items. See the figure below:

Figure 25. New Product Collab area: Create Item
Screen cap shows the new collab area listed

Click on the Create Item to add a new item to the collaboration area. The following screen will come up, where you can provide the required attribute information:

Figure 26. Creating a new item in the New FictXYZ Product Collab Area
Screen cap: enter attributes such as item code, item description, and item status

After entering the required attributes, go to the Options tab. Select the Done button.

The item moves to the next step. Here Classify and Enrich are parallel steps, as shown below:

Figure 27. New Product Collab Area: Classify, Enrich Item
Screen cap: shows classify item, create item, and enrich item options

Click on the Classify Item. It displays this screen, where you select the Categories tab to map the hierarchy.

Figure 28. New Product Collab Area: Category Option
Screen cap: Shows Categories tab on Collaboration Area screen

In the Categories tab, the screen will display shown as below:

Figure 29. New Product Collab Area: Category Tab
Screen cap shows new product collab area for category Tab.

Select the Hierarchy and enter the category ID as search string (for example, 1301) and search for the category. The category details will be fetched as shown below. Then select and click on Apply (which is highlighted in screen), go to Options, and select DONE. Then item moves partially to merge step.

Then click the Enrich Item step.

Figure 30. New Product Collab Area : Enrich Item
Shows product attribute detailos, such as item code, item desc, item status, expiry date.

Provide the required attributes details, save it and select the Options button and click on DONE. Then the item goes to the merge step. If Classify and Enrich steps are successfully completed, the item then moves to the VP Approval step.

Figure 31. New Product Collab Area View: VP Approval
VP approval is listed on the screen.

Click on the VP Approve step and a screen appears where the item can be approved, rejected, or fixed.

Figure 32. New Product Collab Area View: Approved
shows the item attributes and option to approve or reject

If you want to modify the item code, you should modify it in the VP Approve step. Once modification is completed, click on the Approved button. The VP Approve user can also reject the item. To reject the item, select the Rejected button. Once it's approved, the item is saved in the catalog.

To create the Modify FictXYZ Product Collab Area, you need to follow the same steps that you followed to create the New FictXYZ Product Collab Area.

In the following step, you will see how an item can be modified using the Modify FictXYZ Product Collab Area.

  1. On the Collaboration Area screen, search for the item. Once it is retrieved, right-click the mouse button on the item to display the Check outoption. From there, select Modify FictXYZ Product Collab Area.
  2. The item moves to the Modify Item step, as shown below:
    Figure 33. Modify Item View
    This figure shows modify item view.
  3. Click on Modify Item and then click on the Reserve option to modify the attribute values if required. Finally, click on Done as shown below:
    Figure 34. Product Category Collab Area View
    Lists the common attributes of the item
  4. Once the item has been modified, the item moves into the VP Approval step:
    Figure 35. Collab Area Console
    This figure shows approval step on the console
  5. Click on VP Approval, and it will bring up the screen below. Click on either Approved or Rejected as  required. Once it is approved, the item modification will move into the catalog.
    Figure 36. Modify Product Collab Area View
    This figure shows item attributes, with option to approve at bottom of screen.

At this point, you have completed developing the MDM CS Workflow.

Exporting MDM Collaboration Server data for downstream systems

The following steps show how to create the export which produces the CSV output used for downstream systems. An export creates output based on current product data. The output is typically used as input into another system. The data can be stored at its destination.

  1. From the Collaboration Manager, select Exports -> New Export.
    Figure 37. Go to New Export
    Shows selecting New Export
  2. On the New export screen, provide the following information:
    • Name: FictXYZ Product Ctg Export
    • Catalog Items: FictXYZ Product Catalog Entire catalog
    • Mapping: Mapping from source to destination
    • Destination Spec: Dummy Export Spec
    • File Type: FictXYZ Product Export Script
    • Hierarchy: FictXYZ Product Hierarchy
  3. You can see the export that you have created by clicking on Collaboration Manager -> Exports -> Export Console.
  4. Now, to run the job for export that you created, select the check box to the left of the job description. Then, click the green arrow icon at the top of the right pane, and you can see the status of job as shown below:
    Figure 38. Export Started View
    Screen shows list of exports that have been started
  5. Click the time clock icon at the far right of the job description line to see the export job progression.
  6. When the job is finished, the following screen will be presented. The status of a successful job completion should be Completed Running. The Return Value should indicate No warnings and no errors.
    Figure 39. Scheduler Status View
    This figure shows status and return values of the export job.

The export output file is available for viewing. To view this file, right click the catalog.out link and select Open in New Window. Or, output data can be stored in the document store.

Figure 40. Document Store View
Shows listing of export CSV files in the document store

This completes the implementation of the FictXYZ smarter merchandise system using IBM InfoSphere Master Data Management Collaboration Server.

Troubleshooting

 MDM Collaboration Server provides efficient ways to debug and troubleshoot runtime exceptions. Whenever any exception occurs, you can view the exception and follow the instructions below to debug the problem.

First, check log files generated by the MDM Collaboration Server. There are log files for certain events within each of the services for troubleshooting runtime errors. Files to start debug with are: svc.out exception.log and default.log.

The following three ways permit you to view your log files:

  • You can access them through the MDM CS user interface as follows:
    1. Select System Administrator->Log Files to open the Log Directory Listing Panel.
    2. Click each radio button of the log files that you want to view.
    3. Scroll down to the bottom of the Log Directory Listing Panel.
    4. Select either the entire log file check box to view the entire file, or specify the maximum number of the lines that you want to view in the text field.
    5. Click the Submit button to view the log file.
  • You can access them from the log file directory as follows:
    1. Navigate to the $TOP/Logs directory for the default log4j formatted log files or $TOP/logs/cbe for CBE formatted log files
    2. Open the appropriate file directory to locate the log files that you want to view.
  • Access the log files from the Log and Trace Analyzer by using the IBM Support Assistant.

Also, you can test your code using a Java standalone program to debug your Java methods. Follow these steps:

  1. Use Eclipse as an IDE.
  2. Make a local copy of $TOP from your MDM Collaboration Server environment.
  3. Add the MDM Collaboration Server JAR file from $TOP/jars as an external JAR to the class path.
  4. Make a local copy of other external JAR files: db2jcc.jar, db2jcc_license_cu.jar, and tools.jar from your MDM Collaboration Server environment. Add them as external jars to the class path.
  5. Verify db_url with IP address and db port in db.xml  or common.properties (for version 9 and earlier) as shown in the listing below:
Listing 8. db.xml:
<?xml version="1.0"encoding="UTF-8"?> 
	<db_config>
	  <db_userName>[user id]</db_userName> 
	  <db_password_encrypted/><db_password_plain>[password]</db_password_plain> 
	  <db_url>jdbc:db2://[ip_address]:[port]/[MDM Collaboration Server DB]</db_url>
	  <db_class_name>com.ibm.db2.jcc.DB2Driver</db_class_name>
	  <db_type>db2</db_type> 
    </db_config>

Note: The JDBC URL can be shown by running $TOP/bin/test_db.sh. And The db port defaults to 50000 and can be verified by running the command netstat -an | grep <my_db_port> from the database server to see if this port is listening.

  1. The log should be in your local $TOP/logs directory if Java is run from main function locally.
  2. Create a native database connection Java program to verify your local environment (only db driver and db URL without involving MDM CS) if the MDM CS Java program still does not work.
  3. Test your Java code start as shown below:
Listing 9. Testing your Java code
public static void testConnectivity() 
{ 
    userName="Admin";
	password="trinitron";//Default password for Admin
	company="FictXYZ";  // The name of the company 
	context = null; 
	try {
	  logger.logInfo("getcontext..."); 
	  Context context =
	   PIMContextFactory.getContext(userName, password, company);
	   logger.logInfo("Successful"); 
	   SpecManager manager =
	   context.getSpecManager(); 
	   Collection<Spec> collection = manager.getSpecs();
	   Iterator iterator = collection.iterator(); 
	   while (iterator.hasNext()) { 
	      Spec spec =
	      (Spec)iterator.next(); 
		  logger.logInfo("SpecSamples:getListOfSpecs(): " +
	       spec.getName()); 
	   } 
	} catch(Throwable e) {
	  e.printStackTrace(); 
	} finally{ 
	   if(context != null) 
	       context.cleanUp(); 
	}
}//testConnectivity

So, this way you can start designing a smart merchandise IT system (described in Part 1) and implementing with your existing system using IBM MDM CS.


Conclusion

Part 2 of this series explained how to start implementing a smarter merchandise system for a sample business case using IBM InfoSphere Master Data Management Collaboration Server v10.0, whenever there is design specification ready as explained in the sample business case scenario.

After reading this series, you should be familiar with how to get started designing and implementing your own solution using InfoSphere MDM Collaboration Server.


Acknowledgement

The authors would like to thank the following people for their contributions, reviews and suggestions to this article:

  • Jeffery Alspaugh, A Senior IT specialist in IBM InfoSphere MDM Server for PIM
  • Sachin Prasad, IBM InfoSphere MDM  Collaboration Server product specialist
  • Radha M Dey, senior IT specialist in J2EE application build
  • Svayambhu Chaudhuru, BAO - Enterprise Information Management, service area leader

Resources

Learn

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Information management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Information Management
ArticleID=870935
ArticleTitle=InfoSphere MDM Collaboration Server V10.0 design strategy and implementation, Part 2: A guide to designing and implementing solutions using IBM InfoSphere MDM Collaboration Server v10.0
publish-date=04242013