Dany Drouin 270004VXKT Visits (1801)
MDM AE pMDM with RESTful web services
Previously, interactions with MDM Operational server were possible with EJB/RMI, JMS, JAX-WS and JAX-
Possible payloads that are accepted are application/xml and application/json. JSON support was added in v11.4 FP1.
It is important to note that all REST interaction are using one RESTful service “MDMWSRESTful”, PUT method type only and accessed via URI http
The same xml request/response payload used for EJB/RMI is used for REST interactions.
For the full list of capabilities and supported request headers consult the following documentation link:
Interacting with MDMRESTful service
Here’s a sample client leveraging Apache Wink demonstrating an MDM RESTful call:
The above code will submitting an MDM xml payload and expecting back an xml response.
This is determined by the ‘Content-type’ and ‘Accept’ http header properties.
Here’s a look at a getParty xml payload and response:
The same request/response as JSON, using application/json, as both content-type and accept:
The default MDM JSON model is actually based on the core XML schema model (MDMCommon.xsd and MDMDomains.xsd). Internally, MDM will validate the JSON using these schemas.
We use a “Mapped notation” api to build the JSON. A couple things to note about this implementation:
Don’t want to write any code to test your MDM services?
Choose “PUT” as the HTTP method
curl --user "mdmadmin:mdmadmin" -X PUT
If you've used previous versions of the workbench, one of the first changes you'll hit is that you no longer need to run the developer environment setup tool when you create a new workspace. In version 11, no projects need to be imported into the workspace, and you use the same installer to setup a local test server on your development machine as you would to install a production system.
Full development environment install
If you have a completely clean machine, the simplest way to get started is to use the workbench typical install. This will install DB2, Rational Application Developer, and WebSphere Application Server, along with MDM Server and the workbench, i.e. everything you need for a full development and test environment in one go. Here's how to get everything ready to run a typical install...
Firstly, you'll need to download all the typical install images. The following part numbers are required for a full MDM Workbench v11 typical install:
CIM6NEN, CIM6PEN, CIR9NML, CIR9PML, CIR9QML, CIR9RML, CIR9SML, CIR9TML, CIR9UML, CIR9VML, CIE5FML, CIE5GML, CIE5HML, CIE5IML, CI6XNML, CI6XPML, CI6XQML
Important: If you are about to install MDM but downloaded the install images before 17th October 2013, you must download the product refresh first.
Once you have all the install images downloaded, the contents must be extracted into a specific dire
After extracting all the install images, open the install launchpad, which you can find in the MDM\disk1 directory (there are 32 and 64 bit versions). The typical workbench install link is right at the bottom of the launchpad:
When the install starts, you should be able to click through all the panels without changing anything:
Make sure you confirm that the IVT tests pass at the end of the install and, if they did, you're ready to start developing for MDM v11!
Note: you should change the defa
Workbench only install
If you don't want a local server to test changes on, installing the workbench is much quicker, since DB2, WebSphere Application Server and the server install are not required. In this case, you'll only need the following part numbers:
CIM7CML, CIR9TML, CIR9UML, CIR9VML, CIE5FML, CIE5GML, CIE5HML, CIE5IML
The launchpad doesn't support this scenario, so you have to install Installation Manager manually, add
Alternatively, you can use the Installation Manager command line to install Rational Application Developer and the workbench in one step. For example, assuming you extract the install images in the same structure as for a typical install:
imcl install com.
A typical install is ideal for demos or evaluating MDM but to set up developer environments I would recommend installing manually. You'll also need to do this if the typical install does not support your environment. The following blog post describes the manual install process:
There is also a wiki page with an up-to-date* list of install related information.
(* Please update it if it's not up-to-date!)
Doug Cowie 270005CYF0 Visits (1043)
From version 11.4 FixPack 3 the MDM Application Toolkit has a new hierarchy widget, which replaces the now deprecated MDM Tree coach view.
This new widget, the MDM Hierarchy coach view uses the latest in web visualisation technology to render hierarchies in BPM coaches. As well as using this new technology the MDM Hierarchy coach view also has a new method of interacting with the MDM operational server.
To highlight some of the new features, this post presents a step-by-step guide of how to get up and running with the new MDM Hierarchy coach view. I will assume a degree of familiarity with IBM BPM, in particular Process Designer.
Step 1: Drag and drop the MDM Hierarchy coach view from the palette onto the canvas, it is listed under the MDMAT grouping.
Switch to the configuration tab. You will notice that most of the fields have default values. In the rootNodeId field enter the values for the hierarchy and a node in the hierarchy in the format <hie
Step 2: Press the “Run” button in BPM. This will launch a browser, showing the coach you have just created. The hierarchy will be visible, and should render data if it has been set up correctly.
That is all that is required to get the MDM Hierarchy coach view up and running.
The coach view has a set of other configuration options; please see the documentation for more details on the configuration options.
The MDM Hierarchy coach view can be augmented by connecting it to a set of other coach views, which provide pop-up dialogs with additional behaviour that complements the hierarchy. These are the MDM Hierarchy Dialog Add, the MDM Hierarchy Dialog Details, the MDM Hierarchy Dialog Error and MDM Hierarchy Dialog MultiParent.
While each of these coach views can be added independently, the instructions below will guide you through adding them all.
Step 1: Adding the other coach views.
Drag and drop the MDM Hierarchy Dialog Add, the MDM Hierarchy Dialog Details, the MDM Hierarchy Dialog Error and MDM Hierarchy Dialog MultiParent on to the canvas that contains the MDM Hierarchy coach view.
Step 2: Create a new MDM_
Switch to the variables tab. Create a new Private variable, call it “events”. Change the Variable Type to MDM_
Step 3: Configure all of the widgets to use the same, shared event framework. Switch back to the Coaches view. For each coach, select it on the canvas, then select the Configuration tab at the bottom.
Locate the EventFramework configuration option; click the purple button next to the label. Then click the Select button to the right hand side. Find the variable you created in Step 2 (events) and select it. Do this for each of the widgets.
Step 4: Configure the visibility for each of the dialog coach views; this step should not be performed on the MDM Hierarchy coach view.
Select the coach view, then click the Visibility tab at the bottom. Leave source as “Value” then press the purple button next to the Visibility label. Press the Select button, then expand the events variable, expand the appropriate event, then select the visibility entry. Each of the different dialogs should be configured against its specific event. The MDM Hierarchy Dialog Add should be configured to use the addNode event; the MDM Hierarchy Dialog Details should be configured to use the nodeDetails event; the MDM Hierarchy Dialog MuliParent should be configured to use the multiParent event; the MDM Hierarchy Error Details should be configured to use the error event.
Step 5: Click the “Run” button in BPM.
The tree now has additional behaviour, if you right click on a node a pop-up dialog should now appear that will display additional data about the node. The add button on this dialog will launch the add dialog that can be used to add nodes into the hierarchy. If a node in the hierarchy has multiple parents in the hierarchy an icon indicating this is displayed to the right of the node, the MultiParent dialog will be launched if that icon is clicked and allows users to re-focus the hierarchy on the different parent nodes.
This brief post has demonstrated how to use the new MDM Hierarchy and associated coach views. In future posts more advanced topics, such as replacing the ajax service which supply the data to the hierarchy, and how to create custom widgets that use the event framework will be explored.
This blog post details how to use the template models provided in MDM Workbench to create and deploy a new Virtual Configuration Project to an Operational Server, and then deploy the sample data supplied in the template model.
Creating a new Virtual Configuration Project
Creating a connection to the Operational Server
Deploy the new Configuration Project:
Processing and loading the sample data:
Mike Cobbett 11000061J8 Visits (3806)
MDM Workbench v11 is here !
With huge pride, we have just shipped the version eleven of MDM, including the new unified MDM Workbench v11. It's been nearly 2 years in the making, and represents the biggest change to the MDM tooling in recent years. In this article we outline these changes and give the reader familiar with previous MDM tools a gentle introduction to what they can expect when they get their hands on the new tools.
The main changes made for v11 workbench can broadly be categorized under the titles unification, simplification and integration.
Unification is a drive to combine the tools from the v10.1 standard edition (formerly Initiate tools) and the tools from the v10.1 advanced edition into a single set of tools which run in the same Rational Application Developer (RAD) environment. Where the tools were inconsistent, or overlap existed we adopted common approach to make sure both sets of tools work together in a unified tooling environment.
An example of unification: Consistent use of the perspectives, showing the new MDM development perspective.
We want all the tools to be simpler. We aim to cut the time it takes to get value out of the MDM platform; automating where possible to relieve solution developers of repetitive tasks and reducing the amount of knowledge needs to get something working.
Toward this goal the workbench has made these changes :
An example of the way version is simpler can be seen by comparing a version 10.1 workspace against a version 11 workspace:
"No man is an island" as the saying goes, and the same is true for products. MDM tools now play a wider role in enabling the ingestion and distribution of information in an MDM solution.
Enhancements in this area include:
For example: Our list of export wizards has been expanded to help push MDM metadata to more remote systems.
In summary, we hope you like the changes we've made to the tools, and hope you find that creating, configuring and developing an MDM solution is now quicker and easier than ever before.
For more detailed information, and a complete treatment of the MDM version 11 release, please refer to the info
Failed to connect to the JMX port on server
When you first connect from MDM Workbench to WebSphere Application Server (AppServer) where MDM Server is installed, for example to deploy a configuration project or to run a virtual job, you might see this error:
Job Manager Error - Failed to connect to the JMX port on server
There can be several reasons why the connection might fail, for background, here is the stack you are relying on when you connect to the JMX port.
In order for the JMX port connection to be successful, you need every component in this diagram to be in a fully functioning healthy state. And yes, that means there are a lot of places you can check! As a result, it's not practical here to explain every possible area to review, but this should give you some idea of where to start investigating.
To begin, cut the problem in half: there is a message associated with blueprint virtual bridge. Look for this, and it will help you decide whether the problem is more likely to be a runtime issue (below and to the right of the blueprint virtual bridge component) or a configuration issue
1. Look for virtual bridge messages
On the Application Server where MDM is hosted, open SystemOut.log or HPEL logs: if possible restart the AppServer first to make sure you have startup messages:
When the MBean starts successfully, you will see messages like these:
Note that these messages will only appear on startup, so they may not be visible if the logs have wrapped
If you have these success messages the Blueprint virtual bridge is available for JMX requests, and everything to the right of the diagram (MPIJNI, JMS, databases, filesystems) is healthy.
In this case the likely cause of the problems is to the left of the diagram, and probably relates to a configuration issue. More information is available in section 3. When the virtual bridge has started successfully
When the MBean has not started, you see messages like this:
If you have these failure messages the Blueprint virtual bridge is not available. More information is available in the next section, 2. When the virtual bridge has not started
No messages found
If you don't find any messages relating to com.
2. When the virtual bridge has not started
When the blueprint virtual bridge has not started, the next step is to investigate potential runtime issues in one or more of the components on the right side of the diagram.
Note that you can choose whether you use a datastore or filestore for the messaging engine data store: the default is datastore (database).
There may be file system errors, these will usually be reported by the component that depends on the file system, for example the database or the JMS filestore.
In many cases you will be able to find technotes or other links on the internet with information about how to resolve the errors, or if not, contact IBM support and provide the logs that show the errors.
These related links have information about resolving blueprint errors:
3. When the virtual bridge has started successfully
Once you have found the success message, the next step is to investigate the configuration in both WebSphere Application Server and MDM Workbench.
Review the server logs for authorization errors
On the Application Server where MDM is hosted, open SystemOut.log or HPEL logs. Look for errors that reference one or more of:
Errors with any of these codes suggest that you need to re-visit the security configuration in the WebSphere Application Server administrative console, and check userid and password settings in the workbench client. Review the error messages, in many cases you will be able to find technotes or other links on the internet with information about how to resolve them, or if not, contact IBM support and provide the logs that show them.
Review the firewall settings
Verify that you can ping from the Workbench machine to the machine that hosts WebSphere Application Server and MDM Server, using your preferred ping tool.
Optionally you can use "Test Connection" from MDM Workbench, although note that in an ND configuration this tool only checks the dmgr, so it may not be the correct status for the actual server where MDM is hosted.
If you can not connect to the target MDM server, the JMX connection will not work and you need to contact your networking support team to make sure the network is available and if necessary, that appropriate firewall ports are opened.
Review the port and host configuration
bakleks 270007PVJ3 Visits (2787)
This document outlines how Physical MDM customisations can be built from source artefacts in an automated build and test system. This document does not aim to be a complete guide on this topic, but rather to point the way to how some detailed steps can be implemented using examples.
The MDM Advanced or Standard editions both include the MDM Workbench. In version 11.0 and beyond the MDM Workbench is used by solution developers to create artefacts which customise the MDM solution for the physical, virtual and hybrid implementation styles. These source code artefacts are typically built into a Composite Bundle Archive (CBA) and deployed to WebSphere where they augment the functionality already available in the MDM Server Enterprise Business Application (EBA).
A good practice amongst MDM solution developers is to create an automated build process such that customisation source code is checked-in to a code control repository, and an automated build process takes those source files and builds the CBA ready for deploying onto post-build test systems, placing built artefacts into a second repository or shared file system.
Some automated systems take this “build” concept further, by automating the deployment of such built artefacts to test systems, which in turn report back on the “health” of the build, how many tests passed and failed, and generally quickly provide valuable feedback to developers whether recent changes broke the solution or not. Project managers overseeing such projects are able to reduce project risk by adopting this continuous delivery processes, and changes to MDM solutions become more reliable and safer as a result.
To add MDM solutions to such a continuous build environment it is necessary to:
This article is mostly concerned with step #7 – building source artefacts.
2. Materials and prerequisites
This article is accompanied by a collection of example scripts. We do not intend that these are used directly, but as an example of how you may wish to implement your own automated build process.
The current solution consists of four main files:
In order for the scripts to work, the machine running the scripts needs to have the following products installed:
To run the Ant scripts the user needs to run mdm_wb_build.xml as a build file.
The script contains only the “runBuild” target.
The target checks that necessary properties, such as Eclipse Home, date and time stamps and output folder prefix are set. Provided these properties do exist, it creates a folder based on OutputFolderPrefix and date and time, within which “logs”, “CBAExport” and “workspace” folders are created.
The logs folder contains “MDM
generateDevProject: BUILD SUCCESSFUL
workspaceBuild: BUILD SUCCESSFUL
exportCBA: BUILD SUCCESSFUL
End of report.
The CBAExport folder contains all of the exported CBAs.
The workspace folder contains a local copy of build artefacts.
After the directories have been created, the script checks which operating system it is running on and sets the isLinux or isWindows property to “true” as appropriate and calls either runAnt.sh or runAnt.bat to run a headless Eclipse process. The relevant file (either the batch or the shell script) should be available by default in the bin directory in the Eclipse installation directory.
The runAnt script then sets up the log files, environment variables and runs a second script “mdm
3. Step breakdown of the automated build and test system
Given that automated building and testing of MDM solutions is a worthwhile goal, the following sections provide some guidance in some of these areas where actions specific to the MDM tools and development/build environment are necessary, and some points of discussion are presented where choices exist.
3.1 Identify the pieces of the solution which represent the “source code” for the solution.
The source code for an MDM solution will be made up of a collection of Eclipse projects and their contents. MDM development, MDM configuration, MDM hybrid mapping, MDM service tailoring, MDM custom interface, MDM metadata and other MDM-specific projects types. CBA projects will add to the list.
MDM Development projects contain a “module.mdmxmi” file, which contains a model of the customizations which the project aims to create. This file should always be considered to be source code.
At some point the mdmxmi file will be used to generate Java, XML, SQL and other file artefacts, and there are a few different approaches you can take for these files:
The current solution is to only consider files which have been manually changed as “source code”, and “generate artefacts” from the mdmxmi model as part of the automated build process itself. This approach demands that the MDM workbench tools are installed as part of the build environment, because the “generate artefacts” process that turns .mdmxmi files into other artefacts will be a necessary part of the build process.
A project “MDM
3.2 Create a source code repository.
There are many choices regarding which product to use as a source code repository and covering them is not the aim of this document.
3.3 Recognize when a consistent set of code has been checked-in, at which point a “build” is started.
This event may be triggered manually, automated overnight, or whenever a change-set is delivered to the code stream. The capturing of this event is often specific to the code control system being used, though some solution teams augment this by adding a web page that enables build requests to be manually requested.
3.4 Create a build environment.
A build environment should include RAD (or RSA) which can be called in a “headless” manner such that functionality within RAD can be used without a user-interface being present.
MDM Workbench will be required in addition to RAD to perform a complete build of “module.mdmxmi” files.
For the list of platforms that MDM Workbench v11.0 and onwards support – refer to the product release documentation.
3.5. The build environment “boot-straps” itself.
A small script is responsible for “boot-strapping” the process by it checking-out the other build scripts which in turn build the artefacts from solution developers.
3.6 The build scripts check out the artefacts from code control to the local file system.
These actions are specific to the code control system so will not be discussed further here.
3.7. The source artefacts are processed, transforming them into built artefacts.
This phase of the automated system typically consists of a hierarchy of Ant files which decomposes the overall build process into many smaller steps and “Ant targets”. The Maven framework is a common choice of technology to oversee this phase.
These Ant files can be categorized into two types:
For a detailed walkthrough of specific implementations of the build process refer to the Ant scripts provided with this blog entry.
3.8. The build process often executes “unit tests” to further validate that the solution artefacts are healthy and do what they are expected to do.
The tools and approaches used to execute unit tests vary widely dependent on technology choice. Simple Java JUnit tests offer one simple solution, which can be invoked with scripts once the tests and tested code are built.
3.9 Built artefacts are published to a repository.
Every build against which build metadata can be gathered and reported is versioned by the publish process. Build logs, unit test results and results of other tests indicating the “health” of the build are gathered and published to the repository as well.
Products such as Rational Asset Manager can be used here, or for a really basic solution a simple shared folder on a network drive may suffice.
3.10 Automated deploy and test health of overall build.
If the build is considered “good” then further automation can be added to deploy the built solution to a test environment, with higher-level tests (functional and end-to-end system tests) exercising the solution further. Such tests can also report back to the build repository on the health of each build.
The automated deployment of entire systems for testing is often one of the most complex areas of the whole continuous development process. Products such as Rational Urban Code Deploy (UCD) can be used for this stage of the process, though for some environments a set of (reasonably complex) scripts might be sufficient.
For the MDM pieces, we are mostly concerned with deploying and un-deploying SQL scripts, depl
Prior to deploying extensions to the server, it is often necessary to modify the database. This is possible using the SQL scripts found in the MDMSharedResources project in the built workspace. Rollback scripts in the same location should be applied once testing is complete to reset the database back to a known good state.
For CBA deployment, Jython scripts can be used to manipulate the WebSphere server. Detailed documentation of these steps can be found in the WebSphere documentation.
Nic Townsend 2700051ED4 Visits (3898)
MDM v11.3 leverages IBM BPM technology to provide a data stewardship framework under a single UI. However, it may be desirable to modify this UI to match the look and feel of your existing solutions. While BPM Process Designer allows you to restructure page layout with the Coach designer, the best way to modify the look and feel of a Coach is with CSS.
BPM provides three ways to alter CSS out of the box from within Process Designer:
However, there are occasions where these approaches are not suitable:
The ideal way to insert CSS into a Coach would be to load the CSS as a managed file in BPM - that way you only need to edit the managed file and all Coaches that reference the CSS would use the latest version (pending updated snapshots). Unfortunately, BPM does not offer the mechanism out of the box.
Update 05/11 - If you upload a HTML file to BPM that consists of <style> tags wrapping the CSS, you can use the "Managed File" option of the Custom HTML component to load the "HTML" CSS into the Coach. However, this does not work if the HTML file is inside a .zip archive, or if the CSS needs to reference local resources.
By using this method call, you can get a relative URL to the managed file in BPM - eg
Using the above technique, I created a HTML file containing a <script> that would create a link to the URL for a CSS file in the document's <head> element. I then used the "Managed File" option on a Custom HTML component to load the script into a Coach. This meant that my CSS file was referenced inside the <head> element.
You can use this principle to load your own custom CSS files into a Coach. Custom CSS files can be used to override BPM Coach Views or Coaches, or alternatively CSS files can be used to override the MDM Coach Views supplied with MDM V11 onwards.
As an example, I present an updated solution to:
Dave_Kelsey 100000BGBR Visits (2515)
No results found is a common response to a search request, but how do you detect this in your BPM process ?
A “No Results Found” situation causes an exception to be thrown which means you can detect it using an intermediate error event, but you need to be able to handle a real errors as well as a no results found which is a special kind of exception.
Here we have a simple example of a search human service which applies equally to Physical and Virtual MDM Server.
There is an intermediate error event attached to the do search nested service that calls the MDM Search integration service. The gateway decision determines whether it is a no results found or not.
To determine how to configure the decision gateway we need to have a look at the format of the exception we get when a no results found is generated and this differs slightly between virtual and physical MDM Server.
Handling a Virtual MDM Server response
A sample of part of the xml format of the exception is shown here
Here is is the reason code we are interested in. So the decision gateway configuration looks like this
The decision logic being
extracts the reason code and checks for ENOREC.
Handling Physical MDM Response
The implementation remains the same, the test within the decision gateway needs to be altered as the location of the reason code is different to virtual. In this example the reason code that is checked is applicable to a Party or Person Search, however it is possible that the reason code will be different for different types of physical search.
A sample of part of the xml format of the exception is shown here
Here we see the reason code we are interested in is nested under the <errors> tag within an < element>.
So in this case we need the decision logic to be
From an early MDM Workbench news site, the MDM Developers community has evolved and grown to a group of over 200 members, and it would be great to take a break from the usual posts and forum discussions to find out more about some of you with a quick blog interview. Whether you're a new member or a long term contributor, please say hi and tell us a little about yourself.
Feel free to leave a comment and answer any of the following questions that resonate with you, or add your own questions instead. This is just a casual blog interview and meant to be more like a real world conversation, rather than a formal resume or biography!
For fun, and a bit of encouragement, I have a few limited edition MDM Developer community stickers to give away!
Here are a few questions to get you started:
Update: Unfortunately no one replied in time to claim the ticket that prompted this blog post. Luckily if you're one of the first to reply, you could still get one of these, much more exclusive, MDM Developers stickers!