Introduction

IBM InfoSphere Data Architect (IDA) is a comprehensive development environment for data modeling and integration. IDA enables users to discover, model, visualize, and relate diverse and distributed data assets. IDA is an integrated data management offering from IBM and is tightly integrated with the Optim®, Rational, and InfoSphere products built using Eclipse. IDA supports logical, physical, glossary, storage, domain, and integration data modeling. As more enterprise customers use IDA, there is an increasing need to extend IDA to meet their unique data modeling and integration requirements.

As mentioned before, IDA is Eclipse-based and therefore highly extensible by design. In this series, learn more about the extension points, APIs, and factories available with IDA that you can use to extend IDA. See how to extend IDA to:

• Programmatically traverse and modify IDA models (Part 1)
• Add new properties and display them in the Properties View (Part 1)
• Generate customized model reports (Part 2)
• Add model validation rules (Part 2)

This article assumes you have Eclipse plug-in development experience and the basic knowledge on Eclipse EMF and BIRT projects. The sample code provided in this article is tested on IDA 7.5.1 and 7.5.2 and on Eclipse 3.4.1 and 3.4.2.

Generate customized model reports using BIRT

Reporting is an important feature of IDA. It provides information on the whole or part of a model: that is, a list of objects and their relationships. This information can be copied, printed, and distributed as one physical document. Reports are also used to provide compliance information in many organizations. IDA provides a wide range of built-in reports or templates for your logical, physical, glossary, and mapping models. BIRT has been integrated and extended to provide more flexible reporting and customization capabilities since RDA 7003 (version 7 fixpack 3). IDA reporting uses BIRT in combination with the Open Data Access (ODA) component.

In this section, you will do the following:

1. Create a report and specify a data source
   
   
2. Create a Tables data set that is used to display tables in the report
   
   
3. Generate a sample report from existing IDA reports
   
   
4. Customize the report to add a column that reports on the masking method used for a table column

You will lean how the EMF ODA driver works by completing these steps. Then, you can customize the reports the way you want using the BIRT designer.

Step 1. Create a new report and specify a data source

Follow these steps to create a new BIRT report design with an EMF data source:

1. Open the Report Design perspective.
   
   
2. Create a new report using File > New > Report.
   
   
3. Select a parent folder and Simple Listing report template in the new report wizard, and click Finish.
   
   
4. Right-click on the Data Sources folder in the Data Explorer and select New Data Source.
   
   
5. Select EMF Data Source and type in SAMPLE Data Source in the New Data Source window, as shown in Figure 1.

6. Click Next.
   
   
7. Select Add to add the SAMPLE.dbm (created in Part 1 of this series) as the EMF data instance, as shown in Figure 2.

8. Click Finish.

The SAMPLE model, which is a physical model instance, is used in step 4 above as the data source. As mentioned, you can also define the meta-models as data source when designing a report.

Step 2: Create a Tables data set

Follow these steps to create a Tables data set that gets all the tables in a model:

1. Right-click on the Data Sets folder in the Data Explorer and select New Data Set.
   
   
2. Type in Tables Data Set as the data set name, and click Next.
   
   
3. Click Next on the Query Parameters page.
   
   
4. In the Row Mapping page, click on the drop-down arrow and select sample.dbm, as shown in Figure 3.

The database, schema, index, persistent table, and column objects contained in the SAMPLE model are then listed in the Browse area. You can expand objects in this page and get familiar with their structure.

5. Select any of the persistent tables in the Browse area, click the > buttons to add query expressions, and set the query type, as shown in Figure 4.

6. Click Next.
   
   
7. In the Column Mapping page, click on the drop-down arrow, and select SQLTables:PersistentTable, as shown in Figure 5.

The structure of PersistentTable is populated.

8. Select name:EString from the Browse area, click the > button to add it as a column query, as shown in Figure 6.

9. Click Finish.
   
   
10. Click Preview Results. You should see a list of tables, as shown in Figure 7.

11. Click OK to finish. You should see the SAMPLE Data Source and Tables Data Set created in the Data Explorer, as shown in Figure 8.

Step 3: Generate a sample report from IDA built-in reports

The built-in reports provided by IDA are categorized by the model type they report on. Open the Report Explorer view under Reporting, and you can see a complete list of IDA built-in BIRT and XSLT reports as shown in Figure 9. The reports with a file extension of .rptdesign are BIRT reports. Those with the extension of .xsl are classic XSLT reports.

For example, if you have a physical model, you can generate a report for all objects in the model or only for columns, column mapping, or table spaces using the Physical Data Model Report, Column Report, Column Mapping Report, or Table Space Report are listed under the Physical Data Model category.

You need to create a copy of a built-in report to be able to open it. For example, you can open and have a closer look at the Data Source of the Column Report by following these steps:

1. Right-click Column Report under the Physical Data Model category, and Copy and Paste it to a project.
   
   
2. Open the copy of Column Report with Report Editor in the Report Design perspective.
   
   
3. Right-click Data Source in the Data Explorer, and select Edit. You see that Ecore.ecore, db2.ecore, schema.ecore, tables.ecorem, and other SQL ecore models are listed as Ecore meta-models for the report design, as shown in Figure 10.

Follow these steps to configure and generate a column report for the SAMPLE model you created in Part 1:

1. Open the Data Perspective, if it's not already opened.
   
   
2. Click Run > Report > Report Configurations.
   
   
3. Right-click BIRT Report, and select New in the Report Configurations dialog.
   
   
4. Type in the name, set Column Report as Built-in report, add SAMPLE.dbm as Data Source, and select the output location and format, as shown in Figure 11.

5. Click Report to generate a report.

Step 4: Customize IDA reports

IDA customers often have special requirements for the content and format of their model reports. IDA integrated and extended Eclipse BIRT since IDA V7003 to enable customers to customize the reports using BIRT designer to meet their special needs. Because a logical or physical model is an abstraction of the real-world system, it is usually very complicated, and it consists of many building blocks and relationships.

IDA has provided many built-in reports that can be used as templates to help you start with customization. The report list in Figure 9 shows you pairs of BIRT reports in which one member of the pair is labeled Blank (such as the Physical Data Model Report and Blank Physical Data Model Report). In general, reports such as the Physical Data Model and Column reports can be applied directly to a physical model to generate reports, while the blank reports are used as templates for customization. You can copy and paste a built-in report with .rptdesign extension from the Report Explorer, and then open the copy in the BIRT designer to display the report design. If you open a copy of the Column Report, you find defined Data Source and Data Sets, as well as a presentation design, as shown in Figure 12. The Blank Column Report only has the defined Data Source and Data Sets.

The presentation part is empty so that you can create it, as shown in Figure 13.

As a report customization example, you can update the column report to include the masking method, which was added as a new property in Part 1 of this series. You can start the customization either from the column report or the blank column report. Because it's best to keep most of the presentations from the column report, it saves time to start from this report.

1. Copy the column report, right-click the physical data model folder, and paste it as Column Report with Privacy (set file name as ColumnWithPrivacy.rptDesign and select a folder in the paste report dialogs), as shown in Figure 14.

2. Double-click Column Report with Privacy to open it in the BIRT report designer.

If you saved the masking method property as an eAnnotation entry in Part 1 of this series, you need to add it to the Column data set to display it in your report.

To create a new column mapping for the Column data set, complete the following steps:

1. Right-click Column data set in the Data Explorer, and select Edit, as shown in Figure 15.

2. Select Column Mapping, click on the drop-down arrow, and select SQLTables:Column to browse in the Edit Data Set dialog. The browse tree is displayed.
   
   
3. Expand the browse tree, select eAnnotations/details/value, and click the > button in the middle.
   
   
4. Type in Masking Method as the name, and append [1] to the query, as shown in Figure 16.

5. Click Preview Results to preview the newly added column, and click OK. The Masking Method column mapping is created for the Column data set, as shown in Figure 17.

Follow these steps to add a Masking Method field to the report:

1. Insert a column to the right of the Documentation column, as shown in Figure 18.

2. Drag the newly created Masking Method column mapping from the Data Explorer, and drop it in the table details row of the inserted column, as shown in Figure 19.

3. Change the label of the inserted column to be Masking Method.
   
   
4. Click File > Save.

When you generate a report using Column Report with Privacy for the SAMPLE model, you see a Masking Method field and HASHING as the value for BONUS column, as shown in Figure 20.

Back to top

Add validation rules

At any time when you are building a data model, you can analyze the model to verify that it is compliant with the defined constraints. Based on EMF validation framework, IDA provides built-in constraints that not only ensure model integrity, but also help to improve the model quality by providing design suggestions and best practices. In this section, learn about how to extend the IDA built-in constraints to add a new constraint that checks for the existence of a Masking method when a column is set for privacy data.

EMF validation framework

The EMF validation framework provides support for constraint definitions for any EMF meta-model (batch and live constraints), customizable model traversal algorithms, constraint parsing for languages, configurable constraint bindings to application contexts, and validation listeners. (See Resources for more information about Eclipse EMF validation framework).

IDA built-in constraints

IDA model validation uses and extends the EMF validation framework. IDA provides comprehensive built-in constraints to do a model syntax check and to provide design suggestions for logical and physical models, as shown in Figure 21.

You can choose to enable or disable any of the constraints either through Preferences or the Analyze Model dialog. When you right-click a database or schema in a physical data model or a package in a logical data model from the Data Project Explorer and select Analyze Model, the validation results on the enabled constraints appear in the Problems view, as shown in Figure 22.

Add a new constraint

The org.eclipse.emf.validation.constraintProviders extension point is used to add constraints into the model validation framework (see Resources for more information about this extension point). Constraints are grouped into hierarchically structured categories. A constraint category defines the following attributes:

• id—Identifier for the category. The ID is a hierarchical name, delimited by slashes, relative to the ID of the containing category element (if any).
  
  
• name—The localized name of the category.
  
  
• mandatory—Indicates whether the category is mandatory.

IDA defined the constraint categories shown in Listing 1.

				
   <extension
      id="com.ibm.datatools.validation"
      name="Datatools Constraint Provider"
      point="org.eclipse.emf.validation.constraintProviders">
      <category
         name="%VALIDATION.CATEGORY.PHYSICAL"
         id="com.ibm.datatools.validation.physicalmodel">
         <category
            name="%VALIDATION.CATEGORY.SYNTAX"
            id="syntax">
            <category
               name="%VALIDATION.CATEGORY.DATATYPE"
               id="datatypes">
               %VALIDATION.CATEGORY.DATATYPE_DESC       
            </category>
            <category
               name="%VALIDATION.CATEGORY.SQL"
               id="sql_statement">
               %VALIDATION.CATEGORY.SQL_DESC
            </category>
            <category
               name="%VALIDATION.CATEGORY.OBJECTNAME"
               id="object_names">
               %VALIDATION.CATEGORY.OBJECTNAME_DESC
            </category>
            <category
               name="%VALIDATION.CATEGORY.KEY_CONSTRAINT_INDEX"
               id="key_constraints">
               %VALIDATION.CATEGORY.KEY_CONSTRAINT_INDEX_DESC
            </category>
            <category
               name="%VALIDATION.CATEGORY.IDENTITY_COLUMN"
               id="identity_columns">
               %VALIDATION.CATEGORY.IDENTITY_COLUMN_DESC
            </category>
            %VALIDATION.CATEGORY.SYNTAX_DESC
         </category>
         %VALIDATION.CATEGORY.PHYSICAL_DESC
      </category>
   </extension>
      

Constraint providers target one or more EPackages by namespace URI. A group of constraints declares categories in which they are members. Each constraint has a variety of meta-data associated with it. The following attributes are used to define a constraint:

• id—A unique identifier for the constraint.
  
  
• name—A localizable name for the constraint (appears in the GUI).
  
  
• lang—Identifies the language in which the constraint is expressed. The language is not case-sensitive.
  
  
• severity—The severity of the problem if the constraint is violated. This correlates to the severity of tasks in the Tasks view of the Eclipse environment.
  
  
• statusCode—The plug-in unique status code, useful for logging.
  
  
• class—For Java language constraints only, identifies a class implementing the constraint.
  
  
• mode—Describes whether a constraint operates in batch mode, live mode, or feature mode.

Now, follow these steps to add a new constraint to check for the existence of a masking method if a column is set as privacy data:

1. Add a new constraint using the extension point in the plugin.xml file, as shown in Listing 2.

				
<!-- add a new constraint -->
<extension point="org.eclipse.emf.validation.constraintProviders">           
   <constraintProvider>
      <package namespaceUri="http:///org/eclipse/datatools/modelbase/sql/schema.ecore"/>
      <package namespaceUri="http:///org/eclipse/datatools/modelbase/sql/tables.ecore"/>
      <package namespaceUri="http:///org/eclipse/datatools/modelbase/derby/derby.ecore"/>
      <constraints
         categories="com.ibm.datatools.validation.physicalmodel/design/normalization">
         <constraint
            name="Column privacy"
            severity="WARNING"
            statusCode="10"
            class="com.ibm.extendrda.sample.validation.PrivacyDataCheck"
            lang="Java"
            mode="Batch"
            id="com.ibm.datatools.extendValidation.PrivacyDataCheck">
            <description>
               Discover columns that are defined as privacy data,
                  but do not have masking method defined
            </description>
            <param name="extraction" value="elementName"/>
            <message>
               Column {0} is defined as privacy data, 
                  but does not have a masking method defined
            </message>
            <target
               class="Column">
            </target>
         </constraint>            
      </constraints>
   </constraintProvider>         
</extension>			
    

The code in Listing 2 adds the column privacy constraint under the design and normalization category as batch mode with a severity of warning, as shown in Figure 23.

2. Add the implementation class, as shown in Listing 3.

				
public class PrivacyDataCheck extends AbstractModelConstraint {

	public IStatus validate(IValidationContext ctx) {
		EObject target = ctx.getTarget();
		if (target instanceof Column) {
			Column col = (Column) target;
			if (isPrivateData(col) && (!hasMaskingMethod(col))) {
				ctx.addResult(col);
				return ctx.createFailureStatus(
					new Object[] { col.getName() });
			}
		}
		return ctx.createSuccessStatus();
	}

	private boolean isPrivateData(Column column) {
		EAnnotation eannotation = column
			.getEAnnotation(SamplePropertySection.SAMPLE_EANNOTAITN_NAME);
		if (eannotation == null)
			return false;
		String privacyStr = (String) eannotation.getDetails().get(
			SamplePropertySection.SAMPLE_PRIVACY_PROPERTY_NAME);
		if (privacyStr == null)
			return false;
		return Boolean.getBoolean(privacyStr);
	}

	private boolean hasMaskingMethod(Column column) {
		EAnnotation eannotation = column
			.getEAnnotation(SamplePropertySection.SAMPLE_EANNOTAITN_NAME);
		if (eannotation == null)
			return false;
		String maskingStr = (String) eannotation.getDetails().get(
			SamplePropertySection.SAMPLE_MASKING_PROPERTY_NAME);
		if ((maskingStr == null) || (maskingStr.length() <= 0))
			return false;
		return true;
	}
}

Now, if you set a column in the SAMPLE model as privacy data and leave its masking method empty, you get a warning from the added constraint in the Problem view when you run Analyze Model on SAMPLE, as shown in Figure 24.

Back to top

Conclusion

IDA as a comprehensive modeling and integration tool is very extensible by design. In this Part 2 of this two-part series, you learned how to generate customized model reports using BIRT and to add validation constraints to enforce business rules. When you combine these with how to programmatically traverse and modify IDA models and add custom properties from Part 1, you are able to extend IDA to meet your data modeling and integration requirements.

Resources

Learn

• Extend IBM InfoSphere Data Architect to meet your specific data modeling and integration requirements series (developerWorks):
  • "Extend IBM InfoSphere Data Architect to meet your specific data modeling and integration requirements, Part 1: Modifying IDA models and customizing properties" (developerWorks, June 2009): Programmatically traverse and modify IDA models, and add and display custom properties.
  
  
• Learn more about InfoSphere Data Architect.
  
  
• Check out the articles and tutorials on the InfoSphere Data Architect page on developerWorks.
  
  
• Watch this demo for a great Introduction to InfoSphere Data Architect.
  
  
• The article entitled "Migrating from CA ERwin to InfoSphere Data Architect" has lots of best practice advice.
  
  
• Find more information about BIRT at BIRT project.
  
  
• Learn more about ODA at Open Data Access (ODA) Framework.
  
  
• Learn more about EMF validation framework.
  
  
• Find more about EMF Model Validation Constraint Providers.
  
  

Get products and technologies

• Download a trial version of InfoSphere Data Architect.
  
  

Discuss

• Participate in the discussion forum.
  
  
• Check out the Integrated Data Management experts blog or forum, or get involved in the Integrated Data Management community space.
  
  

About the author

Comments

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Introduction
• Generate customized model reports using BIRT
• Add validation rules
• Conclusion
• Resources
• About the author
• Comments

Next steps from IBM

• Try: InfoSphere Data Architect
• Demo: Integrated Data Studio in a real-world scenario
• Webcast: Five things you can do today to cut costs using agile development
• Briefing: Jazz and Rational Team Concert
• Buy: InfoSphere Data Architect

My developerWorks community

Dig deeper into Information Management on developerWorks

DB2 pureScale

See how true transparent scaling architecture can be achieved on distributed platforms

Special offers