Creating a NIEM IEPD, Part 4: Assemble the IEPD

Design an XML information exchange between U.S. government entities

In the first three articles of this series, you learned to model a NIEM exchange and define subset and extension schemas that implement that model. Now you take the final step and assemble the schemas, documentation, and all the other artifacts of an exchange into a complete NIEM-conformant IEPD. This article also describes the process of validating and publishing your IEPD.

Priscilla Walmsley, Managing Director, Datypic

Photo of Priscilla WalmsleyPriscilla Walmsley serves as Managing Director and Senior Consultant at Datypic. She specializes in XML technologies, architecture and implementation. Most recently, she has been working (through Trusted Federal Systems) with the U.S. Department of Justice on LEXS, a NIEM-based IEPD framework. She is the author of Definitive XML Schema (Prentice Hall, 2001), and XQuery (O'Reilly Media, 2007). In addition, she co-wrote Web Service Contract Design and Versioning for SOA (Prentice Hall, 2008). You can reach Priscilla at pwalmsley@datypic.com.



18 May 2010

Also available in Chinese Russian Japanese Portuguese

Information Exchange Package Documentation (IEPD) is a collection of documents describing a National Information Exchange Model (NIEM) exchange. It typically includes schemas, samples, documentation of various kinds, and rendering instructions. It also includes several NIEM-specific artifacts that are required in a NIEM-conformant exchange, such as a metadata document and a catalog file.

In this article series, I have worked through a simple example of a Theft Report IEPD (see Resources for links to the first three articles). Now that my UML model, Component Mapping Table (CMT), and schemas are complete, I have a few steps left to finish my IEPD:

Frequently used acronyms

  • CMT: Component Mapping Template
  • HTML: Hypertext Markup Language
  • IEPD: Information Exchange Package Documentation
  • NIEM: National Information Exchange Model
  • UML: Unified Modeling Language
  • XHTML: Extensible Hypertext Markup Language
  • XML: Extensible Markup Language
  • XSLT: Extensible Stylesheet Transformations
  1. Create more documentation-related artifacts to further explain the exchange. Specifically, I need to add:
    • Master documentation to provide a general overview of the exchange
    • A sample document to show a realistic message structure and to test the schemas
    • Rendering instructions to provide a human-readable view of the sample documents
  2. Assemble my IEPD in a NIEM-conformant way, which involves using the NIEM "Work with IEPDs" tool to upload my artifacts and generate additional documentation files.
  3. Validate my IEPD for completeness and NIEM conformance using the NIEM Conformance Validation Tool.
  4. Publish my IEPD so that it can be discovered by other users.

IEPD documentation

To be NIEM conformant, an IEPD must contain—at a minimum—some form of master documentation and a change log describing the changes since the last version. You can include any documentation that might typically accompany a software application in the IEPD (either in the master documentation or as separate files), such as:

  • UML models (sequence diagrams, use cases, class diagrams)
  • A CMT
  • Business rules (that is, constraints on the data that are not expressed in the schemas or in the model)
  • Requirements definitions
  • Testing and/or conformance statements
  • Memoranda of understanding and letters of endorsement

The documentation artifacts I have developed so far for the Theft Report IEPD are a UML model and a CMT. I need to add master documentation that describes in general terms the purpose and structure of the exchange. I also add a change log, which is essentially empty, because this is the first version of the exchange. See Downloads for a complete IEPD that contains these documents.


Creating sample documents

Sample XML documents are an important part of an IEPD. It's difficult to conceptualize XML documents based solely on a schema, particularly if you're talking about a complex set of interrelated schema documents. Using samples makes it clear which element is the root element in any given document type and provides examples of typical data for each element.

Include at least one sample for each different root element in the exchange schema. If you use a document type for multiple purposes, you should provide a sample for each purpose. It's helpful to create one sample that contains every possible element in order to test the schemas and any software written to process the documents. For a complex exchange, it may also be useful to create a typical sample that contains what is likely to be included in any given document.

Many XML editors generate sample documents for you, but those documents generally contain meaningless data and don't have correctly related ID and IDREF values used in the associations. You should edit any generated samples to make them more typical and meaningful. I also recommend validating your samples using multiple processors because of differences in the way various processors handle schema location hints.

For the Theft Report IEPD, I only have one possible root element (tr:TheftReport). Listing 1 shows the beginning of one complete sample that I created. It has at least one of each element type allowed in the exchange and has representative values for the data.

Listing 1. Beginning of a sample document
<?xml-stylesheet type="text/xsl" href="theftreport.xsl" ?>
<tr:TheftReport xmlns:tr="http://datypic.com/theftreport/exchange/1.0"
 xmlns:nc="http://niem.gov/niem/niem-core/2.0"
 xmlns:trext="http://datypic.com/theftreport/extension/1.0"
 xmlns:j="http://niem.gov/niem/domains/jxdm/4.1"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:s="http://niem.gov/niem/structures/2.0"
 xsi:schemaLocation="http://datypic.com/theftreport/exchange/1.0
                     ../schema/exchange/1.0/theftreport-exchange.xsd">
    <tr:TheftReportDate>2006-05-05</tr:TheftReportDate>
    <trext:Theft s:id="T1">
        <nc:ActivityDate>
            <nc:DateTime>2006-05-04T08:15:00</nc:DateTime>
        </nc:ActivityDate>
    </trext:Theft>
    <trext:Theft s:id="T2">
        <nc:ActivityDate>
            <nc:DateTime>2006-05-04T09:14:00</nc:DateTime>
        </nc:ActivityDate>
    </trext:Theft>
        <nc:ActivityConveyanceAssociation>
            <nc:ActivityReference s:ref="T1"/>
            <nc:ConveyanceReference s:ref="V1"/>
        </nc:ActivityConveyanceAssociation>
        <nc:ActivityConveyanceAssociation>
            <nc:ActivityReference s:ref="T2"/>
            <nc:ConveyanceReference s:ref="B1"/>
        </nc:ActivityConveyanceAssociation>
        <trext:Vehicle s:id="V1">
            <nc:ItemDescriptionText>2001 Subaru Outback</nc:ItemDescriptionText>
            <nc:ItemSerialIdentification>
                 <nc:IdentificationID>123455234234</nc:IdentificationID>
            </nc:ItemSerialIdentification>
            <nc:VehicleColorPrimaryCode>SIL</nc:VehicleColorPrimaryCode>
            <nc:ConveyanceRegistrationPlateIdentification>
                 <nc:IdentificationID>BGE112</nc:IdentificationID>
            </nc:ConveyanceRegistrationPlateIdentification>
            <trext:VehicleTaxClassCode>4</trext:VehicleTaxClassCode>
        </trext:Vehicle>
    ...

Creating rendering instructions

Rendering instructions are another useful tool to help users and implementers of the IEPD conceptualize the exchange. It's difficult to look at a complex XML document and understand its content—particularly in a typical NIEM document, which has a lot of structural elements and associations that connect different parts of a document. Some users get caught up in the complexity of the structure and the names of elements rather than focusing on the actual content.

If you write rendering instructions that turn the XML data into human-readable HTML, you can alleviate this problem. Associated objects can be rejoined and displayed together on the page, extraneous structure can be ignored, and cryptic code list values can be turned into their readable equivalents.

Rendering of XML documents is typically done with an XSLT stylesheet. I generally recommend sticking to XSLT version 1.0 for simple HTML rendering, because that version is supported in major browsers. Providing an XSLT stylesheet in the same directory as the samples allows users to double-click a sample (or open it in an XML editor) and see the rendered HTML version. The processing instruction (starting with <?xml-stylesheet) on the first line of Listing 1 indicates which XSLT stylesheet to use.

For the Theft Report IEPD, I created a fairly simple XSLT 1.0 stylesheet (included in the IEPD in Downloads) that presents the Theft Report in a user-friendly way, as in Figure 1. The XSLT joins each theft with its associated location, vehicle, victim, and witness to present the data logically. (View a text-only version of Figure 1.)

Figure 1. Rendered sample
Screen capture of rendered sample reports

Assembling an IEPD

NIEM provides a "Work with IEPDs" tool for gathering your IEPD artifacts and assembling the complete package (see Resources for a link). Although it is possible to assemble an IEPD by hand, using the tool is generally easier, because it allows you to automatically generate two NIEM-specific artifacts that are required for NIEM conformance: the metadata document and the catalog file, which serves as a table of contents. It also results in an IEPD that has a more consistent file structure.

To use the tool, click Create/Upload IEPD from the menu on the left side of the page. Doing so takes you to the page in Figure 2.

Figure 2. Create/upload an IEPD page
Screen capture of the Create/upload an IEPD page in the Work with IEPDs tool

After you click Begin, you will be asked whether you have an existing IEPD zip file that you want to upload. If you already have a zipped set of IEPD artifacts, you can click Yes and upload the zip file. When working with the tool for the Theft Report example, I clicked No to upload each file individually.

When you click No, the tool displays the page in Figure 3. This page allows you to specify a root directory name. For the Theft Report, I choose TheftReport as the root directory. Also on this page, to add individual artifacts to the IEPD, click the Add Artifact link.

Figure 3. Upload Artifacts page
Screen capture of the Upload Artifacts page in the Work with IEPDs tool

You add the extension and exchange schemas, wantlist, subset, sample document, rendering instructions, CMT, UML model, change log, and master documentation using this page. Note that you should zip the subset schemas into one file; when the tool regenerates the IEPD, it will be unzipped.

For each artifact you add, you choose a type. Doing so is important, because it affects the way the catalog file is generated and the default directories used for the components. You can also choose a directory path on this page. For consistency, click use recommended path for each artifact, and the tool fills in the path. The only change I made to the recommended path was to include a 1.0 subdirectory for the exchange and extensions schemas, because I wanted the directory structure to mirror the namespace name. I also added a description for each artifact. Figure 4 shows the results. (View a text-only version of Figure 4.)

Figure 4. Added artifacts
Screen capture of added artifacts on the Edit Artifacts page, includes list of artifacts

When you are done adding artifacts, click Next to go to the Enter Metadata page in Figure 5. Here, you provide metadata about the IEPD, such as name, description, version, organization, and point of contact. The tool uses this information to generate an XML document called metadata.xml that is included with the IEPD.

Figure 5. Enter Metadata page
Screen capture of the Enter Metadata page in the Work with IEPDs tool

One thing that is slightly confusing about this page is that the fields required for NIEM conformance are not all marked with yellow asterisks. Later, when you validate your IEPD, you will be informed if you missed any required metadata. In general, it's best to fill in as many as are relevant. When you're done with the metadata, click Next to go to the summary page in Figure 6.

Figure 6. IEPD summary page
Screen capture of the IEPD Details summary page in the Work with IEPDs tool

From here, you can click Validate IEPD to determine whether your IEPD contains all the necessary metadata. When it is validated, click Upload IEPD to save the IEPD in your profile so that you can retrieve it under My IEPDs later. Note that uploading your IEPD does not automatically make it visible to other users of the Web site.

Finally, to actually generate the IEPD, click Download from the IEPD Successfully Uploaded page. You can download, modify, or delete any of your IEPDs at any time by clicking My IEPDs from the menu on the left side of the tool pages.

Looking at the generated IEPD, you can see a copy of all of the artifacts that you uploaded plus two new generated files. The metadata.xml document, in Listing 2, provides a standardized cross-IEPD format for information about the exchange.

Listing 2. The metadata document
<?xml version="1.0" encoding="UTF-8"?>
<Metadata>
    <URI>http://www.datypic.com/theftreport</URI>
    <Name>Theft Report</Name>
    <Summary>Exchange to report thefts of motor vehicles and bicycles in the state
                on a daily basis</Summary>
    <Description>Contains information about a theft of a motor vehicle or bicycle,
                including the theft date, location, description and identifiers of
                the stolen property, and victim and witness information.</Description>
    <Version>1.0</Version>
    <URL>http://www.datypic.com/theftreport</URL>
    <CreationDate>03/01/2010</CreationDate>
    <LastRevisionDate>03/01/2010</LastRevisionDate>
    <NextRevisionDate>06/01/2010</NextRevisionDate>
    <NIEMVersion>2.0</NIEMVersion>
    <Security>Public</Security>
    <Maturity>2</Maturity>
    <Status>Final</Status>
    <Schedule/>
    <Lineage/>
    <Relationships/>
    <Keywords/>
    <Domain>Justice, </Domain>
    <ExchangePartners/>
    <Process/>
    <TriggeringEvent/>
    <Conditions/>
    <Endorsements/>
    <Sponsors>Datypic</Sponsors>
    <Purpose>Report thefts of motor vehicles and bicycles to interested parties</Purpose>
    <MessageExchangePatterns>publish/subscribe</MessageExchangePatterns>
    <CommunicationsEnvironment/>
    <ExchangePartnerCategories>Law Enforcement, DMV,
                                  Department of Revenue</ExchangePartnerCategories>
    <AuthoritativeSource>
        <Category>none</Category>
        <Organization>
            <Name>Datypic</Name>
            <Address1/>
            <Address2/>
            <City/>
            <State/>
            <Zip/>
            <Country/>
            <URL>http://www.datypic.com</URL>
        </Organization>
        <PointOfContact>
            <Name>Priscilla Walmsley</Name>
            <Address1/>
            <Address2/>
            <City/>
            <State/>
            <Zip/>
            <Country/>
            <Phone>231-555-1212</Phone>
            <Fax/>
            <Email>pwalmsley@datypic.com</Email>
        </PointOfContact>
    </AuthoritativeSource>
</Metadata>

The catalog.html file is an XHTML document that serves as a table of contents of the artifacts in the IEPD. It is shown in its rendered human-readable form in Figure 7. The catalog is also machine-readable, thanks to rddl:purpose embedded in the XHTML. (View a text-only version of Figure 7.)

Figure 7. Rendered IEPD catalog file
Screen capture of rendered IEPD catalog file

Validating the IEPD

Once you create an IEPD, you can test it for conformance using the NIEM Conformance Validation Tool (see Resources for a link). This process ensures that all required artifacts are present in the IEPD. It also tests the schemas against some of the rules for NIEM-conformant schemas defined in the NIEM Naming and Design Rules (NDR) document.

From the Conformance Validation Tool main page, click Begin to bring up the page shown in Figure 8. It prompts you to upload a file, which will be your entire IEPD as a zip file. The tool uses the catalog file to determine the location and type of all artifacts. After you click through the page that asks you to verify the purpose of your artifacts, a new section of this page, called My Validations, appears with a conformance report for your IEPD.

Figure 8. Conformance tool
Screen capture of Conformance Validation tool in the Work with IEPDs tool

Figure 9 shows the summary page of the conformance report, which is in Microsoft® Office Excel® format. It has several worksheets:

  • Summary is the basic statistical information in Figure 9.
  • NDR - All Rules provides a list of all the rules in the NDR and, for those that can be automatically checked, whether the schemas in the IEPD passed. For each rule that cannot be checked automatically, a drop-down list allows you to indicate that you checked it manually.
  • NDR - Schemas is a summary of all of the schema documents and whether they are NIEM conformant.
  • NDR - Rules Auto Failed is a list of each instance of an NDR rule violation.
  • IEPD - Metadata is a list of all of the metadata fields, signaling an error if they are required but absent.
  • IEPD - Catalog is a list of all of the artifact types, signaling an error if they are required but absent.

(View a text-only version of Figure 9.)

Figure 9. Conformance report summary
Screen capture of Conformance report summary

A complete copy of the conformance report for the Theft Report IEPD is available from Downloads.


Submitting an IEPD to public repositories

IEPDs are meant to be shared. Although NIEM helps interoperability by providing common definitions and techniques, XML documents from two IEPDs that use different subsets of NIEM are not interchangeable. Therefore, reuse of IEPDs is critical to achieving interoperability.

For others to use your IEPD, it has to be available publicly for others to discover. Two major Web sites provide directories of existing IEPDs, and submitting your IEPD to these sites is easy.

First, you can list your IEPD on the NIEM Web site itself to make it available under the Search function of the Work with IEPDs tool. To do this within the Work with IEPDs tool, bring up your IEPD summary page. Click the Edit link in the top right, then click Edit Visibility/Sharing to bring up the Edit Artifact Visibility page in Figure 10.

Figure 10. Editing IEPD visibility
Screen capture of the Editing IEPD visibility page in the Work with IEPDs tool

From here, select Shared to make the IEPD visible. You can also hide individual artifacts (by clearing the check box beside them) even if the IEPD itself is visible. This allows users to be aware of the IEPD even if individual artifacts are classified. Click Update Visibility when you're done.

The second place to submit your IEPD is to the Office of Justice Programs (OJP) IEPD Clearinghouse. To do this, visit the clearinghouse Web site (see the link in Resources), and click Submit IEPD Information to get a form on which you can upload your IEPD.


LEXS: An alternative to creating an entire IEPD

Now that you understand the process of developing a NIEM IEPD, consider again whether this is a task you should undertake. Part 1 of this series suggested that you consider reusing an existing IEPD before you create your own from scratch. Not only will you improve your interoperability with other applications, but you will also save yourself a lot of work.

However, it is sometimes difficult to find an IEPD that meets all of your requirements. In these cases, a good solution is Logical Entity Exchange Specification (LEXS). LEXS is a NIEM IEPD framework that balances the competing goals of interoperability and flexibility by separating documents into a digest and a structured payload. The digest, which contains the most commonly used NIEM components, has a fixed structure and is interoperable across all LEXS-based IEPDs. The structured payload allows individual LEXS-based IEPDs to extend and customize the LEXS base model. LEXS also provides solutions for handling message exchange, search, subscriptions, attachments, and rendering. See Resources for a link to the LEXS Web site.


Conclusion

This is the final article in a series that describes the process of creating a NIEM IEPD. Throughout this series, you saw how to model an exchange, create an appropriate NIEM subset for the model, and write your own extensions of NIEM. This article has described the final step: assembling the schemas, documentation, and other artifacts into an IEPD. Following these guidelines for a NIEM-conformant exchange will help you to capitalize on the promise of NIEM: To facilitate information sharing among public and private-sector organizations.


Downloads

DescriptionNameSize
Complete IEPDniem4iepd.zip157KB
IEPD Conformance Reportniem4confreport.zip26KB

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into XML on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=XML, Industries
ArticleID=489967
ArticleTitle=Creating a NIEM IEPD, Part 4: Assemble the IEPD
publish-date=05182010