Introduction

In today's ever-changing business world, more and more developers are looking for ways to design and build applications that can communicate with a variety of third-party applications and services. Developers have always had a choice of development environments and tools, but with those choices came communication methods and protocols that required extensive coding to enable two distinct applications to communicate with one another.

A popular method of such communication is known as Electronic Data Interchange (EDI). EDI is the transfer of information between systems via such mediums as the Internet. A great many businesses adopted EDI to place orders, send invoices, and otherwise exchange information. As EDI became more and more popular, it eventually became known as the X12 standard, ratified by the American National Standards Institute (ANSI).

When EDI communications became commonplace, developers often had to enhance applications more than once to accommodate changes in how their companies did business with other companies. This became costly to businesses, requiring development resources to continuously maintain EDI-based applications. Businesses had to evolve to survive, and with that evolution came the increased need to update data exchange methods.

In 1986, the International Organization for Standardization developed and standardized a new system for defining information. This system is known as Standard Generalized Markup Language (SGML), and became the ISO 8879 standard. SGML is a complex system that specifies rules for tagging elements in a document, which can then be used for formatting information in various ways. With the growth of the Internet, another markup language became popular, fostering more interest in SGML, since this new markup language used SGML rules. This technology is known as HyperText Markup Language (HTML).

HTML allows the creation of specially formatted documents that support links to other documents and the embedding of graphic images, audio, and video files. Tools, known as Web browsers, were created to display this form of document. HTML uses special tags that define the content and layout of the document, which helps to create forms and other content-rich documents that make the World Wide Web what it is. As powerful as HTML has become, it is still not well-suited for data interchange between disparate systems.

An overview of XML

What is XML? XML stands for eXtensible Markup Language, and is a specification that was developed by the World Wide Web Consortium (W3C®). It allows application developers to create their own customized tags, enabling the definition, transmission, validation, and interpretation of data between applications and between organizations.

XML is a very flexible text format that is derived from SGML. Originally designed to meet the challenges of large-scale electronic publishing, XML is now playing an increasingly important role in the exchange of a wide variety of data on the Web and elsewhere.

Some key points of XML are:

• Tags can mean anything, describing what is relevant in your data.
• An XML document is highly structured.
• The structure is a hierarchy of nested data elements.
• XML efficiently provides description for one-to-one or one-to-many relationships.
• It is a perfect fit in the MultiValue world.

XML documents produced are most often "well-formed" to show both logical and physical structure. The structure is defined in the XML specification, which can be found on the W3C website. XML is an open and evolving standard which is designed to be easy to create and use.

Documents are described by the use of a document type definition (DTD) or XML schema (XSD), which are XML documents that describe the data contained within the primary document. Applications use DTDs and XML schemas to "map" relevant data within XML documents to their local database.

The Document Object Model (DOM) provides a standard way to manipulate XML documents. The DOM API, explained later in this article, may be used to delete, remove, and update an XML document, as well as create new XML documents.

The DOM represents a document as a tree of nodes. Each node has a parent (except for the "root" node), and optional children. The DOM provides functions to traverse and manipulate these nodes. Another technology, XPath (also supported in the XML for U2 BASIC API), provides the ability to locate nodes in the DOM based on search criteria.

Unlike traditional first normal form databases, U2 data files are three dimensional in nature. A data record contains a unique item ID (record key) and fields that can repeat horizontally or vertically. System delimiters are used to separate the item ID from the data section, and to separate the data elements themselves. This type of data model is known as MultiValue or Nested Relational.

Figure 1 below illustrates the contents of a CUSTOMER record in the HS.SALES account shipped with UniVerse.

In Figure 1 above, the item ID "2" is displayed at the top of the record. Each attribute (field) is numbered to make it more human-readable, and each value within each attribute is separated by a system delimiter. If this record contained subvalued data, it would contain different delimiters to separate the subvalues within each value. This form of data storage saves file space and allows related data to be stored together, improving retrieval performance. (For more information on the Nested Relational data model, please review the Nested Relational Databases White Paper (see Resources).

XML is a natural partner for U2 databases because of its nested nature and ease of use.

You can display similar information in U2, as shown in Listing 1 below.

:LISTDICT STUDENT

SORT DICT STUDENT TYP LOC CONV MNAME FORMAT SM ASSOC BY TYP BY LOC BY @ID 11:06:
10 Dec 15 2005 1
@ID............ TYP LOC.......... CONV MNAME.......... FORMAT SM ASSOC.....

@ID             D               0      STUDENT         12R### S
                                                       -##-##
                                                       ##
ID              D               0      STUDENT         12R### S
                                                       -##-##
                                                       ##
LNAME           D               1      Last Name       15T    S
FNAME           D               2      First Name      10L    S
MAJOR           D               3      Major           4L     S
MINOR           D               4      Minor           4L     S
ADVISOR         D               5      Advisor         8L     S
SEMESTER        D               6      Term            4L     MV CGA
COURSE_NBR      D               7      Crs #           5L     MS CGA
COURSE_GRD      D               8      GD              3L     MS CGA
COURSE_HOURS    I   TRANS('COURSE      Hours           5R     MS CGA
                    S',COURSE_NBR
                    ,CREDITS,'X')
COURSE_NAME     I   TRANS('COURSE      Course Name     25L    MS CGA
                    S',COURSE_NBR

                    ,'NAME','X')
CGA             PH  SEMESTER COUR
                    SE_NBR COURSE
                    _NAME COURSE_
                    GRD COURSE_HO
                    URS TEACHER
@ORIGINAL       SQ            @ID
@SYNONYM        SQ             ID
GPA1            V   SUBR('GPA1',C MD3  GPA             5R     S
                    OURSE_HOURS,C
                    OURSE_GRD)
TEACHER         V   TRANS('COURSE      Teacher         10L    MS CGA
                    S',COURSE_NBR
                    ,'TEACHER','X
                    ')
17 records listed



:

U2 XML features

The primary goal for XML support within UniData and UniVerse is to provide a standard means of sharing information between external systems. U2's XML support aims to:

• Produce XML documents from SQL and UniQuery (UniData) and RetrieVe (UniVerse).
• Provide a command that updates U2 files from XML documents.
• Provide a command that produces XML documents from U2 files.
• Help you to create robust applications using powerful APIs built on open source standards.
• Work within client/server applications, such as with UniObjects for Java or UniObjects for .NET.
• Provide a simple means of mapping U2 data to XML documents through the use of an Eclipse-based graphical tool.

XML in U2 is managed through the use of handles, similar to files and sockets. The structures used by the XML APIs remain internal to the database run machine. They provide greater performance over arrays due to their advanced design. Functions are included to copy XML documents to and from these structures, including writing them to disk.

U2 provides an XML repository in each new account. For UniData, this file is a directory called _XML_, while in UniVerse the directory is called <XML>. These names remain in line with their respective historical naming conventions. Each new account created in UniData will have a directory called _XML_ installed automatically. UniVerse will create its &XML& the first time an XML function or command references it, whether expressed or implied.

To help control the footprint of user processes, the XML libraries are dynamically loaded on both UNIX and Windows systems. They will load when needed, thus a user process that does not execute an application or command leveraging U2 XML capabilities will not load the libraries. If you are planning to leverage this technology on UNIX systems, you will need to set an environment variable to enable the use of shared libraries. The typical methods of transport of XML from U2 BASIC are callHTTP and SOAP (SOAP is described later in this article).

Because the XML capabilities are built directly into each U2 database, the power of XML can be more efficiently realized.

Using UniQuery and RetrieVe to produce XML

With the release of UniData V6.0 and UniVerse V10.0, the first U2 XML capabilities were introduced. A significant part of the offering was the introduction of several keywords that enable the production of XML documents from UniData UniQuery, UniVerse RetrieVe, and from the SQL engines of both databases.

Using the UniQuery/RetrieVe LIST and SORT statements, you can generate XML documents and DTDs. This is called "XML OUT," meaning that U2 queries can output in XML format. UniData V6.1 and subsequently UniVerse V10.1 added the ability to produce XML schemas, as well as BASIC APIs, which will be discussed in a later section of this article.

The following is a list of query keywords added to UniData V6.1 and UniVerse V10.1, and present in all subsequent releases. They may be used with LIST, SORT, and SQL SELECT statements.

• TOXML -- directs processing to the XML parser
• ELEMENTS -- directs the XML parser to construct the document as "element-centric" (the default is attribute-centric)
• WITHDTD -- precedes the XML document with a DTD
• WITHSCHEMA -- precedes the XML document with an XML schema
• XMLDATA -- use in a SQL statement to specify an extraction file
• XMLMAPPING -- instructs the parser to look for a user-specified XMAP file
• TO (pre-existing for UniData, new for UniVerse) -- redirects output to a file (the _XML_ or &XML& file by default)

Listing 2 below illustrates how to produce a DTD and XML document from a LIST statement.

:LIST STUDENT ALL WITH FNAME = "Harry" TOXML WITHDTD
<?xml version="1.0"?>
<!DOCTYPE ROOT[
<!ELEMENT ROOT (STUDENT*)>
<!ELEMENT STUDENT ( CGA-MV* )>
<!ATTLIST STUDENT
        ID CDATA #REQUIRED
        LNAME CDATA #IMPLIED
        FNAME CDATA #IMPLIED
        MAJOR CDATA #IMPLIED
        MINOR CDATA #IMPLIED
        ADVISOR CDATA #IMPLIED
>
<!ELEMENT CGA-MV ( CGA-MS* )>
<!ATTLIST CGA-MV
        SEMESTER CDATA #IMPLIED
>
<!ELEMENT CGA-MS  EMPTY>
<!ATTLIST CGA-MS
        COURSE_NBR CDATA #IMPLIED
        COURSE_GRD CDATA #IMPLIED
>
]>
<ROOT>
<STUDENT ID = "521814564" LNAME = "Smith" FNAME = "Harry" MAJOR = "CH" MINOR = "
PY" ADVISOR = "Carnes">
  <CGA-MV SEMESTER = "FA93">
    <CGA-MS COURSE_NBR = "CS130" COURSE_GRD = "A"/>
    <CGA-MS COURSE_NBR = "CS100" COURSE_GRD = "B"/>
    <CGA-MS COURSE_NBR = "PY100" COURSE_GRD = "B"/>
  </CGA-MV>
  <CGA-MV SEMESTER = "SP94">
    <CGA-MS COURSE_NBR = "CS131" COURSE_GRD = "B"/>
    <CGA-MS COURSE_NBR = "CS101" COURSE_GRD = "B"/>
    <CGA-MS COURSE_NBR = "PE220" COURSE_GRD = "A"/>
  </CGA-MV>
</STUDENT>
</ROOT>
:

The DTD in the above example allows external applications to understand the contents of the XML document that follows. It is identifiable with the comment tag "DOCTYPE ROOT [" and closed by "]>" just prior to the actual XML document.

The format of the example XML document is attribute-centric, meaning the document is not nested with respect to the U2 data contained within it. Element-centric documents (like the one shown in Listing 3 below) are formatted to nest the elements using separate tags. This is useful when the focus is on multivalued data and not singlevalued data.

:LIST STUDENT ALL WITH FNAME = "Harry" TOXML ELEMENTS
<?xml version="1.0"?>
<ROOT>
<STUDENT>
  <ID>521814564</ID>
  <LNAME>Smith</LNAME>
  <FNAME>Harry</FNAME>
  <MAJOR>CH</MAJOR>
  <MINOR>PY</MINOR>
  <ADVISOR>Carnes</ADVISOR>
  <CGA-MV>
    <SEMESTER>FA93</SEMESTER>
    <CGA-MS>
      <COURSE_NBR>CS130</COURSE_NBR>
      <COURSE_GRD>A</COURSE_GRD>
    </CGA-MS>
    <CGA-MS>
      <COURSE_NBR>CS100</COURSE_NBR>
      <COURSE_GRD>B</COURSE_GRD>
    </CGA-MS>
    <CGA-MS>
      <COURSE_NBR>PY100</COURSE_NBR>
      <COURSE_GRD>B</COURSE_GRD>
    </CGA-MS>
  </CGA-MV>
  <CGA-MV>
    <SEMESTER>SP94</SEMESTER>
    <CGA-MS>
      <COURSE_NBR>CS131</COURSE_NBR>
      <COURSE_GRD>B</COURSE_GRD>
    </CGA-MS>
    <CGA-MS>
      <COURSE_NBR>CS101</COURSE_NBR>
      <COURSE_GRD>B</COURSE_GRD>
    </CGA-MS>
    <CGA-MS>
      <COURSE_NBR>PE220</COURSE_NBR>
      <COURSE_GRD>A</COURSE_GRD>
    </CGA-MS>
  </CGA-MV>
</STUDENT>
</ROOT>
:

The following is an example of how to create a simple XML document from a UniVerse SQL statement:

:SELECT FNAME, LNAME, MAJOR, MINOR, SEMESTER FROM STUDENT.F 
WHERE FNAME = 'Harry' TOXML;


<?xml version="1.0" encoding="UTF-8"?>
<ROOT>
<STUDENT.F FNAME = "Harry" LNAME = "Smith" MAJOR = "CH" MINOR = "PY">
  <STUDENT.F_CGA-MV SEMESTER = "FA93"/>
  <STUDENT.F_CGA-MV SEMESTER = "SP94"/>
</STUDENT.F></ROOT>
:

Unlike UniData, UniVerse treats subvalues in queries similar to multivalues. Because of this, UniVerse needs an additional mapping file to create a properly formed XML document from a query where subvalues are to be displayed. If only multivalues are required in the query, a mapping file is optional.

DB.TOXML command

UniData and UniVerse can populate data files from XML documents through the use of a command called DB.TOXML. This is a program that uses the XML capabilities to extract data from a U2 file and generate an XML document for use by external applications.

The syntax for the DB.TOXML command is:
DB.TOXML "xml_doc_filename" "xmap_filename" "condition"
where xml_doc_filename is the name of the XML document to be generated, xmap_filename is the name of the mapping file (also known as the XMAP file) which maps the data to an XML document, and condition is any conditional query phrase, such as WITH SCHOOL = "CO002".

The DB.TOXML command queries the database using any optional conditions and generates an XML document. To perform this action, DB.TOXML creates space in memory to hold the XML document it will create, and also loads in memory the mapping file to be used to format the XML document.

:DB.TOXML DW.XML DW.MAP "WITH FNAME = 'Harry'"
DB.TOXML success
:CT _XML_ DW.XML
_XML_:

DW.XML
<?xml version="1.0" encoding="UTF-8"?>
<ROOT>
  <STUDENT ID="521814564" FNAME="Harry" LNAME="Smith" MAJOR="CH" MINOR="PY" ADVI
SOR="Carnes">
    <CGA-MV SEMESTER="FA93">
      <CGA-MS COURSE_GRD="A" COURSE_NBR="CS130"/>
      <CGA-MS COURSE_GRD="B" COURSE_NBR="CS100"/>
      <CGA-MS COURSE_GRD="B" COURSE_NBR="PY100"/>
    </CGA-MV>
    <CGA-MV SEMESTER="SP94">
      <CGA-MS COURSE_GRD="B" COURSE_NBR="CS131"/>
      <CGA-MS COURSE_GRD="B" COURSE_NBR="CS101"/>
      <CGA-MS COURSE_GRD="A" COURSE_NBR="PE220"/>
    </CGA-MV>
  </STUDENT>
</ROOT>
:

In the above example, the XMAP file "DW.MAP" was created using the IBM U2 XML/DB Tool (XMAP and the U2 XML/DB Tool described later in this article) and an XML schema named DW.xsd. Listing 6 shows how the schema was created.

:SORT STUDENT ALL TOXML WITHSCHEMA TO DW
:CT _XML_ DW.xsd
_XML_:

DW.xsd
<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <xsd:annotation>
  <xsd:documentation xml:lang="en">
    account: C:\IBM\ud61\demo
    command: SORT STUDENT ALL TOXML WITHSCHEMA TO DW
  </xsd:documentation>
  </xsd:annotation>
  <xsd:element name="ROOT">
    <xsd:complexType>
    <xsd:sequence maxOccurs="unbounded">
      <xsd:element ref="STUDENT" maxOccurs="unbounded"/>
    </xsd:sequence>
    </xsd:complexType>
  </xsd:element>
  <xsd:element name="STUDENT">
    <xsd:complexType>
    <xsd:sequence maxOccurs="unbounded">
      <xsd:element ref="CGA-MV" minOccurs="0" maxOccurs="unbounded"/>
    </xsd:sequence>
    <xsd:attribute name="ID"/>
    <xsd:attribute name="LNAME"/>
    <xsd:attribute name="FNAME"/>
    <xsd:attribute name="MAJOR"/>
    <xsd:attribute name="MINOR"/>
    <xsd:attribute name="ADVISOR"/>
    </xsd:complexType>
  </xsd:element>
  <xsd:element name="CGA-MV">
    <xsd:complexType>
    <xsd:sequence maxOccurs="unbounded">
      <xsd:element ref="CGA-MS" minOccurs="0" maxOccurs="unbounded"/>
    </xsd:sequence>
    <xsd:attribute name="SEMESTER"/>
    </xsd:complexType>
  </xsd:element>
  <xsd:element name="CGA-MS">
    <xsd:complexType>
    <xsd:attribute name="COURSE_NBR"/>
    <xsd:attribute name="COURSE_GRD"/>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>
:

XML.TODB command

DB.TOXML was designed to provide an easy way to produce XML documents from the U2 query engines. Conversely, a means of populating the database from XML documents is also important. With this in mind, the XML.TOSB command was created to read XML documents and write the appropriate content to U2 files. The syntax for the XML.TODB command is:
XML.TODB <xml_document> <xmap_file>
where "xml_document" is the XML document name to be read and "xmap_file is the mapping file to be used.

XML.TODB reads the specified XML and XMAP files from disk and creates internal structures for them. Using the information stored in the XMAP file, XML.TODB determines the U2 file to open. The command then finds and extracts each record in the XML document and writes them to the opened file. This is accomplished by using the U2 XML APIs, XDOM, and XMAP.

Listing 7 demonstrates the use of the XML.TODB command. Everything it needs to perform its intended actions exists in the XML document and XMAP file supplied by the user, providing a simple means of updating U2 records and files from XML.

:LIST STUDENT                                :LIST STUDENT

LIST STUDENT 14:59:49 Jun 27 2004 1          LIST STUDENT 15:00:24 Jun 27 2004 1
STUDENT.....                                 STUDENT.....

 291-22-2021                                  291-22-2021
 424-32-5656                                  521-81-4564 <- New record
 414-44-6545                                  424-32-5656
 221-34-5665                                  414-44-6545
 978-76-6676                                  221-34-5665
5 records listed                              978-76-6676

:XML.TODB STUDENT.xml STUDENT.map            6 records listed
XML.TODB Success                             :
:

Displaying XML documents

UniData V6.1 and UniVerse V10.0 introduced a feature of displaying XML documents from ECL (UniData) and TCL (UniVerse) command lines. Doing so is a three-step process that includes preparing the system for an XML document, listing the contents of the XML file referencing an extraction file and any required U2 dictionary field, and then releasing the memory allocated by the prepare process.

PREPARE.XML

Use this function to prepare the U2 system listing XML documents. The syntax is as follows:
PREPARE.XML "xml_document" xml_data
Where xml_document is the name or path name for the document and xml_data is the name to assign the work area in memory for later reference in a query.

Here is an example of using PREPARE.XML:
PREPARE.XML "&XML&/MYSTUDENT.XML" STUDENT_XML
PREPARE.XML successful.

LIST XMLDATA

Use the RetrieVe LIST command with the XMLDATA option to list the XML data. The syntax is as follows:
LIST XMLDATA xml_data "extraction_file" [fields]
where xml_data is the name of the work area given in PREPARE.XML, extraction_file references the path of the extraction file (the quotes are required), and fields references the field names of the dictionary referenced in the extraction file (extraction file is explained later in this article).

Listing 8 below illustrates how to use LIST XMLDATA, based on the PREPARE.XML example above.

:LIST XMLDATA XML_DATA STUDENT.map ID FNAME LNAME MAJOR ID.SUP

LIST XMLDATA XML_DATA STUDENT.map ID FNAME LNAME MAJOR ID.SUP 11:59:13am
15 Dec 2005  PAGE    1

STUDENT....  221-34-5665
First Name. Susan
Last Name.. Miller
Major...... EG

STUDENT....  291-22-2021
First Name. jojo
Last Name.. Smith
Major...... CS

STUDENT....  414-44-6545
First Name. Karl
Last Name.. Offenbach
Major...... CS

STUDENT....  424-32-5656
First Name. Sally
Last Name.. Martin
Major...... PY

STUDENT....  521-81-4564
First Name. Harry
Last Name.. Smith
Major...... CH

STUDENT....  978-76-6676
First Name. Gerhardt
Last Name.. Muller
Major...... FA


6 records listed.
:

Listing 9 shows a similar query using UniVerse SQL.

:SELECT ID, FNAME, LNAME, SEMESTER, COURSE_NBR FROM XMLDATA XML_DATA "STUDENT.map";


STUDENT....  221-34-5665
First Name. Susan
Last Name.. Miller
Term Crs #.........................
FA93 EG110
     MA220
     PY100
SP94 EG140
     EG240
     MA221

STUDENT....  291-22-2021
First Name. jojo
Last Name.. Smith
Term Crs #.........................
SP94 FA100

STUDENT....  414-44-6545
First Name. Karl
Last Name.. Offenbach
Term Crs #.........................
FA93 CS104
     MA101
     FA100
SP94 CS105
     MA102
     PY100

STUDENT....  424-32-5656
First Name. Sally
Last Name.. Martin
Term Crs #.........................
SP94 PY100
     PE100

STUDENT....  521-81-4564
First Name. Harry
Last Name.. Smith
Term Crs #.........................
FA93 CS130
     CS100
     PY100
Press any key to continue...



SP94 CS131
     CS101
     PE220

STUDENT....  978-76-6676
First Name. Gerhardt
Last Name.. Muller
Term Crs #.........................
FA93 FA120
     FA230
     HY101
SP94 FA121
     FA231
     HY102


6 records listed.
:

RELEASE.XML

Use RELEASE.XML to free the memory allocated by PREPARE.XML. The syntax is as follows:
RELEASE.XML xml_data
where xml_data is the name given to the work space from PREPARE.XML

To complete the examples used above in PREPARE.XML and LIST XMLDATA, release the XML structure as follows:
:RELEASE.XML XML_DATA
RELEASE.XML successful.
:

Extraction files

In order to receive XML documents into U2, a special XML-based file structure was created called the U2 Extraction file. This file contains the rules required for mapping the U2 file structure to an XML document structure.

Below is some text taken directly from the UniVerse V10.1 BASIC Extensions Guide (see Resources), beginning on page 6-2:

Extraction files must be manually created. XMAP and the U2 XML/DB Tool were created to simplify the process and provide more robust functionality. Thus, extension files are less frequently used by developers in favor of the newer technology.

Developers wishing to explore extraction files should refer to the UniVerse V10.1 BASIC Extensions guide, the UniData V6.1 UniBasic Extensions guide, or the UniData V7.1 UniBasic Extensions guide (see Resources) for more information.

XML in U2 BASIC

Both IBM UniData and IBM UniVerse have extensive APIs for creating and shredding XML documents. In this section, the XDOM, XMAP, and SOAP client functions are reviewed, with sample code to demonstrate how to use them.

XDOM API

XDOM is short for XML Document Object Model, and is the name of a set of functions you can use to create, copy, traverse, and otherwise manipulate an XML document from within U2 BASIC. This is a very powerful API. Although there is some coding involved when creating or importing XML documents over loading an existing document, program execution is often faster than coding plain text, due to the improved efficiency of the XDOM memory management. Complete information on the use of each XDOM function may be found in the UniVerse V10.1 BASIC Extensions guide, the UniData V6.1 UniBasic Extensions guide, or the UniData V7.1 UniBasic Extensions guide (see Resources) for more information.

The XDOM API allows you to:

• Traverse the DOM tree in any direction.
• Search the tree for starting elements using the XPath language.
• Cut and paste nodes, and create sub-trees.
• Create new DOM trees.
• Clone the DOM in any granularity: single element, sub-tree, or the entire DOM.
• Append to any part of the DOM.
• Copy the DOM to a variable or to a file.

The following list represents the supported DOM node types:

• Document
• DocumentFragment
• DocumentType
• EntityReference
• Element
• Attribute
• ProcessingInstruction
• Comment
• Text
• CDATASection
• Entity
• Notation

The functions and descriptions are located in the BASIC Extension guides listed above, and as such this article focuses more on examples of how one might use them.

XMAP API

XMAP, sometimes referred to as U2XMAP, is a set of functions that allow XML documents to relate to U2 data files. It allows XML to be shredded and input into U2, as well as allowing U2 data structures to be mapped to elements within an XML document. UniData V6.1 and UniVerse V10.1 and higher versions support unlimited levels of nesting within an XML document, going beyond their native three-dimensional data model. Mapping files, combined with the XMAP API, let you logically relate one or more U2 files to a single XML document.

The U2 XMAP API:

• Provides the ability to convert XML data into U2 records (and vice versa).
• Uses U2XMAP internal structure to store mapping information.
• Integrates with U2 XDOM API to provide simple yet powerful tools for working with XML.

UniData and UniVerse supports three modes for mapping data to XML. They are attribute-centric, element-centric, and mixed.

For specific information regarding this API, consult the UniVerse V10.1 BASIC Extensions guide, the UniData V6.1 UniBasic Extensions guide, or the UniData V7.1 UniBasic Extensions guide (see Resources).

XMLExecute function

With the release of UniVerse V10.1 and subsequently UniData V6.1, a new function was added to U2 BASIC that permits the execution of XML-based queries similar to those that can be executed from ECL or TCL. XMLExecute allows a program to send an XML query to a dynamic array. It takes 4 arguments, which are:

• Cmd -- the query to execute (including options, such as WITHSCHEMA, WITHDTD, and so on).
• Options -- field mark (@FM) delimited list of some global element settings from U2XMAP. (If element values are to change, then a value mark will separate the element name from the new value. For example: root="mystudent")
• Return_var -- the XML document created by Cmd is stored in this variable.
• XMLSchema -- this contains the schema generated if WITHSCHEMA is part of Cmd.

       CMD = "LIST STUDENT LNAME SEMESTER COURSE_NBR '414446545'"
       OPTIONS = 'WITHDTD':@FM:'XMLMAPPING':@VM:'STUDENT.MAP'
       XMLSCHEMA = ''
       MYXML = ''
       STATUS = XMLExecute(CMD,OPTIONS,MYXML,XMLSCHEMA)
       PRINT XMLSCHEMA
       PRINT MYXML
       PRINT "XMLExecute finished"
    END

:RUN BP XMLEXECUTE.EXAMPLE
xmlns:U2xml CDATA #IMPLIED
xmlns:IBM CDATA #IMPLIED
xmlns:ibm CDATA #IMPLIED
<!-- DTD for account: C:\IBM\ud61\demo
    command: LIST STUDENT LNAME SEMESTER COURSE_NBR '414446545' TOXML 
WITHDTD XMLMAPPING 'STUDENT.MAP'
-->
<!ELEMENT SCHOOL (STUDENT_rec*)>
<!ATTLIST SCHOOL
>
<!ELEMENT STUDENT_rec ( CGA-MV* )>
<!ATTLIST STUDENT_rec
        _ID CDATA #REQUIRED
        LNAME CDATA #IMPLIED
>
<!ELEMENT CGA-MV ( SEMESTER* , CGA-MS* )>
<!ELEMENT SEMESTER  (#PCDATA) >
<!ELEMENT CGA-MS ( COURSE_NBR* )>
<!ELEMENT COURSE_NBR  (#PCDATA) >

<SCHOOL
xmlns:U2xml="http://www.ibm.com/U2-xml"
xmlns:IBM="http://www.IBM.com"
xmlns:ibm="http://www.ibm.com"
>
<STUDENT_rec _ID = "414446545" LNAME = "Offenbach">
  <CGA-MV>
    <SEMESTER>FA93</SEMESTER>
    <CGA-MS>
      <COURSE_NBR>CS104</COURSE_NBR>
    </CGA-MS>
    <CGA-MS>
      <COURSE_NBR>MA101</COURSE_NBR>
    </CGA-MS>
    <CGA-MS>
      <COURSE_NBR>FA100</COURSE_NBR>
    </CGA-MS>
  </CGA-MV>
  <CGA-MV>
    <SEMESTER>SP94</SEMESTER>
    <CGA-MS>
      <COURSE_NBR>CS105</COURSE_NBR>
    </CGA-MS>
    <CGA-MS>
      <COURSE_NBR>MA102</COURSE_NBR>
    </CGA-MS>
    <CGA-MS>
      <COURSE_NBR>PY100</COURSE_NBR>
    </CGA-MS>
  </CGA-MV>
</STUDENT_rec>
</SCHOOL>

XMLExecute finished
:

U2 XML/DB tool

As of the writing of this article, there have been two releases of the IBM U2 XML/DB tool. The first release was made available with both the UniData V6.1 and UniVerse V10.1 Client media. The next generation of the XML/DB tool was released with UniData V7.1, with most features backward-compatible with UniData V6.1 and UniVerse V10.1.

Both the original and later versions of the XML/DB tool leverage UniObjects for Java (UOJ) to connect to either UniData or UniVerse. They allow developers to create mapping files using information from U2 files, DTDs, and XML schemas. Using a graphical point-and-click environment, developers can create a map file within minutes, potentially saving a great deal of time. In this article, the focus will be on the UniData V7.1 version.

At the time of this writing, more features of this version of the tool were available for UniData V7.1 than for UniVerse V10.1 or UniData V6.1. This is due to the release timing of these U2 versions. UniData V7.1 is the newer release.

Some of the advanced features of this new tool (version 1.2) are:

• Eclipse framework upgrade
• Dictionary editor (works with UniData V7.1 and later only)
• Generate new U2 files and XMAP files based on DTDs (works with UniData V7.1 and later only)
• Generate new XML documents based on XMAP and U2 files (works with UniData V7.1 and later only)
• Import XML data into one or more U2 files (works with UniData V7.1 and later only)
• Editor for XML documents (works with UniData V7.1 and later only)
• Filters for accounts and files
• Delete map files, DTDs, and schemas
• IBM branding

For more information about how to use this tool, see the UniVerse V10.1 BASIC Extensions guide, the UniVerse V10.1 BASIC Extensions guide, the UniData V6.1 UniBasic Extensions guide, or the UniData V7.1 UniBasic Extensions guide (see Resources).

Additional XML BASIC Functions

UniVerse V10.0 introduced the first round of XML functions in U2 BASIC. They exist in all subsequent U2 releases. These functions are:

• PrepareXML -- Allocates memory for the XML document, opens the document, determines the file structure of the document, and returns the file structure (a handle is assigned). A DTD is required.
• OpenXMLData -- Opens the document.
• ReadXMLData -- Extracts the data from the document and places into a dynamic array.
• CloseXMLData -- Closes the document.
• ReleaseXML -- Releases the memory allocated for the XML document.
• XMLError -- Retrieves the error text from last XML function used.

For simple applications, these functions may be useful for developers. The XDOM and XMAP APIs are more robust, therefore are more often used by developers today. For more information about these BASIC functions, see the UniVerse V10.1 BASIC Extensions guide, the UniData V6.1 UniBasic Extensions guide, or the UniData V7.1 UniBasic Extensions guide (see Resources).

Back to top

Programming examples

This section contains some code samples. Map files, XML documents, and some dictionary listings are provided for reference. These programs will generally work for UniVerse V10.1, UniData V6.1/7.1 and later releases.

U2 XMAP example (UniData)

<?xml version="1.0" ?>
 <ROOT>
 <SCHOOL SCHOOLID="CO001" NAME="Fairview" SCHOOL_DISTRICT="VBSD">
    <CLASS CLASSOF="2004">
       <STUDENT ID="521814564" NAME="Harry Smith" DOB="1985-02-08">
          <TERM SEMESTER="FA02">
             <COURSE NAME="MA130" GRADE="A" />
             <COURSE NAME="CH100" GRADE="B" />
             <COURSE NAME="PY100" GRADE="B" />
          </TERM>
          <TERM SEMESTER="SP03">
             <COURSE NAME="MA131" GRADE="B" />
             <COURSE NAME="CH101" GRADE="B" />
             <COURSE NAME="PE220" GRADE="A" />
          </TERM>
       </STUDENT>
    </CLASS>
 </SCHOOL>
 <SCHOOL SCHOOLID="CO002" NAME="Golden" SCHOOL_DISTRICT="ACSD">
    <CLASS CLASSOF="2004">
       <STUDENT ID="291222021" NAME="Jojo Smith" DOB="1985-08-06">
          <TERM SEMESTER="SP03">
             <COURSE NAME="FR101" GRADE="B" />
          </TERM>
       </STUDENT>
    </CLASS>
    <CLASS CLASSOF="2005">
       <STUDENT ID="424325656" NAME="Sally Martin" DOB="1985=12-01">
          <TERM SEMESTER="FA02">
             <COURSE NAME="PY100" GRADE="C" />
             <COURSE NAME="PE100" GRADE="C" />
          </TERM>
       </STUDENT>
    </CLASS>
 </SCHOOL>
 </ROOT>

 <?xml version="1.0" ?>
 <!-- DOCTYPE U2XMAP SYSTEM "U2XMAP.DTD"  -->
 <U2XMAP Version="1.0" Name="XMAP1">
 <!-- Table/Class map  -->
 <TABLECLASSMAP MapName="M2" StartNode="CLASS/STUDENT" TableName="STUDENT">
    <ColumnMap Node="@ID" Column="@ID" />
    <ColumnMap Node="@NAME" Column="NAME" />
    <ColumnMap Node="@DOB" Column="DOB" />
    <ColumnMap Node="TERM, @SEMESTER" Column="SEMESTER" />
    <ColumnMap Node="TERM, COURSE, @NAME" Column="COURSE_NBR" />
    <ColumnMap Node="TERM, COURSE, @GRADE" Column="COURSE_GRD" />
 </TABLECLASSMAP>
 </U2XMAP>

 $INCLUDE UNIVERSE.INCLUDE XML.H

 * Parse XML document and build DOM tree in memory

       Status = XDOMOpen('STUDENT.XML',XML.FROM.FILE,domH)
       If Status <> XML.SUCCESS Then
          * We hit an error
          Print 'Error with XDOMOpen().'
          Gosub PrintError
          Stop
       End

 * Position at a specific node

       Status = XDOMLocate(domH,'ROOT/SCHOOL[2]','',domHandle)
       If Status <> XML.SUCCESS Then
          * We hit an error
          Print 'Error from XDOMLocate().'
          Gosub PrintError
          Stop
       End

 * Open XMAP dataset for reading

         Status = XMAPOpen(domHandle, XML.FROM.DOM, 'STUDENT.MAP2', XML.FROM.FILE
       If Status <> XML.SUCCESS Then
          * We hit an error
          Print 'Error with XMAPOpen().'
          Gosub PrintError
          Stop
       End

       Open 'STUDENT' to F1 Else Stop 'Error opening file STUDENT.'

 * Read records from XMAP dataset, write to STUDENT file

       Record = ''
       Loop
          Status = XMAPReadNext(Xfile, 'STUDENT', Record)
       Until Status = XML.EOF Do
          If Status <> XML.SUCCESS Then
             Print 'XMAPReadNext() error.'
             Gosub PrintError
             Stop
          End
          ID = Record<1>
          Rec = FIELD(Record, @FM, 2, 999)
          Write Rec to F1, ID Else Stop 'Write to file STUDENT failed.'
          Print 'ID ':ID:' written to the STUDENT file from STUDENT.XML.'
       Repeat
       If Status = XML.EOF Then
          Print 'EOF reached in document.'
       End

 * Close all handles before exiting

       Status = XMAPClose(Xfile)
       If Status <> XML.SUCCESS Then
          * We hit an error
          Gosub PrintError
          Stop
       End
       Status = XDOMClose(domHandle)
       If Status <> XML.SUCCESS Then
          * We hit an error
          Print 'Error with XDOMClose().'
          Gosub PrintError
          Stop
       End
       Status = XDOMClose(domH)
       If Status <> XML.SUCCESS Then
          * We hit an error
          Print 'Error with XDOMClose().'
          Gosub PrintError
          Stop
       End
       Close F1

       Stop

 PrintError:

 * Using XMLGetError() we can find out the very last error code and
 * its corresponding message. This allows us to display a descriptive
 * reason for the problem.

       Tmp = XMLGetError(Code,Msg)
       Print
       Print 'The error description and code:'
       Print '------------------------------------------'
       Print 'Error desc:' ; Print Msg
       Print 'Error code: ':Code
       Print '------------------------------------------'
       Return

    End

:LIST XSTUDENT ALL

LIST XSTUDENT ALL 12:17:46 Dec 15 2005 1

No records listed.

:RUN BP TEST.XMAP
ID 291222021 written to the XSTUDENT file from XSTUDENT.XML.
ID 424325656 written to the XSTUDENT file from XSTUDENT.XML.
EOF reached in document.
:

:LIST XSTUDENT ALL

LIST XSTUDENT ALL 12:18:03 Dec 15 2005 1
XSTUDENT    291222021
Name        Jojo Smith
Dummy
Semester   Courser NO GRADE
SP03       FR101      B
Class ID
Order
School ID
DOB         06 Aug 85

XSTUDENT    424325656
Name        Sally Martin
Dummy
Semester   Courser NO GRADE
FA02       PY100      C
           PE100      C
Class ID
Order
School ID
DOB         01 Dec 85

2 records listed
:

The TEST.XMAP program above uses several XDOM functions, including one that uses XPath for locating a specific node, and several XMAP functions. They combine to open the example XML document, locate a specific node, open an XMAP file, loop through the document starting at the found node, remove the data from the document, and write it to a U2 file called XSTUDENT.

U2 XDOM example (UniVerse)

 <ADDRBOOK cmt="my address book">
 <ENTRY ID="id1">
 <NAME>Name One</NAME>
 <ADDRESS>101 Some Way</ADDRESS>
 <PHONENUM DESC="Work">303-555-1111</PHONENUM>
 <PHONENUM DESC="Fax">303-555-2222</PHONENUM>
 <PHONENUM DESC="Pager">303-555-3333</PHONENUM>
 <EMAIL>name.one@some.com</EMAIL>
 </ENTRY>
 <ENTRY ID="id2">
 <NAME>Name Two</NAME>
 <ADDRESS>202 Some Way</ADDRESS>
 <PHONENUM DESC="Work">303-555-1111</PHONENUM>
 <PHONENUM DESC="Fax">303-555-2222</PHONENUM>
 <PHONENUM DESC="Home">303-555-3333</PHONENUM>
 <EMAIL>name.two@some.com</EMAIL>
 </ENTRY>
 </ADDRBOOK>

 $INCLUDE UNIVERSE.INCLUDE XML.H
 * Program to demonstrate XDOM

 * Open our sample document as a DOM tree
 doc = 'BP/sample.xml'
 Status = XDOMOpen(doc, 0, domHandle)
 Gosub checkStatus
 Print 'DOM open status = ':Status

 * Locate the ENTRY node, passing a string to XPath
 Status = XDOMLocate(domHandle, '/ADDRBOOK/ENTRY[@ID="id1"]', '', nodeHandle)
 Print 'DOM locate status = ':Status
 Gosub checkStatus

 * Create a new element node at the current location in the tree
 nodeName = 'PHONENUM'
 Status=XDOMCreateNode(nodeHandle, nodeName, '', XDOM.ELEMENT.NODE, newElem)
 Print 'New element node create status = ':Status
 Gosub checkStatus

 * Create the attribute for the new element node
 Attr = 'DESC'
 attrValue = 'Cell'
 Status = XDOMCreateNode(nodeHandle, Attr, attrValue, XDOM.ATTR.NODE, newAttr)
 Print 'New attribute node create status = ':Status
 Gosub checkStatus

 * Create a new text node at the current location in the tree
 dataString = '444-555-4444'
 Status = XDOMCreateNode(nodeHandle, '', dataString, XDOM.TEXT.NODE, newText)
 Print 'Create text node status = ':Status
 Gosub checkStatus

 * Add the new node as the next sibling in the DOM tree using XPath to find
 * the new location of where we want to place the node
 Status = XDOMAppend(nodeHandle, 'PHONENUM[2]', '', newElem, XDOM.NODUP)
 Print 'DOM append completed node = ':Status
 Gosub checkStatus

 * Add the text node as the next child after the attribute note using newElem
 * as the handle to which the text will added
 Status = XDOMAddChild(newElem, '', '', newText, XDOM.NODUP)
 Print 'DOM add child status = ':Status
 Gosub checkStatus

 * Add the attribute to our node by using XDOMAppend, because attributes are
 * added with this function. We just append to the element node we are
 * building.
 Status = XDOMAppend(newElem, '', '', newAttr, XDOM.NODUP)
 Print 'DOM append completed attr node = ':Status
 Gosub checkStatus

 * Write the modified document to the &XML& file
 xmlString = 'OUT.xml'
 Status = XDOMWrite(domHandle, xmlString, XML.TO.FILE)
 Print 'DOM write to file = ':Status
 Print 'DOM ':xmlString:' written to &XML& file'
 Gosub checkStatus

 Stop

 checkStatus:
 * Test for an error and display if found
 If Status <> XML.SUCCESS Then
    Status = XMLGetError(errCode,errMsg)
    Print errCode:' ':errMsg
    Stop 'Stop the program'
 End
 Return
End

DOM open status = 0
DOM locate status = 0
New element node create status = 0
New attribute node create status = 0
Create text node status = 0
DOM append completed node = 0
DOM add child status = 0
DOM append completed attr node = 0
DOM write to file = 0
DOM OUT.xml written to &XML& file

 <ADDRBOOK cmt="my address book">
 <ENTRY ID="id1">
 <NAME>Name One</NAME>
 <ADDRESS>101 Some Way</ADDRESS>
 <PHONENUM DESC="Work">303-555-1111</PHONENUM>
 <PHONENUM DESC="Fax">303-555-2222</PHONENUM>
 <PHONENUM DESC="Cell">444-555-4444</PHONENUM>
 <PHONENUM DESC="Pager">303-555-3333</PHONENUM>
 <EMAIL>name.one@some.com</EMAIL>
 </ENTRY>
 <ENTRY ID="id2">
 <NAME>Name Two</NAME>
 <ADDRESS>202 Some Way</ADDRESS>
 <PHONENUM DESC="Work">303-555-1111</PHONENUM>
 <PHONENUM DESC="Fax">303-555-2222</PHONENUM>
 <PHONENUM DESC="Home">303-555-3333</PHONENUM>
 <EMAIL>name.two@some.com</EMAIL>
 </ENTRY>
 </ADDRBOOK>

In the example above, the TEST.XDOM program inserted a node at a specific location found by an XPath string. It begins by opening an XML document located in the local BP file, and creates a DOM structure from the document. The new DOM is given a handle. Using XPath, it locates the first node and assigns the found node a handle of its own. While the internal pointer is at the found node, the program then creates a new element called PHONENUM, followed by a new attribute called DESC with a value of Cell, then a new text node 444-555-4444. Next, the element is appended after the second occurrence of PHONENUM within the first ENTRY ID located earlier in the program. This is accomplished using the XDOMAppend function using an XPath string. The text node is added as a child to the new PHONENUM element node and then the attribute is appended to the node. The newly modified DOM structure is then written to a file called OUT.xml.

It is worthwhile to note that the XDOM API does not "beautify" XML documents, but the output is in accordance with XML specifications.

Back to top

Summary

XML is a continuously developing technology that is ideal for organizations to exchange information with one another. IBM U2 provides the openness of XML so that developers can create powerful and efficient applications which can easily exchange information. U2 XML empowers businesses to share information according to individual business rules, yet remain open and application agnostic.

Combined with technologies as UniObjects for Java, UniObjects for .NET, and the U2 BASIC callHTTP and SOAP client APIs, U2 XML is the ideal choice for the format of data exchange, as it easily maps to U2's nested data structures.

Resources

Learn

• U2 SOAP API client: Explore some of the U2 Extended BASIC APIs, namely SOAP, DOM, XPath, XMLDB, XMAP, and the XML DB mapping tool.
  
  
• Learn about the U2 data model by reading the article IBM U2: The Big Picture (developerWorks, August 2005).
  
  
• Find more details about the Nested Relational database model in this Nested Relational Databases White Paper.
  
  
• For UniData V6.1, the UniBasic Extensions, Version 6.1 manual describes the following extensions to UniBasic: UniBasic Socket API, Using CallHTTP, Using SSL with CallHTTP and the Socket Interface, and Using WebSphere MQ with UniData.
  
  
• For UniData V7.1, the UniBasic Extensions, Version 7.1 manual describes the following extensions to UniBasic: UniBasic Socket API, Using CallHTTP, Using SSL with CallHTTP and the Socket Interface, and Using WebSphere MQ with UniData.
  
  
• For UniVerse, the BASIC Extensions V 10.1 (G251-1908-00) manual describes the following extensions to UniVerse BASIC: UniVerse BASIC Socket API, Using CallHTTP, Using SSL with CallHTTP and the Socket Interface, and Using WebSphere MQ with UniVerse.
  
  
• Learn more about IBM WebSphere® and related products.
  
  
• Read "Connect WebSphere Voice Server SDK to U2 using RedBack JavaBeans" to learn how to access U2 data via IBM RedBack® from phone and other devices.
  
  
• "Zeroing in on UniData problems" (developerWorks, September 2004) assists with the problem determination process for someone administering or supporting UniData databases.
  
  
• Learn more about U2 by visiting IBM U2 Website or by contacting U2AskUs.
  
  

Discuss

• Visit the International UniVerse & UniData User Group.
  
  

About the author

LeRoy has been in the MultiValue computing business for 20 years, with much of that time in development roles. He joined the U2 Support team in 2000, advancing to the role of Advanced Technical Support for UniVerse in 2001. In August 2004, Roy joined the U2 Product Management team as Product Manager for IBM UniData and IBM UniVerse with responsibilities of helping to guide the future of these databases.

Comments

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Introduction
• An overview of XML
• U2 XML features
• XML in U2 BASIC
• Programming examples
• Summary
• Resources
• About the author
• Comments

Next steps from IBM

With the no-charge DB2 Express-C, pureXML, Zend Core PHP, and Java you can develope advanced Web 2.0 applications quickly that matter.

---

• Try: Download a free trial version of DB2 Express-C, which includes Zend Core PHP and pureXML technology that is ideally suited for managing both XML and relational data.
• Article: See how DB2 pureXML schema capabilities can evolve schemas without making any changes to your applications.
• Tutorial: This tutorial shows you how to use DB2's pureXML to expose the native XML capabilities of DB2.
• Buy: DB2 9 for Linux, UNIX and Windows

My developerWorks community

Dig deeper into Information Management on developerWorks

DB2 pureScale

See how true transparent scaling architecture can be achieved on distributed platforms

Special offers