Skip to main content

By clicking Submit, you agree to the developerWorks terms of use.

The first time you sign into developerWorks, a profile is created for you. Select information in your developerWorks profile is displayed to the public, but you may edit the information at any time. Your first name, last name (unless you choose to hide them), and display name will accompany the content that you post.

All information submitted is secure.

  • Close [x]

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerworks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

By clicking Submit, you agree to the developerWorks terms of use.

All information submitted is secure.

  • Close [x]

Parsing comma-separated values

Doug Tidwell (dtidwell@us.ibm.com), developerWorks staff
Doug Tidwell is a Senior Programmer at IBM. He has been writing code since the early days of the Reagan Administration. Although he swears he has been writing Java code since the late 1950s, the fact that he was born in the mid-1960s has led many to doubt this claim. His job as a Cyber Evangelist is to help customers evaluate and implement new technology. Thanks to a special arrangement with his employer, his salary is now paid entirely in chocolate truffles. He can be reached at dtidwell@us.ibm.com.

Summary:  Ask the expert: My assignment on an XML pilot project is a transformation from data formatted in comma-delimited files (CDF) that the customer wants to transform into XML data. Seems logical enough, but I can't find any reference to a CDF-to-XML translator. What would you suggest? --Tom Watson, consultant
Doug Tidwell responds: Tom, I think I have a solution. It all starts with getting the data.

Date:  24 Apr 2000
Level:  Introductory
Also available in:   Japanese

Activity:  3508 views
Comments:  

Getting the data

To make a comma-separated value (CSV) file to work with, I used DB2's EXPORT command for the sample database of employee records that ships with the product. The command I used was:

EXPORT to test.csv of DEL select * from employees

That generated the file test.csv , which you can download.


Parsing the comma-separated value (CSV) file

After I had the data file, I started looking around for some code to parse comma-separated values. I didn't find exactly what I wanted, but I did stumble across the Java class StreamTokenizer. This class lets you design a rudimentary parser fairly easily. You select the delimiter between tokens, and it parses the file, converts data to strings and integers, and does some other nice things. You can view the code I wrote or download) it.

To use this file, I typed:

java csvParser test.csv output.xml

That command opens and parses the CSV file and then converts it to XML. The XML for one employee looks like Listing 1. (You can also download output.xml.)


Listing 1. The XML output for one employee record from the CSV data sample
<?xml version="1.0"?>
<document>
  <row>
    <column1>000010</column1>
    <column2>CHRISTINE</column2>
    <column3>I</column3>
    <column4>HAAS</column4>
    <column5>A00</column5>
    <column6>3978</column6>
    <column7>19650101</column7>
    <column8>PRES</column8>
    <column9>18</column9>
    <column10>F</column10>
    <column11>19330824</column11>
    <column12>52750</column12>
    <column13>1000</column13>
    <column14>4220</column14>
  </row>
  <row>
    <column1>000020</column1>
  ...
</document>

When I started working on this, I thought the first line of the CSV file would contain the column names from DB2. I was going to use those names as the XML tag names. I didn't immediately find a way to get DB2 to export data in this format, so I just made up the column names, using the imaginative naming scheme you see in Listing 1.


Converting the generated XML

Now that I had this XML-tagged data, I needed to write an XSLT style sheet that would convert the XML above into a more useful (and understandable) tag set. You can view the style sheet listing in a separate window, and you can also download it and view it locally.

To convert the generated XML document using the rules in the style sheet and then write the output to the file called employees.xml, I typed:

java org.apache.xalan.xslt.Process -in output.xml -xsl csv-stylesheet.xsl 
  -out employees.xml

If you want to run the transform code, you'll need the Xerces XML parser and version 1.0.0 of the Xalan style sheet processor (see Resources), both of which are free and available at xml.apache.org. The resulting XML looks like Listing 2. (You can also download employees.xml and employees.dtd.)


Listing 2. XML output of the document converted using csv-stylesheet.xsl
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE employees SYSTEM "employees.dtd">
<employees>
  <employee sex="F">
    <serial_number>000010</serial_number>
    <name>
      <first_name>CHRISTINE</first_name>
      <middle_initial>I</middle_initial>
      <last_name>HAAS</last_name>
    </name>
    <department>A00</department>
    <phone>3978</phone>
    <date_of_hire year="1965" month="01" day="01"/>
    <job_title>PRES</job_title>
    <years_of_education>18</years_of_education>
    <date_of_birth year="1933" month="08" day="24"/>
    <salary>52750</salary>
    <bonus>1000</bonus>
    <commission>4220</commission>
  </employee>
  <employee sex="M">
  ...
  </employee>
</employees>




Figure 1. Excerpt of the HTML table converted from the CSV data via the style sheet employee-table.xsl
EmployeeSexSerial NumberDepartmentJob TitleDate of HireYears of Education
PhoneDate of BirthSalaryBonusCommissionTotal Compensation
ADAMSON, BRUCEM000150D11DESIGNER02/12/197216
451005/17/1947$25,280.00$500.00$2,022.00$27,802.00
BROWN, DAVIDM000200D11DESIGNER03/03/196616
450105/29/1941$27,740.00$600.00$2,217.00$30,557.00
GEYER, JOHN B.M000050E01MANAGER08/17/194916
678909/15/1925$40,175.00$800.00$3,214.00$44,189.00
GOUNOT, JASON R.M000340E21FIELDREP05/05/194716
569805/17/1926$23,840.00$500.00$1,907.00$26,247.00
HAAS, CHRISTINE I.F000010A00PRES01/01/196518
397808/24/1933$52,750.00$1,000.00$4,220.00$57,970.00

You can view the entire table in a separate window, and also view the HTML source.

I hope this helps. I've certainly learned some things along the way, and I hope you have too.


Resources

  • If you want to transform the XML documents, you'll need the Xerces XML parser and the Xalan style sheet processor, both of which are free and available at xml.apache.org.

About the author

Doug Tidwell

Doug Tidwell is a Senior Programmer at IBM. He has been writing code since the early days of the Reagan Administration. Although he swears he has been writing Java code since the late 1950s, the fact that he was born in the mid-1960s has led many to doubt this claim. His job as a Cyber Evangelist is to help customers evaluate and implement new technology. Thanks to a special arrangement with his employer, his salary is now paid entirely in chocolate truffles. He can be reached at dtidwell@us.ibm.com.

Report abuse help

Report abuse

Thank you. This entry has been flagged for moderator attention.


Report abuse help

Report abuse

Report abuse submission failed. Please try again later.


developerWorks: Sign in


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Select information in your developerWorks profile is displayed to the public, but you may edit the information at any time. Your first name, last name (unless you choose to hide them), and display name will accompany the content that you post.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


Rate this article

Comments

Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=XML
ArticleID=11950
ArticleTitle=Parsing comma-separated values
publish-date=04242000
author1-email=dtidwell@us.ibm.com
author1-email-cc=

Tags

Help
Use the search field to find all types of content in My developerWorks with that tag.

Use the slider bar to see more or fewer tags.

For articles in technology zones (such as Java technology, Linux, Open source, XML), Popular tags shows the top tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), Popular tags shows the top tags for just that product zone.

For articles in technology zones (such as Java technology, Linux, Open source, XML), My tags shows your tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), My tags shows your tags for just that product zone.

Use the search field to find all types of content in My developerWorks with that tag. Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere). My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Try IBM PureSystems. No charge.

Special offers