Introduction to persistence

All programmers have had to solve the problem of data persistence. Saving documents to a file and transactions to a database, for example, are tasks that require a fair amount of work for even the simplest of applications. Data persistence can be defined as a contract, negotiated between a program and a storage device, that specifies how data are to be stored and retrieved.

CPAN, defined:
"CPAN is the Comprehensive Perl Archive Network. Comprehensive: the aim is to contain all the Perl material you will need. Archive: 760 megabytes as of September 1999. Network: CPAN is mirrored at more than one hundred sites around the world."

This article concentrates on making object attributes persistent in cases where both the writer and the reader know the attributes of the object beforehand. This article does not explore the topics of discovering object attributes dynamically and storing more than object attributes (behavior, for example), as these topics merit more than a few paragraphs for a full consideration.

Familiarity with object-oriented programming, databases, Perl, and Perl inheritance will be helpful in reading this article.

Making object attributes persistent is only a part of the software development process. First, the requirements of persistence must be determined. What needs to be saved? What needs to be restored? Will there be a need for humans to edit the saved data before it is restored? Questions like these will determine how the software achieves data persistence. Unfortunately, there is no single questionnaire that determines a project's exact persistence needs. The software architect must, through intelligent design, define and satisfy the project's requirements. It is important that the resulting solutions be solutions to the problem and not just industry buzzwords. XML, for instance, will not solve data persistence problems any more than bricks will build a house by themselves.

Back to top

A persistent data example

The Perl file simple shows how simple persistence can be implemented for a predefined set of keywords. This approach has many problems: the file format is arbitrary, only one set of keywords is saved, objects are not even on the horizon, data values are limited to one line of text, and so on.

Back to top

Simple uses of the Persistent classes: Persistent::File

The Persistent classes aim to make data persistence easy. To this end, you must read the documentation. This is a small price to pay for a convenient, pre-built class that will handle data persistence for you. So put a few minutes or a few hours into this task, and you'll save yourself time later. Look at the home page of the Persistent classes (see Resources) for documentation and examples.

Our first example, first, will be based on the simple example given in the documentation for the Persistent::File module, and will illuminate the steps taken to achieve persistence.

my %problems = (
		'Homework 1, Problem 1' => [0,5,2],
		'Homework 1, Problem 2' => [1,8,5],
		'Homework 1, Problem 3' => [2,2,4],
		'Homework 1, Problem 4' => [3,0,3],
		'Homework 1, Problem 5' => [4,1,2],
	       );

First, we define the data that we will store in the datastore. In the Persistent classes, the datastore is an abstract storage device, which can be a file or a database, for example. This example is self-contained, but the data could come from any source.

use English;
eval 
{
};
print "An error occurred: $EVAL_ERROR\n" if $EVAL_ERROR;

We wrap everything we do in an eval statement, since we don't want our program to die when an error occurs during execution. The English module lets us use $EVAL_ERROR instead of $@ for error messages.

# create a persistent object from a file
my $equation = new Persistent::File('variables.txt');

Here, we create a new Persistent::File object whose contents will be stored in the file variables.txt.

$equation->add_attribute('name', 'ID', 'VarChar', undef, 80);
$equation->add_attribute('x', 'Persistent', 'Number', 0, 10);
$equation->add_attribute('y', 'Persistent', 'Number', 0, 10);
$equation->add_attribute('z', 'Persistent', 'Number', 0, 10);
$equation->add_attribute('answer', 'Transient', 'Number', undef, 10);

We create five attributes: a unique identifier, three persistent data members, and a non-persistent (transient) data member. The Persistent classes will automatically create the necessary functions ($equation->answer(), for example) to access the data members.

$equation->restore_all();
$equation->delete while $equation->restore_next();

The restore_all method retrieves the contents of the entire data store. The restore_where and restore methods can be used to select objects matching some criteria and a single object, respectively. The delete method removes the current equation, and the restore_next method moves to the next restored object.

# now store the problems in the datastore
foreach my $key (keys %problems)
{
  $equation->clear;
  $equation->name($key);
  $equation->x($problems{$key}->[0]);
  $equation->y($problems{$key}->[1]);
  $equation->z($problems{$key}->[2]);
  $equation->save;
}

We clear old attributes, set new attributes, and then save the object in the datastore. Because we just cleared the datastore, we know that there will be no conflicting name keys, but normally we would check so we don't cause an exception when trying to overwrite an existing key with the save method.

# query the datastore for equations from homework 1
$equation->restore_where(qq{name =~ 'Homework 1'});
while ($equation->restore_next()) 
{
  # do something with each equation
}

Finally, we look through the datastore for objects whose name key contains the string 'Homework 1' and perform an operation on each of those objects.

This example is not that impressive in itself, but consider:

• We can now add any number of attributes to our persistent objects
• We can selectively retrieve any of the stored objects
• Any other program that knows the attributes of our objects can use the same datastore (more on this under "Inheriting from Persistent classes").

By using pre-built modules, we have just solved a hard problem in an easy way. This is fun.

Back to top

Using Persistent::DBI

Read the Persistent::DBI documentation for an up-to-date discussion of the differences between Persistent::DBI and the Persistent::Base class. Basically, Persistent::DBI allows us to connect to a database and behaves exactly like the other Persistent::Base classes once the connection is established. We could substitute the line shown in Listing 3 with the following:

my $database = 'test_database';
my $host     = 'db_host';
my $user     = 'dali';
my $password = 'MeltingClocks';
my $table    = 'persistence_test_table';

my $equation = new Persistent::MySQL(
                  "DBI:mysql:database=$database;host=$hostname",
                  $user, 
                  $password, 
                  $table);

Note that Persistent::MySQL is a subclass of Persistent::DBI. Using Persistent::DBI by itself won't work; we have to use the module particular to our database. The good news is that we don't have to change our code if we switch databases, except for the places -- such as the use statement and the new statement -- where we mention Persistent::MySQL by name. See "Inheriting from Persistent classes" for a way to isolate programs from the database choice by using inheritance.

# query the datastore for equations from homework 1
$equation->restore_where(qq{name LIKE '%Homework 1%'});

Since we are now using SQL SELECT statements, the restore_where argument changes from a Perl pattern match to the syntax of an SQL WHERE clause.

The table must exist beforehand; Persistent::MySQL won't create a table. Either write a separate program to create the tables, or add the necessary functionality in your class with inheritance.

The table must match the definition of your attributes; for example, the primary database key for the equations table must be the primary key of the Persistent class as well. (This is a case where good design must come before any implementation work.)

Listing 8 and listing 9 show the only things we have to change in first in order to use SQL. The script second contains those changes.

Back to top

Inheriting from Persistent classes

The file Equation.pm contains all the code necessary to inherit from a Persistent::DBI class. Simple, isn't it? Basically the add_attribute invocations are done in the local initialize method. More can be done here: create a table, for example, if the table doesn't exist in the database. The basic approach, however, is very simple.

Security note:
Usually, creating a table is restricted to privileged users in a database, so it would be best to discuss the table creation with your database administrator to make sure that you don't spend time writing a program that won't work because of security policies.

#! /usr/bin/perl -w

package Equation;
@ISA = qw(Persistent::MySQL);

use strict;
use Persistent::MySQL;

sub initialize
{
  my $self = shift;
  $self->SUPER::initialize(@_);
 
  # define attributes of the object (the contract)
  # this is the primary object identifier key, a 10-character name
  $self->add_attribute('name', 'ID', 'VarChar', undef, 80);
  # x, y, and z are persistent numbers with default values of 0 and length of 10
  $self->add_attribute('x', 'Persistent', 'Number', 0, 10);
  $self->add_attribute('y', 'Persistent', 'Number', 0, 10);
  $self->add_attribute('z', 'Persistent', 'Number', 0, 10);
  # this attribute will NOT be saved
  $self->add_attribute('answer', 'Transient', 'Number', undef, 10);
}
1;

Note the @ISA line; it tells Perl that the Equation class is derived from Persistent::SQL.

The file third is second without the add_attribute lines or "use Persistent::MySQL", and with Listing 11 added:

use lib '.';
use Equation;

This tells Perl to use the Equation module, equation, and to look for Equation.pm in the current directory.

The approach shown here can be extended easily to much more complex problems. The information in Resources should be sufficient to get you started on that path.

Back to top

Conclusion

The CPAN Persistent classes are powerful allies in your search for data persistence solutions. With experience, you may find that the best way to use Persistent classes is to inherit from them. Remember that good design and a clear definition of the attributes are just as important as the Persistent classes in achieving successful data persistence.

Resources

• Read Ted's other Perl articles in the "Cultured Perl" series on developerWorks.
  
  
• The Persistent classes' home page, by Dave Winters, has additional information about the Persistent classes on this page, which you can also find on the CPAN search page.
  
  
• CPAN (the Comprehensive Perl Archive Network) contains all the Perl modules you ever wanted.
  
  
• See perl.com for all things Perl.
  
  
• Programming Perl, 2nd Edition, by Larry Wall, Tom Christiansen, and Randal L. Schwartz (O'Reilly, 1996) is the best guide to Perl today, but a little outdated with 5.005 and 5.6.0 out now.
  
  
• Object Oriented Perl, by Damian Conway (Manning Publications, 2000) is an excellent guide to modules and object orientation.
  
  
• Effective Perl Programming, by Joseph Hall and Randal Schwartz (Addison Wesley, 1998) is the definitive source of Perl tips and tricks in book form, relating to the language rather than specific tasks. The Web site also has lots of useful information.
  
  
• James Hoffman's SQL tutorial is a very useful and interesting tutorial with lots of links to even more SQL information. Highly recommended.
  
  
• See the "perltoot" and DBI perldoc pages.
  
  
• Perl files and modules referenced in this article:
  • simple file
  • first file
  • second file
  • third file
  • Equation module
  
  
  

About the author

Teodor Zlatanov graduated with an M.S. in computer engineering from Boston University in 1999. He has worked as a programmer since 1992, using Perl, Java, C, and C++. His interests are in open source work on text parsing, 3-tier client-server database architectures, UNIX system administration, CORBA, and project management. He can be contacted at tzz@bu.edu.

Comments

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Introduction to persistence
• A persistent data example
• Simple uses of the Persistent classes
• Using Persistent::DBI
• Inheriting from Persistent classes
• Conclusion
• Resources
• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

My developerWorks community

Dig deeper into Linux on developerWorks

Rate a product. Write a review.

Write a review for Rational products today!

Special offers