Introduction

From the SPEC Web site, the "Standard Performance Evaluation Corporation (SPEC) is a non-profit corporation formed to establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers. SPEC develops suites of benchmarks and also reviews and publishes submitted results from our member organizations and other benchmark licensees."

IBM® WebSphere® Application Server and IBM DB2® Universal Database™ (UDB) have achieved price performance leadership among the high-end SPECjAppServer2002 results. The benchmark results and the full disclosure report are posted on the Standard Performance Evaluation Corporation (SPEC) website [1]. This article takes you through the steps required to set up a single node sandbox SPECjAppServer2002 environment on the IBM platform. New users should get a quick start with the J2EE™ performance benchmark for learning purpose. This article provides the readers with a solid foundation so that they can understand the issues well enough to move on to more advanced J2EE topologies and performance tuning concepts.

Goals

We will touch on the following major objectives:

• To provide SPECjAppServer2002 setup instructions for IBM products. SPEC.org provides a generic user guide [1] on how to set up the SPECjAppServer2002 application. Vendor-specific information is not included in the documentation, but we provide the instructions for setting up the application on WebSphere and DB2.
• To simplify configuration and improve efficiency. The benchmark kit contains a thousand files. Setting up the kit involves editing many files in roughly 50 different places. We propose the use of script automation to simplify the tasks. Sample scripts are provided for references. My experience is that the scripting approach can cut the setup time from a day to an hour.
• To provide cookbook instructions to the new users. IBM has comprehensive documentation on WebSphere and DB2 administration from setting up a test system to building a huge cluster with high availability support. One of the goals of this article is to deliver specific instructions on how to set up the benchmark in the simplest possible configuration for learning purposes. This will serve as a basis for those who are interested in moving on to more advanced J2EE topologies and performance work.

Prerequisites

Hardware

For learning purposes, any model Intel Windows® desktop machine will do. Listed below is the hardware we used in this article:

• IBM eServer™ x345
• 2 x 2.8 GHz CPUs
• 6 x SCSI disks
• 4 GB memory

Software

Listed below is the software we used in this article:

• Microsoft® Windows 2000 Server Services Pack 4
• IBM DB2 UDB v8.1
• IBM WebSphere Application Server v5.1
• SPECjAppServer2002 v1.14 benchmark kit
• SPECjAppServer2002 install scripts
• ActiveState PERL v5.8.0

Back to top

The SPECjAppServer2002 benchmark

SPECjAppServer2002 is an industry-standard benchmark designed to measure the performance of Java™ application servers. The benchmark stresses the EJB container of the application server and tests its efficiency. The emphasis of the benchmark is on testing the EJB container, and the benchmark driver will talk to the container through RMI directly. TheWweb layer is not stressed in this benchmark.

The goal of any e-business benchmark is to maximize the total number of transactions and to minimize the total cost of the system. SPECjAppServer2002 measures performance using the following two metrics:

• TOPS (Total Operations per Second) – the total number of customer orders transactions and manufacturing work orders divided by the measurement interval in seconds.
• $/TOPS – the price per performance ratio of the system including hardware, software, and support.

These two metrics allow J2EE vendors to compare systems (of different hardware and software) in the J2EE performance community.

The J2EE application simulates a heavyweight manufacturing, supply-chain management, and procurement system. Designed to emulate a supply-chain environment, the workload is partitioned into four logical domains based on different business areas: corporation, customer, manufacturing, and supplier. The corporate and customer domains are responsible for processing customer order transactions. The corporate domain manages a global list of customers, parts, and suppliers for the customer domain. The customer domain manages customer orders. The supplier and manufacturing domains work together, in order to model the activity of production lines in a manufacturing plant. The supplier domain is responsible for interactions with suppliers. The manufacturing domain models two types of production lines operated by the companies manufacturing facilities: planned lines and large order lines. While planned lines run on a schedule, processing a pre-defined number of work orders, large order lines dynamically respond to large orders generated in the customer domain.

Back to top

Install IBM DB2 UDB

DB2 UDB has a very easy-to-use installation program. For our purpose, choose the Typical installation option. To keep things simple, install the product to C:\SQLLIB. If you use a response file, make sure that you have the keys shown below.

FILE=C:\SQLLIB\
INSTALL_TYPE=TYPICAL
			

To verify the installation, open a DB2 Command Window and issue the commands shown below.

C:\GettingStartedWithSpecJ> db2level
DB21085I  Instance "DB2" uses "32" bits and DB2 code release "SQL08010" with
level identifier "01010106".
Informational tokens are "DB2 v8.1.0.36", "s021023", "", and FixPak "0".
Product is installed at "C:\SQLLIB".

C:\GettingStartedWithSpecJ> db2stop
01-08-2004 11:02:35     0   0   SQL1064N  DB2STOP processing was successful.
SQL1064N  DB2STOP processing was successful.

C:\GettingStartedWithSpecJ> db2start
01-08-2004 11:02:39     0   0   SQL1063N  DB2START processing was successful.
SQL1063N  DB2START processing was successful.

C:\GettingStartedWithSpecJ> db2sampl

C:\GettingStartedWithSpecJ> db2 list db directory

 System Database Directory

 Number of entries in the directory = 1

Database 1 entry:

 Database alias                       = SAMPLE
 Database name                        = SAMPLE
 Database drive                       = C:\DB2
 Database release level               = a.00
 Comment                              =
 Directory entry type                 = Indirect
 Catalog database partition number    = 0
			

This will make sure that the basic functionalities of starting the database manager, stopping the database manager, and creating the database are all working correctly.

Back to top

Install IBM WebSphere Application Server

WebSphere Application Server also provides a straightforward installation program for no-tears installation. During the installation, choose the Custom install option and remove the “Embedded Messaging”, “IBM HTTP Server”, and “Web Server Plugin” components. To make life easier, install the product in C:\was51\b0344.02. If you prefer to use a response file, make sure that you have the keys set as shown below:

-P wasBean.installLocation="C:\was51\b0344.02"

-P mqSeriesBean.active="false"
-P mqSeriesServerBean.active="false"
-P mqSeriesClientBean.active="false"

-P pluginBean.active="false"
-P ihsPluginBean.active="false"
			

Before going on to the next step, make sure you test the installation. To verify the installation, click on the links on the FirstSteps wizard that pops up after the installation. Below is some sample test output.

Start the Server
ADMU0116I: Tool information is being logged in file
           C:\was51\b0344.02\logs\server1\startServer.log
ADMU3100I: Reading configuration for server: server1
ADMU3200I: Server launched. Waiting for initialization status.
ADMU3000I: Server server1 open for e-business; process id is 340

Verify Installation
IVTL0095I: defaulting to host pizza43 and port 9080
IVTL0010I: Connecting to the WebSphere Application Server pizza43 on port: 9080
IVTL0015I: WebSphere Application Server pizza43 is running on port: 9080
 
 
IVTL0050I: Servlet Engine Verification Status - Passed 
 
IVTL0055I: JSP Verification Status - Passed 
 
IVTL0060I: EJB Verification Status - Passed 
 
IVTL0070I: IVT Verification Succeeded
 
IVTL0080I: Installation Verification is complete

Stop the Server
ADMU0116I: Tool information is being logged in file
           C:\was51\b0344.02\logs\server1\stopServer.log
ADMU3100I: Reading configuration for server: server1
ADMU3201I: Server stop request issued. Waiting for stop status.
ADMU4000I: Server server1 stop completed.
			

This will ensure that all the components are installed successfully.

Back to top

Obtain the SPECjAppServer code

The latest SPECjAppServer2002 code is available from the SPECjAppServer2002 result Web page [1]. For our case, we will use the code in the full disclosure archive submitted by IBM on Nov 12, 2003. Copy the two EAR files into C:/was51/b0344.02/installableapps.

02/04/2003  08:36p              43,952 SPECj2002_Emulator.ear
09/10/2003  05:33p             571,516 SPECjAppServer2002_DB2_v5.1_opt.ear
			

The SPECjAppServer EAR file is the benchmark application, which will be deployed on the application server.

Back to top

Install the SPECjAppServer benchmark kit

The SPECjAppServer CD (available from SPEC [1] with a nominal fee) comes with an InstallShield program, setup.exe. If the install program does not locate any JVM, browse to the one that comes with DB2, C:\SQLLIB\java\jdk\bin\java.exe. To keep things simple, install the kit under C:\SPECjAppServer2002. Inside the kit are the benchmark facilities for loading the database and driving the benchmark.

Back to top

Install PERL

We need to use PERL to make the installation process easier. Any standard PERL will serve our purpose. The following listing shows the version of PERL we are using [2].

C:\SPECjAppServer2002\script>perl –v

This is perl, v5.8.0 built for MSWin32-x86-multi-thread
(with 1 registered patch, see perl -V for more detail)

Copyright 1987-2002, Larry Wall

Binary build 806 provided by ActiveState Corp. http://www.ActiveState.com
Built 00:45:44 Mar 31 2003
			

Back to top

Unpack the configuration scripts

Download the ZIP file attached to this documentation and extract the content into C:\SPECjAppServer2002\script. There are four user files in the package.

• setenv.pl
• specjdbsetup.pl
• specjdrvsetup.pl
• specjwassetup.pl

The setenv.pl file consists of the parameters for the setup. The specjdbsetup.pl script is used to create the SPECjAppServer database. The specjwassetup.pl script handles all the deployment tasks of the application. The specjdrvsetup.pl script takes care of the SPECjAppServer benchmark driver setup.

Back to top

Provide the configuration parameters

The setenv.pl file comes with a set of default parameters. If you followed the instructions in the previous sections closely, most of the default values should be good enough.

In the first block of the parameters, we have the directory information of the products installed. Customize them if necessary.

#---------------------------------------------
# Server install directories
#---------------------------------------------
$SpecjKitHome='c:/SPECjAppServer2002';
$JavaHome='c:/sqllib/java/jdk';
$WasHome='c:/was51/b0344.02';
$WasNode='pizza43';                     #Hostname
$Db2Home='c:/sqllib';
$WinHome='c:/winnt';
			

The second block of parameters describes the location of the data. Customize them if desired.

#---------------------------------------------
# Database and log directories
#---------------------------------------------
$SpecjDbDir='c:\specdb';
$SpecjLogDir='c:\speclog';
$SpecjPipe='c:/temp';
			

The parameters in the last blocks point to the location of the SPECjAppServer code and the emulator.

#---------------------------------------------
# EAR install parameters
#---------------------------------------------
$SpecjEar='c:/was51/b0344.02/installableapps/SPECjAppServer2002_DB2_v5.1_opt.ear
';
$EmulatorEar='c:/was51/b0344.02/installableapps/SPECj2002_Emulator.ear';
$SpecjEarDeployed='c:/was51/b0344.02/installedapps/SPECjAppServer2002_DB2_v5.1_opt.ear';
$EmulatorEarDeployed='c:/was51/b0344.02/installedapps/SPECj2002_Emulator.ear';
			

Back to top

Create the SPECjAppServer database

To create the SPECjAppServer database, simply run perl specjdbsetup.pl in a DB2 Command Window. The script will not only create the database, but also dumps out information for problem determination. It is always wise to pipe the output to a file for problem determination, if applicable.

The script will create a user with login name “spec” and password “spec.” You can update your Windows security policy from Control Panel -> Administrative Tools -> Local Security Policy, if necessary.

Administrator            db2admin                 dssadmin
Guest                    spec                     TsInternetUser
The command completed successfully.
			

The SPECjAppServer database consists of 22 user tables in total. Once the database is created, the script will list all the tables in the database.

list tables 
 Table/View                      Schema         
------------------------------- ---------------
C_CUSTOMER                      SPEC           
C_DISCOUNT                      SPEC           
C_PARTS                         SPEC           
C_RULE                          SPEC           
C_SITE                          SPEC           
C_SUPPLIER                      SPEC           
M_BOM                           SPEC           
M_INVENTORY                     SPEC           
M_LARGEORDER                    SPEC           
M_PARTS                         SPEC           
M_WORKORDER                     SPEC           
O_CUSTOMER                      SPEC           
O_ITEM                          SPEC           
O_ORDERLINE                     SPEC           
O_ORDERS                        SPEC           
S_COMPONENT                     SPEC           
S_PURCHASE_ORDER                SPEC           
S_PURCHASE_ORDERLINE            SPEC           
S_SITE                          SPEC           
S_SUPP_COMPONENT                SPEC           
S_SUPPLIER                      SPEC           
U_SEQUENCES                     SPEC           
 
  22 record(s) selected. 
			
			

Then the script will configure the database loader and load the data into the database. Once we have everything, the script will establish a connection to the database and issue a query to the customer table as a sanity check.

select count(*) from C_CUSTOMER

1          
-----------
        750

  1 record(s) selected.
			

The diligent reader should look into the script to understand what exactly is happening behind the scene.

Back to top

Deploy the SPECjAppServer application

To deploy the SPECjAppServer application, simply issue perl specjwassetup.pl from a DB2 Command Window. Be sure to pipe the output to a file for further study.

To start, the script will make sure that all the aliases specdelivery, specemulator, and specdb just created are all reachable. The specdelivery alias represents the IP address of the WebSphere system. The specemulator alias represents the address of the SPECjAppServer benchmark driver or emulator. The specdb alias represents the address of the DB2 UDB server. In our case, everything is local to us and goes through the 127.0.0.1 address.

Pinging specdelivery [127.0.0.1] with 32 bytes of data:

Reply from 127.0.0.1: bytes=32 time<10ms TTL=128
Reply from 127.0.0.1: bytes=32 time<10ms TTL=128
Reply from 127.0.0.1: bytes=32 time<10ms TTL=128
Reply from 127.0.0.1: bytes=32 time<10ms TTL=128

Ping statistics for 127.0.0.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum =  0ms, Average =  0ms
			

Moreover, a DB2 JDBC™ provider and data sources are created through WebSphere JACL scripting. Note that a Type 2 driver was used in the publication, as the Type 4 driver was not certified at that time. For those who are interested in using different JDBC drivers, please refer to the WebSphere InfoCenter [3].

Creating JAAS AuthData SPECAuthData...

Checking for JDBC Provider...
Creating DB2 JDBC Provider on server1(cells/pizza43/nodes/pizza43/servers/server1:server.xml#Server_1)...

Checking for SPECDB datasource...
Creating SPECDB...
Creating datasource properties...
Creating connection factory...

Checking for SPECUtilDB datasource...
Creating SPECUtilDB...
Creating datasource properties...
Creating connection factory...

Script completed...
Saving config...
			

Finally, the script deploys and installs the SPECjAppServer Web application and benchmark emulator.

			
ADMA5013I: Application SPECj2002_Emulator installed successfully.
…
ADMA5013I: Application SPECjAppServer installed successfully.
…			
			

The WebSphere administrative (wsadmin) scripting program is a powerful, non-graphical command interpreter environment that enables you to execute administrative operations in a scripting language. The script is an excellent example of using the program.

Back to top

Set up the SPECjAppServer benchmark driver

To set up the SPECjAppServer benchmark driver, simply issue perl specjdrvsetup.pl at the DB2 Command Window.

The setup script will copy all the deployed classes to the SPECjAppServer benchmark kit. The Jar files contain the objects used in the application.

corp.jar
mfg.jar
orders.jar
supplier.jar
util.jar
        5 file(s) copied.
			

The script will then modify various files in the benchmark driver directories to provide parameters to the driver. Now, the driver should be able to create the agents and start the benchmark.

The following environment settings are in effect for SPECjAppServer 2002
* ========================= *
   JAVA_HOME=c:/was51/b0344.02/java
    JAS_HOME=c:/SPECjAppServer2002
  CONFIG_DIR=c:/SPECjAppServer2002/config
  APPSSERVER=r.1.3.1
* ========================= *
"Apps server is r.1.3.1"
Driver Host: PIZZA43
Binding controller to //pizza43/Controller
Controller: Registering L1 on machine 9.26.99.1
Controller: Registering M1 on machine 9.26.99.1
L1 started ...
M1 started ...
Controller: Registering O1 on machine 9.26.99.1
O1 started ...
RunID for this run is : 2
Output directory for this run is : c:/SPECjAppServer2002/output\2
triggerTime = 30 seconds
benchStartTime = 30640
Configuring 1 OrdersAgents...
OrdersAgent O1, Thread 0 started
OrdersAgent O1, Thread 1 started
OrdersAgent O1, Thread 2 started
OrdersAgent O1, Thread 3 started
OrdersAgent O1, Thread 4 started
Configuring LargeOLAgent...
Configuring 1 MfgAgents...
MfgAgent L1, Thread 0 started
MfgAgent M1, Thread 0 started
MfgAgent M1, Thread 1 started
MfgAgent M1, Thread 2 started
M1:1: Widget cnt = 22
M1:1: End of run. Removing beans
M1:0: Widget cnt = 22
M1:0: End of run. Removing beans
M1:2: Widget cnt = 22
M1:2: End of run. Removing beans
O1:1: End of run. Removing beans
O1:3: End of run. Removing beans
O1:4: End of run. Removing beans
O1:2: End of run. Removing beans
O1:0: End of run. Removing beans
Gathering OrdersStats ...
Gathering MfgStats ...
Timer: startSec in getOffsetTime = 1073600249391
SPECjAppServerReport: Printing report from 1 OrdersAgents
O1:1:  Exiting...
OrdersReport: Printing Summary report...
Transactions/sec = 1.6
OrdersReport: Summary finished. Now printing detail ...
SPECjAppServerReport: Printing report from 2 Mfg Agents
O1:3:  Exiting...
Printing summary report ...
widgets/sec = 18.7
Summary finished. Now printing detail ...
summary file is c:/SPECjAppServer2002/output\2\SPECjAppServer.summary
			

All the results are stored under the c:\SPECjAppServer2002\output directory. Now, you can go and check out your first set of results.

Back to top

Create a bigger database

In the SPECjAppServer2002 benchmark, the size of the system is specified by the injection rate. An injection rate of 1 maps roughly to 8 concurrent clients. During the setup in the previous sections, a database that is supporting the minimal number of users (injection rate 1) has been created and the corresponding load has applied to the system for testing purpose. Although the details of how to run the benchmark is out of the scope of our discussion, it is still worth going through some of the key aspects.

The size of the database is determined at the point where we load the database. To load the database, we use the loaddb.bat script where the injection rate is passed in as a parameter. For example, the commands below will delete the data in the database and load the database supporting an injection rate of 50:

cd c:\SPECjAppServer2002\bin
setenv.bat
loaddb.bat 50
			

Before we kick-off a run, we have to specify the load and the duration of the run. To start a run, we use the driver.bat script, which reads the run parameters from the run.properties file. The txRate parameter is the injection rate. The rampUp, stdyState, and rampDown parameters correspond to the run durations. For example:

txRate = 50
rampUp = 300
stdyState = 600
rampDown = 300
			

and the command below will start a run:

cd c:\SPECjAppServer2002\bin
driver.bat
			

Back to top

Conclusion

In this article, we take the reader through the steps required to set up a single node SPECjAppServer2002 application. The experiences built up here should help the readers to move on to more advanced J2EE topologies and performance tuning concepts. With the help of the configuration scripts used in this article, setting up SPECjAppServer2002 benchmark on DB2 UDB and WebSphere Application Server can be easy. It is our experience that the scripting approach can turn the task from a whole day of struggling to a single enjoyable learning hour.

Acknowledgements

The author would like to thank Peter Shum, the manager of DB2 WebSphere Performance team for his expert advice on benchmarking.

The author would like to thank Christopher J Blythe from the WebSphere Performance team for his great help on WebSphere administration and JACL automation.

The author also wants to thank Robin Dunlop from the DB2 Performance team for his help with ironing out the scripts and the input to the documentation.

Back to top

Download

Information about download methods

Resources

1. Standard Performance Evaluation Corporation (SPEC) information at SPEC.org
2. Find official SPECjAppServer2002 results at the SPEC Website.
3. ActiveState at ActiveState ActivePerl.
4. Websphere InfoCenter at WAS Information Center Reference Library.

About the author

Rate this article

Comments

Back to top

Table of contents

• Introduction
• The SPECjAppServer2002 benchmark
• Install IBM DB2 UDB
• Install IBM WebSphere Application Server
• Obtain the SPECjAppServer code
• Install the SPECjAppServer benchmark kit
• Install PERL
• Unpack the configuration scripts
• Provide the configuration parameters
• Create the SPECjAppServer database
• Deploy the SPECjAppServer application
• Set up the SPECjAppServer benchmark driver
• Create a bigger database
• Conclusion
• Download
• Resources
• About the author
• Comments

Next steps from IBM

• Try: DB2 Enterprise
• Demo: Create an electronic form solution with DB2 pureXML and Lotus Forms
• Community: DB2
• Buy: DB2 Enterprise

Dig deeper into Information Management on developerWorks

Try IBM PureSystems. No charge.

Experience today!

---

Get started with the cloud-based trial and pattern development kit.

Special offers