IBM WebSphere Developer Technical Journal: Lessons learned migrating IBM's intranet to WebSphere Portal

Find out what the IBM intranet team learned when they migrated the IBM intranet to WebSphere® Portal.

Kerry Thompson (thompson@us.ibm.com), Technical Writer, IBM Corporation

Kerry Thompson is a technical writer in the IBM Lotus Software Division. She has worked on documentation for Notes and Domino, LearningSpace, the Learning Management System, and Lotus Workplace during her 12 years with Lotus.



Deak Shearer (deak@us.ibm.com), Certified IT Architect, IBM Global Services

Deak Shearer is a Certified IT Architect with IBM Global Services.



Dima Rekesh (dima@us.ibm.com), IT Architect, IBM w3 Architecture and Development

Dima Rekesh is an IT Architect with IBM's w3 Architecture and Development team.



12 May 2004

Introduction

The w3.ibm.com site is IBM's intranet portal for its employees. In February of 2003, the Web development team decided to switch the site's infrastructure from WebSphere Application Server to WebSphere Portal. The previous version of IBM's intranet portal used a custom portal framework. While lightweight and scalable, it was not entirely compatible with WebSphere Portal portlets, which prevented w3.ibm.com from leveraging existing libraries of portlets built for WebSphere Portal. The goal of the migration was to move to a WebSphere Portal architecture while preserving the existing look, feel, functionality, and performance characteristics of the existing site. A team of seven people completed the migration process in just over three months and the new Web site went live in June 2003.

At the time this development plan began, Version 5 of WebSphere Portal and WebSphere Application Server were not yet available. While some of the challenges encountered with this project were either unique to w3.ibm.com, or characteristic of these earlier product versions, the approach taken and resolutions developed may be beneficial to experienced Web application developers and architects involved in migrating to or developing applications for the WebSphere Portal environment.

This article summarizes some of the major points in the migration process, and highlights some of the lessons we learned along the way.


Design objectives

The w3.ibm.com Web site (hereafter referred to as w3) opens with a default Home page. If the employees are signed in, they see their personalized home pages from which they can jump to IBM business news, search for information about other employees in the IBM BluePages, and get benefits and career information.

Figure 1. IBM's intranet
IBM's intranet

As we began to plan the project, these goals formed the nucleus of the design plan:

  • Preserve the existing codebase
    The objective was to migrate the w3 site quickly. To that end, we aimed to use as much of the existing code base as possible. This task was made easier by the existing architecture of the w3 site. Because the custom portal framework powering the site was loosely derived from an earlier version of WebSphere Portal, the portlets written for it did not require rearchitecting. We only had to replace custom portlet containers with their WebSphere Portal counterparts. The actual JSPs that displayed the content of each portlet required almost no changes. The most challenging task was to customize the WebSphere Portal themes and skins (shared design components of the portal page that are not controlled by individual portlets) so as to make them compliant with the site's original design. This goal also helped to narrow our focus and significantly reduce testing efforts because we knew that with very few exceptions, the code had already been running in production for a long time.
  • Customize the Web site design
    We chose to customize the main w3 page to look like the existing w3 page, rather than use a standard WebSphere Portal page. We made this decision because of the very short project deadline and because w3.ibm.com was to be the largest WebSphere Portal installation to date. This decision gave us the ability to bring the WebSphere Portal version of the site down if needed and replace it with the older version without users noticing any differences in the site. We changed the theme and skin for the Home page and then made further modifications to meet some of our other design goals for the Back button, the Essential Links section, and the Help and Edit links.
  • Make error messages consistent
    Because the site was to use WebSphere Portal, WebSphere Application Server, and IBM HTTP Server to serve pages, the default design of the error message pages would have been inconsistent. By modifying several of the configuration files, we changed which pages display for certain errors to make the "look and feel" of the pages consistent across servers.
  • Share session information across separate parts of the application
    We had to customize the w3 application code to share profile changes and changes in personalized portlet settings between the two EAR files that make up our application.
  • Make security changes to identify users
    We needed to customize the security code that identified users by a w3 profiling cookie to integrate it with the WebSphere Portal security model.
  • Develop and test for optimal performance
    We had a goal for how quickly a page on the site should appear. We investigated several issues to reach that goal.

Preserve the existing code base

The original code base for w3 was developed to run on WebSphere Application Server V3.5. We repackaged it to be compliant with the J2EE packaging requirements implemented in WebSphere Application Server V4. Then the Portlet API was used to wrapper each of the pre-existing w3 portlet applications. This jump start allowed us to cut development time down to about seven weeks.


Customize the Web site design

The team wanted a Web site that looked like an IBM intranet site rather than a standard portal. We accomplished this by switching the theme and skin, and then made some further modifications to work around some default WebSphere Portal features that we chose not to take advantage of.

Cleaning up existing JSPs

To begin editing the JSP files, we copied the files out of the WAR directory into a Web project created in WebSphere Studio Application Developer. We then created an Ant script to take the project, WAR it back up, and then unzipped it, expanding it back in place over the existing file system. (Ant is a Java-based build tool associated with the Jakarta Apache project.) We did not use XML access or the Admin interface for redeploying because we were not changing anything in the portlet.xml file.

We ran WebSphere Studio Application Developer's "clean up document" function to make the code a bit closer to XHTML. In the process, all parameters were changed to lowercase, along with the quotes, breaking the JavaScript calls that the portal"s Quick Customizer created.

After restoring the JavaScript for each page, we then started adding the custom design. At first, we did not want to change the code too much. But since we had to support Netscape Navigator™ 4.x, we found that it was helpful making the page entirely XHTML compliant. We used xmllint on Linux and CSE HTML Validator Lite 3.5 on Windows® to help flag changes we needed to make.

One important thing we learned for tracking down bad XHMTL was to put at the top and bottom of each JSP file the name and version number of the file in question. Making use of version variables from our source control system, we put

<!-- $Id: $ -->

at the top of each page, allowing us to see exactly which JSP had the bad XHTML. (It would have been a good idea to put this labelling at the bottom also.)

Maintaining the Back button behavior

The Back button worked differently under WebSphere Portal than it did under WebSphere Application Server. We wanted the Back button to return the user to the prior view under all conditions, but there were some conditions when that was not the result. For example, if the user opened the Home page with the portlet in normal view mode, maximized the page, clicked Back to return to the normal view mode, then clicked Refresh, the portlet would display at the maximized size rather than in the normal view mode. Additionally, we also found conditions in which the Back button disabled the actionPerformed() function, which enables users to navigate between portal pages in WebSphere Portal.

To work around these issues, we decided to have WebSphere Portal render the w3 Home page stored in one EAR file, and then use another EAR file to reuse code that renders the Maximize, Edit, and Help pages as JSPs.

Modifying portlet.xml to capture Help and Edit URLs

Due to some of the migration issues we encountered, we decided to have WebSphere Portal manage only the normal view of the portlet. We modified the skins so that rather than generate URLs to the Help and Edit pages, they would instead take the URL definitions from the portlet.xml file. To do this, the WebSphere Portal development team provided an unsupported API that enabled access to the portlet's defined parameters from portlet.xml. This enabled the capturing of the external links to Help, Edit, and Maximize, as defined in portlet.xml. For the sake of efficiency, and to avoid offering users a choice of portlet skins, the skins were merged into a single skin whose color parameters were also defined in the portlet.xml. Thus, the look and feel, as well as the usage of the portlet, could all be controlled from one file.

The URLs created in WebSphere Portal are based on a base URI and a numbering scheme that identifies components in the portal. The URL for a single page is assigned when the page is created, but can change when the page is updated or when a user chooses to customize an instance of the page. We investigated how to consistently identify these URLs for collecting site metrics and for bookmarking purposes. One suggestion offered to the team was to implement a URL remapper that would sit between the proxy and WebSphere Portal layers to translate URLs. (We could also have waited for the release of WebSphere Portal V5, which would have a built-in URL mapping capability.) Because the w3 site only had one page being rendered by WebSphere Portal, URL mapping was not an issue for our project.

Including the Essential links portlet in the theme

In w3, The Essential links portlet gives employees a list of links that let them find their favorite areas of the Web site quickly. We wanted to have the Essential links portlet display in the left navigation bar, but the standard WebSphere Portal theme would not let us put navigation into a portlet column, nor put a portlet in the navigation area. To circumvent this issue, we modified the theme. First, we added the Essential links portlet to the theme as a server side include. We then modified the theme generation code to show the Essential links only when the user views the theme as the Home page. This gave us the desired effect of showing the portlet in the left navigation bar of the theme.

Suppressing unneeded navigation elements

To avoid confusing non-administrative users, we added a mechanism to suppress the default links in the left navigation menu. This method was based on cookie detection, whereby the cookie is only set if an administrator logs in. Only administrators see the default behavior, which consists of a link in the left navigation bar that goes to the front page of the Edit Layout section. By contrast, non-administrative users get to the Edit Layout page from different links: either using the Edit link in the main navigation bar, or the Edit Page Layout link in the upper right menu.

Working around limitations with minimized portlets

The previous w3 site could maintain a minimized portlet state in a near real-time fashion. When a user minimized a portlet and then opened multiple browsers to the w3 site, each browser displayed the minimized portlet. We were not able to maintain this feature in the new w3 site.

WebSphere Portal can be configured to save a portlet's state after a user's session ends. However, we did not take advantage of the full range of WebSphere Portal states in our site. We used WebSphere Portal for the minimized portlet state and switched to our own JSPs for the Maximize, Edit, and Help states, which means that a minimized state is only saved after a session ends. In addressing this, we decided to set the session timeout to a low value to limit the number of active sessions, and then turned off session persistence to reduce the extra overhead incurred by saving states after the user's session ends. The w3 state would then only maintain minimized portlets during the course of a single session.


Make error messages consistent

To customize the error message from IBM HTTP Server, we updated the /usr/HTTPServer/conf/httpd.conf file with the appropriate ErrorDocument directives. Following is an example of the directives we added:

#
# Customized error messages for the w3 application
#
ErrorDocument 400 /error400.html
ErrorDocument 401 /error401.html
ErrorDocument 403 /index_default7.html
ErrorDocument 404 /error404.html
ErrorDocument 500 /error500.html
ErrorDocument 503 /error503.html

To customize the error messages from WebSphere Portal, we updated the web.xml file under the ../wps.ear/wps.war/WEB-INF directory. Following is an example of the tags we added:

      <error-page id="ErrorCodeErrorPage_1">
         <error-code>404</error-code>
         <location>/w3screens/error404.jsp</location>
      </error-page>
      <error-page id="ErrorCodeErrorPage_2">
         <error-code>503</error-code>
         <location>/w3screens/error503.jsp</location>
      </error-page>
      <error-page id="ErrorCodeErrorPage_3">
         <error-code>403</error-code>
         <location>/w3screens/error403.jsp</location>
      </error-page>
      <error-page id="ErrorCodeErrorPage_4">
         <error-code>500</error-code>
         <location>/w3screens/error500.jsp</location>
      </error-page>      
      <error-page id="ErrorCodeErrorPage_5">
         <error-code>400</error-code>
         <location>/w3screens/error400.jsp</location>
      </error-page>

To customize the error messages from WebSphere Application Services, we updated the web.xml file for each of EAR/WAR file deployed. Following is an example of the tags we added:

      <error-page id="ErrorCodeErrorPage_1">
         <error-code>404</error-code>
         <location>/w3screens/error404.jsp</location>
      </error-page>
      <error-page id="ErrorCodeErrorPage_2">
         <error-code>503</error-code>
         <location>/w3screens/error503.jsp</location>
      </error-page>
      <error-page id="ErrorCodeErrorPage_3">
         <error-code>403</error-code>
         <location>/w3screens/error403.jsp</location>
      </error-page>
      <error-page id="ErrorCodeErrorPage_4">
         <error-code>500</error-code>
         <location>/w3screens/error500.jsp</location>
      </error-page>      
      <error-page id="ErrorCodeErrorPage_5">
         <error-code>400</error-code>
         <location>/w3screens/error400.jsp</location>
      </error-page>

To turn off the error message that displays when users log in again after shutting down their browsers without logging off, we made the following change in <WAS>/lib/app/config/services/ConfigService.properties:

set
timeout.resume.session=true

Share session information across separate parts of the application

As mentioned earlier, our w3 application uses two EARs: one for the w3 Home page and another for the Maximized, Edit, and Help portlet pages. The components in our application use HTTP Session to place objects in session, saving processing time across repeated invocations. In other applications, a failure to locate an object in session simply results in another request to the data source. This degrades response time, but may not necessarily be critical.

For our w3 site, however, there are times when users would not receive the information they expected if sessions did not retain objects across HTTP requests. For example, suppose a user opens the w3 Home page (served by the first EAR), clicks Edit in the MarketReport portlet (invoking the second EAR), and adds a new stock symbol. When he submits the change (which invokes the second EAR again), he is then redirected to the Home page (which is still running in the first EAR). The user expects to see the new stock in the MarketReport portlet on the Home page.

To get this to occur, we had to figure out how to enable communication to the UserProfile object, stored in Session, between the two EARs. We accomplished this by using our custom w3.portlets.PortletJspHelper class. The class includes the public static java.util.WeakHashMap userProfiles variable, which stores all the user profiles that have been recently retrieved from the database. This variable is visible from both EARs and is checked before the profiles are retrieved from the DB2 database. This enables sessions to have the most recent information.


Making security changes to identify users

For our new w3 site, we needed to:

  • Identify users by a w3 profiling cookie (as opposed to prompting them for a user ID and password).
  • Integrate the w3 profile with the WebSphere Portal profile system.
  • Give the users the option to sign in with their intranet user ID and password to gain access to secure site areas.
  • Provide some level of security for the users who need to administer WebSphere Application Server and WebSphere Portal.

Three security components came into play to meet our security requirements:

  • Trust Association Interceptor
  • Custom User Registry
  • Custom Member Repository.

Modifying the Trust Association Interceptor

Trust Association Interceptor (TAI) is a WebSphere Application Server facility that runs in the application server JVM. Its purpose is to check whether or not the user is authenticated, and if the user is authenticated, to retrieve the appropriate user ID. There can be more than one TAI; each TAI in turn examines the request and decides whether or not it can handle the request. If it cannot, the request is examined by the next TAI in the list or, if the end of the list is reached, authentication is handled according to the method defined in the particular Web application. WebSphere Portal utilizes a login form (default processing). TAIs are only consulted for Web requests. They are not consulted when a user, for example, tries to administer WebSphere Application Server.

A TAI is not invoked each time a user requests a page. True to the session paradigm, WebSphere Application Server invokes a TAI to validate a user's authentication only once per session. After the TAI successfully completes, the user receives the LTPA cookie, which is valid across requests.

To ensure proper and secure authentication for the users who can administer WebSphere Portal, our custom TAI was configured to stop processing the request when it encounters the WebSphere Portal admin cookie. When this occurs, authentication is handled by WebSphere Portal login form.

Modifying the Custom User Registry

The Custom User Registry is a WebSphere Application Server facility that runs in the admin server JVM and, consequently, does not have direct access to the application classes located in the application server JVM. The custom registry is the only module consulted when users administer WebSphere Application Server, helping to look up a user record, verify the password, get group membership, and so on. In the context of Web processing, the call registry is used to retrieve the user's unique ID, display name, and group membership, based on the user ID reported by the TAI.

Because the custom registry runs in the WebSphere Application Server admin server, it does not have direct access to user profile information present in memory caches of the main application server. Our tests indicated that remote calls to retrieve this information would adversely affect performance. We were not providing full integration with the w3 entitlement system, so the amount of information that the custom registry was returning was limited. For non-administrators, it only had to return the unique user ID and the real user name.

Basing our custom registry on the sample we received from the WebSphere Application Server security group (which essentially used an XML file to store user and group accounts), we modified the registry to have the unique ID and real name information passed to the custom registry by the TAI as part of the actual call to retrieve them. In doing so, we constructed the non-unique user ID to include these fields, and leveraged the fact that the TAI runs in the application server JVM and has easy access to this information.

Our custom registry provides only enough group support to allow administrators to maintain WebSphere Portal. We did not fully integrate it with the w3 entitlement system, which combines the benefits of Blue Groups as well as the enterprise taxonomy. As a result, we do not restrict access to individual portlets based on user profile. (We hope to integrate the w3 segmentation with WebSphere Portal groups in the future, but have not yet done so.)

Modifying the Custom Member Repository

This WebSphere Portal-only facility runs in the application server JVM. Like the Custom User Registry, the Custom User Repository is capable of retrieving the entire user record by the user's unique ID, which the custom registry returns. It has some facilities for defining the supported user attributes that WebSphere Portal supports and mapping those to the attributes of the w3 profiling record.

If the TAI finds no w3 profiling cookie in the request (because the user does not have a profile), its validateEstablishedTrust method will throw a WebTrustAssociationFailedException exception, causing a redirect to the error 403 page. The desired behavior here was to present the user with the static, uncustomized w3 page that contains the w3 (not WebSphere Portal) login box. We accomplished that by registering the 403 page, as described earlier in the Make error messages consistent section.

Combining WebSphere Application Server security and basic athentication

As mentioned earlier, w3.ibm.com has several secure areas (for example, the profile settings pages), access to which was protected by HTTP Basic Authentication. As we quickly learned, however, servlets and JSPs were not able to fully leverage that protection because WebSphere Application Server was setting the RUSER variable available to these components to a null value. This was a known issue with Version 4 of WebSphere Application Server V4 security, and there were two possible ways to work around it: run these pages on a different WebSphere Application Server where security was disabled, or to stop using basic authentication. We chose the first option, primarily because it required less time and effort.


Develop and test for optimal performance

Our target performance criteria was to have a page display in less than 250ms on the server. Our team explored the following factors to acheive performance improvements:

First time user login

The first time a user logs in, WebSphere Portal creates table entries for each new user, which slows down new-user logins. We have approximately 200,000 unique user IDs in the w3 profile database. The w3 users are uniquely identified by an HKEY ID. To address this factor, we preloaded each of the users into the portal prior to releasing the site live. This was done early in the testing phase to perfect the process of pre-loading and to get a starting performance benchmark.

The pre-loading was incorporated as part of the load testing, which was set up to read a list of all of the HKEYs and log each user into the portal once. The users were not logged out, but we allowed each user session to expire after two minutes. This initial activity was somewhat overwhelming for the servers when we first tried this, and we had to make multiple runs to get all of the users logged in at least once. After the portal databases were primed with all of the w3 user HKEYs, we ran the DB2 reorgchk utility to update the database statistics and produce a report of each of the tables that required reorganization. We then reorganized the appropriate database tables. It was important to do this priming of the portal databases prior to running any meaningful performance load tests.

Login and page rendering time

BluePages is the IBM employee LDAP directory and the source of user authentication. The BlueGroups LDAP directory is the source for group authentication. The portal made several calls to both LDAP servers to complete the portal login sequence, and made additional calls in the case of users who were members of nested groups in the BlueGroups directory. In addition, the Portal made calls to LDAP each time it rendered a portal page. It appeared that this default behavior could be resulting in performance problems.

To speed up login and page rendering time, we customized WebSphere Portal security features to eliminate all calls to LDAP. Instead, we took advantage of the CustomUserRegistry and MemberRepository interfaces provided with WebSphere Portal, enabling us to integrate the portal security functions with the w3 User Profile mechanism, which already existed in the w3 application. We then turned off Nested Groups functionality in the portal by editing the \AppServer\lib\app\config\services\AccessControlService.properties file to change accesscontrol.nestedgroups to false.

Netscape rendering time

The Netscape rendering issue affected client-side performance of the browser when Netscape tried to render a portal page. (There was no issue on the server-side for serving portal pages to Netscape clients.) We noticed that performance worsened when there were nested tables in the HTML that rendered a portal page. We therefore reduced the number of nested tables in the themes and skins. Currently, the Home page and the Quick Customizer display no more than four nested tables for Netscape Navigator 4.x. The Quick Customizer has two separate versions, one for Netscape, and one for every other browser. The Home page shows no more than four nested tables for all browsers.

There was a rendering bug that was discovered with style= attributes in HTML tags with Netscape 4.7x. We converted all style attributes to class attributes and defined CSS classes to perform the same function as the inline defined style attributes.

Memory leaks

The w3 load testing revealed a memory leak issue running on WebSphere Application Server V4.0.5 and WebSphere Portal V4.2.1. The leak appeared to be in WebSphere Application Server and improved significantly after applying WebSphere Application Server PTF # PQ69054.

Parallel portlet rendering time

Parallel portlet rendering attempts to execute the code for each portlet in parallel to speed up the rendering of the portal page. WebSphere Portal offers a parallel portlet rendering feature that can be defined in the portlet and Web XML configuration files. We had wanted to use this feature to speed up the rendering of the Home page, especially for establishing the initial WebSphere Portal session. However, due to apparent thread management issues that were detected during development, we were unable to exploit this feature. We were, however, able to accomplish our performance goals without needing to rely on parallel portlet rendering.

A non-profiled (static) Home page

Our data sources indicate that only about one third of w3.ibm.com users are typically signed in with their user IDs and passwords. For these users, the Home page is generated dynamically, based on their individual profile preferences. The remaining users, however, are not signed in. Because the system cannot identify these users, it serves the same page to all of them.

WebSphere Portal is certainly capable of displaying non-profiled pages, but we saw an opportunity here to reduce our hardware requirements three-fold. Because two out of three users see the same page, we could save time by generating a page asynchronously just once, and subsequently serving it as a static HTML file. The component used for this task was implemented as a cron job. At predefined intervals, it retrieves the dynamic version of the Home page designed for a non-profiled user, and saves the output in an HTML file. This file is then served to all users who are not logged in.

Session size

We did not make changes to the code that stores transient information in session objects. However, in WebSphere Portal, our sessions size was significantly larger (approzimately 150K) than it had been previously. Since we were not able to control the session size, we resorted to increasing the available heap space (see the next section) while reducing the session timeout.

Application server JVM heap size

Despite our extensive use of memory caches, the earlier version of w3 ran in production with about 450MB of available heap space. It was clear from the beginning that WebSphere Portal would require more. Tests revealed that the optimal heap size for the new system had risen to approximately 2GB. To increase this setting, we added the following line to startupServer.sh file:

export LDR_CNTRL=MAXDATA=0x20000000

Lessons learned

We achieved satisfactory successes during our development and testing cycles, and were pleased with our development software and the performance of WebSphere Portal. Setbacks that we encountered were primarily due to hardware limitations and general migration challenges between the old application and the new.

In our development and testing, we used three environments:

  • Local development machines
    Each development team member's workstation had the following software:
    • WebSphere Studio Application Developer for code editing
    • WebSphere Portal installed locally for testing
    • CVS repository used for code library management
    • Ant, a tool for building code packages.
    Ant and CVS were used in conjunction with WebSphere Studio Application Developer to publish and package content easily using command-line Ant deployment. Without that capability and the time taken to put that into effect early on, we would not have been able to work efficiently to update code from CVS to WebSphere Portal.
  • Development test server
    For integrating and testing code from multiple developers, we set up a test server with the following software:
    • AIX®
    • WebSphere Portal
    • WebSphere Application Server
    • DB2®.
  • Full testing server cluster
    For complete integration, functional, system, and performance testing, we used a two-node cluster of WebSphere Portal on two IBM pSeries 650 machines with WebSphere Portal and WebSphere Application Server clustered on two machines, with DB2 installed on the first machine in the cluster (we had limited machines available).

What we learned about packaging

Originally, we packaged every portlet as a separate WAR file. This was convenient for the developers because it allowed each portlet to be updated independently. However, this approach had a negative impact on memory because every portlet gets packaged as a separate EAR file, and in WebSphere Portal, each EAR file gets its own session context. Our w3 Home page can have up to ten portlets, resulting in up to ten session contexts per page. Each session context is very memory intensive, which means that ten times more memory would have to be allocated to accommodate each session context.

Our next strategy was to reduce the number of EAR files to as few as possible. For the initial release, we shipped with two EAR files: one for the J2EE application with all the non-portal specific code, and one WAR file with just the portlet controllers and their associated JSP files.

The current w3.ibm.com Home page is handled by WebSphere Portal, while the Maximized, Edit, and Help portlet pages are independent JSPs and servlets that run in their own independent EAR. This architectural decision was based on the requirements to ensure that Back button worked as expected and to have predictable, bookmarkable, and fixed URLs for the Maximized, Help, and Edit pages, as well as any pages associated with them. We have since found out that it is possible to deploy a single EAR file containing all of our code, install it through the WebSphere Application Server facilities, and then use a specific parameter of the xmlaccess tool to extract and import the portlet information from it. (We have not yet tested this approach.)

Planning and testing site hardware

We knew it was important (as it is for all sites that have high-volume throughput) to do benchmarking with our application early in the development process and then to size the servers appropriately.

The w3 site collects traffic statistics and logs URLs with a tool called SurfAid. This site/URL-specific traffic information was used as the basis for planning our load testing with WebSphere Portal. Data was examined for monthly, weekly, daily, and peak hour time periods. The base unit of measure for determining our expected load was "page views per second," which measures how many pages the portal server can serve for each second during peak load. We decided that the servers should be able to serve approximately 20 page views/second on a normal day during peak load and up to 40 page views/second on special event days during the peak load.

Initial recommendations from the WebSphere Portal developers indicated that two p640 4-way 1GHz machines with 8GB RAM would serve 100 page views/second at 80% capacity. We found this to be an optimistic benchmark that can vary depending on the characteristics of the application running on WebSphere Portal. For the w3 application, we were able to achieve 60 page views/second per server for extended periods of time on a cluster of four p650 4-way 1.2 GHz machines. In our performance tests, we ran a 200% load test (adjusted for the two-machine cluster) of 24 page views/second and the two-machine cluster of p650 1.45 GHz test machines ran at approximately 65% CPU utilization. We attributed our slower performance to the fact that the w3 application runs on the same servers as the w3 WebSphere Portal application and that our application is much more complex than those used for the WebSphere Portal team's own performance tests.

The previous w3 site consisted of a cluster of seven Web Traffic Express proxy servers, four WebSphere Application Server V4.0 servers, and two DB2 servers. The migration project added an additional four WebSphere Portal servers and two DB2 servers configured for HACMP failover. The final hardware and software configuration for the w3 site is shown below.

Server Type Quantity Software Hardware
Proxy server3WebSphere Edge Server 4.0.1.32WinterHawk Thin, 2way, 375MHz, 2GB RAM
Proxy server4WebSphere Edge Server 4.0.1.32p640 (7026-B80), 4way, 375MHz, 3GB RAM
Application server4WebSphere Application Server 4.0.5
IBM HTTP Server 1.3.19.3
AIX 4.3.3
Silver Wides (PPC 604e) 4 X 332 MHz
3GB RAM
2 X SSA adapt. (FC6230)
Application staging server1WebSphere Application Server 4.0.5
IBM HTTP Server 1.3.19.3
AIX 4.3.3
Silver Wides (PPC 604e) 4 X 332 MHz
3GB RAM
2 X SSA adapt. (FC6230)
Portal application server4WebSphere Portal 4.2.1 Extend
WebSphere Application Server 4.0.5
IBM HTTP Server 1.3.19.3
AIX 5.1
p650 (7038-6M2), 4way, 1.2GHz, 16GB RAM
Portal application staging server1WebSphere Portal 4.2.1 Extend
WebSphere Application Server 4.0.5
IBM HTTP Server 1.3.19.3
AIX 5.1
p650 (7038-6M2), 4way, 1.2GHz, 16GB RAM
Database server2DB2 7.2.8
AIX 5.1
p650 (7038-6M2-450M) 4way, 1.45GHz, 8GB RAM

Installing and updating w3

The w3 installation process uses a silent installation approach through xmlaccess. The installation addresses installing the base product and application code in a cluster environment. The clustering addresses horizontal clustering with one WebSphere Portal/WebSphere Application Server clone per machine node. The application installation process utilizes Ant scripts and xmlaccess.

Content and JAR file changes are not complete redeployments of the application. They are incremental installations where we simply move the incremental changes to the production server and recycle the application servers. The incremental builds and auto deploy are automated using Ant. xmlaccess is used to install the portlet code into the Portal environment and for portlet.xml changes.


Conclusion

Although, we were unable to resolve every issue that we found during the original migration effort, we were extremely satisfied with what we were able to accomplish in a relatively short period of time. As a result, our application was a hybrid WebSphere Application Server/WebSphere Portal application, and we looked forward anxiously to addressing remaining challenges in future revisions.

Figure 2. IBM's On Demand Workplace
IBM's On Demand Workplace

In the time since this original project and article were completed, additional development and significant enhancements have been applied to the w3 site, as is evident in Figure 2. The new site complies with new intranet standards for look-and-feel, offering a better, more intuitive user interface that features a convenient tabbed interface and a number of new portlets. The developers plan to describe these enhancements and modifications, and share their experiences again, in a future article.

Resources

To learn more about server performance with WebSphere Portal or WebSphere Application Server, you may find the following guides helpful.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Select information in your profile (name, country/region, and company) is displayed to the public and will accompany any content you post. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=14625
ArticleTitle=IBM WebSphere Developer Technical Journal: Lessons learned migrating IBM's intranet to WebSphere Portal
publish-date=05122004