Upgrade from FileNet P8 3.5 to 4.0: An architectural shift

Considerations for high availability, scalability, and application development

The new IBM® FileNet® P8 4.0 release introduces a major architectural shift in the platform and framework. As a result, customers who want to upgrade to this latest version need to understand the key differences in these respective versions. This article highlights the architectural, high availability, scalability, and development issues that you should consider when upgrading your environment from 3.5 to 4.0.


Marc Grosse (mgrosse@us.ibm.com), Technical Alliance Manager, IBM Japan, Software Group

Marc Grosse photoMarc Grosse is a technical alliance manager in the Enterprise Content Management (ECM) Global Technical Sales team within IBM Information Management. Marc is currently working with Accenture around partner business development, solution recruitment, and skills enablement. Previously, he was responsible for providing technical support in recruiting and enabling business partners across the OmniFind suite of products. Marc has been with IBM for 3 years and has in-depth knowledge of the ECM suite of products. Prior to IBM, Marc spent 6 years working in project management, delivering customer solutions with software vendors in the Enterprise Content Management and Business Process Management space. Marc is certified as a Content Management and Discovery technical sales specialist. You can reach him at mgrosse@us.ibm.com.

07 February 2008

Also available in Russian


Over the past few years, IBM FileNet Enterprise Content Management (ECM) made a strategic investment to re-engineer the previous FileNet ECM Panagon platform to improve scalability and performance, and to take advantage of J2EE standards. IBM FileNet P8 4.0, the newest product release, is designed with the goal of offering a platform of products and solutions that are flexible to be implemented in a variety of deployment scenarios, including operating systems, application servers, and geographically dispersed servers and end users. These improvements offer a distinct change over previous versions that were limited to the Windows® platform and not designed to leverage J2EE standards.

The P8 4.0 release includes more than 75 new features designed to enhance the deployment and development of the platform. With this in mind, the intent of this article is to highlight the several features that partners need to address when upgrading their applications and environment from the previous P8 3.5.2 release to 4.0. Specific instructions and details relative to these key concepts are referenced within P8 4.0 production documentation or technical white papers.

Architecture considerations

The FileNet P8 4.0 release introduces a major architectural shift in the platform and framework. While no architecture changes were made to the application engine (AE), the content engine (CE) has undergone a significant change. This component, which was previously a C++ application running as a Microsoft Windows service, is rewritten as a Java™-based implementation running within a J2EE application server environment. As a result, the CE can be deployed on the J2EE container of the respective support application servers (see Hardware/Software Support matrix for full compatibilities), and has now become platform agnostic running on multiple operating systems (Windows, Linux®, UNIX®). In addition, the new J2EE CE can now leverage resources managed by the application server such as connection pooling, EJBs, and the Java connection architecture (JCA). Previously, these components had to be developed and managed within the CE. For new installations, the configuration and deployment of the CE remains open; however, there are key considerations when upgrading from the 3.5.2 CE to the 4.0 CE. These considerations include:

  • The Global Configuration Database (GCD), previously stored in the local file system, has been reformatted as an XML schema and is stored in the database used by the CE. This removes the need to replicate the GCD manually and thus can be incorporated into standard database backup scripts.
  • While the CE can run on multiple platforms, the FileNet Enterprise Manager can only run in a Windows environment. Therefore, during the upgrade, it is required that a separate Windows machine be allocated in the 4.0 environment to administer the Content and Process Engines (PEs). (The PE architecture considerations are addressed later in this article.)
  • In 3.5.2, communication between the AE and CE leveraged SOAP on the default port 8008. In 4.0, these engines can communicate using the default EJB transport or Web services transport. In most deployments, these two engines will be separated by a firewall. It then becomes paramount that the port opened on the firewall be configured to allow this communication.

Content caching is another key architectural change where partners need to take notice of when upgrading. Previous versions of P8 limited caching to document retrievals only. The ability to cache write operations has been added. New documents or updates to existing retrieved documents are now be stored in that respective location's cache. Thus, any business operation involving this newly created or modified content can access the content locally and not have to travel across the WAN to access the content from the central location cache. This feature again reduces traffic across the WAN and maximizes the user's experience for accessing new content dramatically. Content caching can now be shared across multiple CEs within a local site. Previous versions of P8 did not allow cache to be shared across the servers in the farm. This reduces cache storage requirements per physical server and increases the hit rate of clients accessing the remote farm.

Finally, the PE architecture was modified to take advantage of CORBA ORB communications between the PE and CE server components. More importantly, connection points have replaced the Process Routers from 3.5. These connection points are no longer a process running on a FileNet P8 component; they are now stored in the CE GCD, and they can be managed from Enterprise Manager. The impact of this change is that these Process Routers are no longer started and stopped as Services within the Windows Management Content, easing daily administration of the P8 environment.

Scalability and high availability

P8 4.0 introduces farming and high availability deployments for each of the platform services (CE, AE, PE). In 3.5.2, the CE and PE components required an active/passive cluster configuration. While this solution was considered highly available, it lacked true scalability because client requests could not be spread across multiple machines. The CE no longer comprises separate Object and FileStore Services. These services are now collocated within the CE application deployed within the application server. In addition, multiple CEs contained within multiple J2EE server instances can be housed on one physical server. In essence, the CE can support both horizontal clustering and high availability within multiple deployment scenarios. The two limitations within a true high availability configuration are that within each farm, the respective operation systems must be the same, and the same J2EE application server must be of the same version.

The AE, which leverages J2EE specification in the 3.5.2, remains unchanged in its ability to scale both vertically and horizontally.

While the PE is not a true J2EE-compliant application, it has the ability to scale horizontally, thus removing the need to use active/passive clustering for this component. A software or hardware load balancer can be implemented in front of the farmed PEs to increase processing power, and to achieve high availability and performance across the farm. At any time, servers can be added or removed from the farm while the PE farm is running. More importantly, administering the servers can be done from any server on the farm.

Application development considerations

When upgrading applications from the 3.5 product set to 4.0, partners need to be aware of the various options and considerations before any upgrade can take place. A 4.0 .NET and Java API were added to the CE and contain the following new features and functionality:

  • Compound document support
  • Richer support for batch, query, and transaction management
  • JAAS integration
  • Communication using Web service transport or native EJB transport

Existing applications using the 3.5 Java and Web services APIs that do not need to leverage these new features can merely take advantage of the backwards compatibility provided by the compatibility layers built in the 4.0 architecture. A potential impact to this approach is that the 3.5 API calls are passed to the 4.0 APIs prior to communicating with the CE and thus, will be slower than calling the 4.0 APIs directly. This performance hit might be minimal and ultimately dependent on transaction throughput.

Applications using the 3.x COM API can leverage the backwards compatibility, but additional code changes may be required, as some methods and objects were not deprecated. Most importantly, these COM applications that execute queries against the CE database using ADO and the OLEDB provider will have to be migrated to .NET or Java API, as these features were not extended in the compatibility layer. As a result, the COM API will be the slowest interface in 4.0, as these calls are translated into .NET calls, which use an additional component -- the Web Services listener -- to communicate with the CE EJB layer.

Two different approaches are available for application migration when the new 4.0 features are needed. The Java APIs have been packaged in a fashion that allows partners to use both the 3.5 and 4.0 APIs concurrently. The application can continue to use the 3.5 object model and only access the 4.0 object model to take advantage of new features. This approach reduces both development and application migration time. The other approach is to completely rewrite the application to use the 4.0 Java API. The advantage of this approach is that the Java API uses EJB transport, the fastest communication protocol to talk with the CE. In general, it is recommended that applications requiring maximum performance should use the native 4.0 Java API, instead of the backwards compatibility layers.

While building client applications on the P8 platform, partners may have written custom modules that leverage the customization capabilities on the platform. P8 3.5 provided the ability to use ActiveScript for writing Event Action customizations for the CE. This is no longer supported as P8 4.0. The new Event Action Framework only allows partners to register Java programs that are linked to events fired on the CE. Since Java modules are only supported, these customizations have to be rewritten as Java programs. Despite this major change, Java scripting is still supported for pre-installs, post-installs, and import operations. Existing VBscripts written on the 3.5 platform have to be rewritten in JavaScript.

Implementing security in 4.0

Previous versions of P8 relied heavily on passing username and passwords for authentication to both the CE and PE. Due to the architectural changes to the CE, a much greater set of authentication options are now available. The two authentication standards that are now supported include the Java Authentication and Authorization Service (JAAS) and its Web Services counterpart (WS-Security).

JAAS, a pluggable framework within J2EE, allows application server vendors, authentication, and single sign-on providers to deliver authentication packages to be leveraged by all J2EE applications and clients. Now that the CE is deployed as a J2EE application, it can leverage the built-in JAAS modules delivered with the application server vendor to pass credentials to/from the existing client application and the CE. Previously, both the CE and PE contained application code to manage these authentication actions. To further expand on JAAS authentication, the PE no longer requires credentials to be passed from the CE. The PE now relies on the CE for full authentication. Once the user request has been authenticated and passed using JAAS, the PE is then passed the authenticated token from the CE to complete any process-specific transaction.

These JAAS and WS-Security standards also affect how customers implement custom single sign-on solutions on P8 3.5. With the architecture change in the 4.0 release, these custom modules developed within the Extensible Authentication Framework (EAF) will no longer work after an upgrade is performed. A gap analysis should be performed to evaluate the custom SSO solution in 3.5 with the JAAS login module provided by the chosen SSO solution provider to determine if the JAAS SSO module contains the functionality addressed in the custom SSO integration. For applications leveraging the Web Services API, P8 4.0 provides a Web Services Extensible Authentication Network (WS-EAF) for implementing SSO solutions for credential types, which are not supported by out-of-the-box functionality. Additional information on the WS-EAF can be found in the P8 4.0 online help.

The P8 3.5 EAF also allowed implementing a client-side DLL to allow the P8 Office integration to leverage the SSO solution. Unfortunately, there is no general approach to upgrading this functionality. It is recommended to review the WS-EAF to determine if the existing functionality can be ported over with this framework.


The goal of this article is to highlight the technical components partners need to consider when upgrading their respective application or a customer's environment from P8 3.5 to 4.0. These key items include the following:

  • P8 Content Engine - a J2EE application contained in the application server
  • Horizontal and vertical scaling of the Content Engine and Process Engine
  • Implementing content caching
  • JAAS and WS-Security integration
  • Extended compatibility layer for previous applications
  • Java compatibility in the Event Action Framework



Get products and technologies

  • Build your next development project with IBM trial software, available for download directly from developerWorks.



developerWorks: Sign in

Required fields are indicated with an asterisk (*).

Need an IBM ID?
Forgot your IBM ID?

Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.


All information submitted is secure.

Dig deeper into Information management on developerWorks

Zone=Information Management
ArticleTitle=Upgrade from FileNet P8 3.5 to 4.0: An architectural shift