Performance is an area in which IT teams often seek consultative advice, whether it be for general guidance or to address specific critical performance issues that threaten the stability of an application or website. While either is a clear and valid reason to discuss website performance, what might be less clear is exactly what is meant by “performance,” what are the things that impact it, who needs to be involved to improve it, and perhaps most important: why it needs to be an integral part of the development process.
This article outlines the important ideas behind the performance lifecycle to help you prepare for what is often one of the most challenging, critical, and neglected elements of web project development, both from a project management standpoint and an execution standpoint. Questions to be addressed include:
- Why do both the business sponsors and IT sponsors need to care about and understand performance fundamentals?
- What does “performance” encompass?
- Why is it necessary to have a business plan in order to achieve a high-performing website?
- What is the performance lifecycle?
- How should the business and IT teams collaborate to build an effective performance strategy?
Performance has a direct impact on business objectives, and vice versa. For example, the highest priority for nearly all retailers, is to drive revenue and turn a profit. To do this, one retail business goal might be to maintain a competitive edge with exciting and unique customer experiences. This can include offering products and services to the consumer in a meaningful way that encourages them to make a purchase and return for more, or providing an experience that illustrates the value of the goods and services offered and lets customers communicate with like-minded individuals, and so on, all to maintain an advantage over a competitor who is just a click away.
To achieve these critical success factors, an underlying requirement is that the website must provide a reliable user experience, giving shoppers what they want with very rapid response times. If you consider that all of this must be done in a profitable way, a key metric becomes the return on investment of the hardware and software considered in proportion to the amount of revenue that will be gained; in performance terms, we talk about the amount of the hardware and software as aspects of the capacity plan.
So, what if you do not consider the performance of a website, either before launch or prior to other significant events on the site? Could the effects really be that detrimental? The answer is yes, and emphatically so. Here are just a few possible outcomes of not adequately considering performance in your development cycle:
- Website failures during peak times. In the brick and mortar world, this is akin to not being able to open the front doors of your store. This leads to:
- Lost revenue.
- Negative customer experience, which tarnishes your brand and encourages your customers to visit competitors.
- Potential legal issues and penalties. Examples might include affiliates and partners to whom you have commitments, which you are unable to meet with an unstable or unavailable site, or scenarios where there are time commitments for completing a transaction (especially important in financial transactions).
- Increased costs for your customer support organization, as your call center becomes busy during times of online crises.
- Website slow downs, which lead to:
- Perception of the site being unavailable. All of the negative aspects of an unavailable site apply here. With the online world always getting faster and competition constantly growing, it takes much less time than you might think for users to get impatient with your site.
- Compounded performance problems. Perception that a user request has not gone through might cause users to try again, creating frustration, bottlenecks, and additional complexities for handling duplicate transactions.
- Patchwork solutions by adding more hardware and software than originally anticipated to mask website inefficiencies:
- While this may work in the short term, the costs of such a strategy take away from profitability, and could tarnish the reputation of the development or IT team responsible for the website.
- This strategy could drain funds from other important projects.
The story is not all doom and gloom though. While the intent is to clearly show that by ignoring performance the negative impacts can be significant, the remainder of this article presents ideas to help you begin implementing a high performing site that meets critical business needs at all times, including the most important times for your business.
Application performance is often equated to responsiveness. Since speed is subjective, its definition varies depending on the audience and organization. A retail company might consider its website performance to be fast if it can serve catalog pages and place orders in a few seconds. In contrast, a stock trading application measures speed in milliseconds. As a user, you might tolerate a few seconds to perform a banking transaction online, but if a shopping site is slow, a viable option might simply be to navigate away to an alternative site.
The retail example highlights the importance of application performance and how it connects to a company’s bottom line, as every minute on a busy day could translate to significant sales won or lost. Performance, therefore, is not merely a benefit for user experience. It contributes to a site’s reputation, boosts customer loyalty, and can have significant impact on site revenue. Ultimately, performance is not only about optimization and speed. It is about contributing to the bottom line of a business.
Another aspect of performance that varies greatly across projects is the execution of the performance strategy, which can vary by organization. Too frequently, the idea of performance testing is equated with adding a test phase, believing that performance risks can be covered by simply running a performance test right before launch. While this is better than assuming performance will come naturally or be taken care of by the support teams, the reality is that to ensure a site will perform, much more than a few tests will be required. A test phase might discover bottlenecks in the application, but the fix might require a solution that goes back to the fundamentals of application design and architecture. Considering performance only at the end of development leads to not being able to address issues on time or having to patch and get by. In the long term, the "band-aid" mode leads to an increasingly complex application that becomes more difficult to manage and optimize. The point of the performance lifecycle is about considering performance in all aspects of the project, not as only a one-time event.
Traditional project plans leave significant performance activities unmanaged for the majority of the project. Performance is considered in the plan as a test phase near the end of the cycle, when there is usually insufficient time and resources to achieve performance objectives because problems discovered so late can be too severe to be corrected before the rapidly approaching deployment date. Furthermore, because development is already finished at this stage, solutions are often provided as afterthoughts or workarounds, which can complicate the application logic for future development. In some circumstances, the root cause of bad performance could be due to system architecture, which would be next to impossible to address this late in the project cycle.
The premise of the performance lifecycle is that performance is considered throughout the entire project lifecycle. Adapting the performance lifecycle approach shifts performance from a point in time activity to a parallel and equally important track alongside a project’s development activities. This raises the visibility of performance throughout the project and its deliverables become key aspects of the project plan. Additionally, achieving pre-defined performance criteria becomes part of the website or application launch approval process. The key performance metrics are continually addressed and tracked against targets.
Figure 1 shows the placement and timing of performance-related activities during the traditional design, implementation, testing, and launch stages of the project lifecycle.
Figure 1. Performance lifecycle at a glance
Activities for each phase of the performance lifecycle include:
- Review and validate non-functinoal requirements with IT and business stakeholders.
- Assist with application flow design.
- Educate developers on performance.
- Review component design focused on performance.
- Prepare system maintenance strategies.
- Identify key performance indicators (KPI) for monitoring
- Coding and unit testing
- Developer-performed application profiling (phase 1 of performance validation).
- Identify initial cache strategy.
- Implement system maintenance strategies.
- Build performance verification environment (including data).
- Finalize performance testing plan.
- Plan priorities and risk mitigation with business stakeholders.
- Instrument code to track KPI.
- Performance testing
- Create scripts and test data, validate environment, and so on.
- Multiple-staged test approach applied.
- Test single system and scale-up.
- Run endurance tests.
- Test availability at end of this phase.
- Plan priorities and risk mitigation with business stakeholders.
- Refine KPI monitoring strategy.
- Launch and post-launch
- Migrate tuned settings to production.
- Monitor and troubleshoot production system.
- Analyze production access logs, advise on script rework.
- Performance test initial round of post-launch fixes.
- Ongoing interlock with business stakeholders to understand future plans.
As your project plan adopts the performance lifecycle, your resource plan also needs to reflect the continued involvement of performance resources. Key performance roles are shown in Figure 2. The performance architect, project manager, and product specialists are often engaged for the entire lifecycle, with additional test specialists and analysts added during the test phase.
Figure 2. Project planning key performance roles
Responsibilities for each of these key roles include:
- Performance architect
- Performance team leader
- Overall leadership in performance aspects of solution architecture and design
- Ownership of performance-related non-functional requirements
- Leads performance verification activities
- Project manager
- Overall coordination of performance engagement activities
- Reports results
- Drives issue resolution
- Links performance with project manager for overall project
- Performance specialist: Middleware performance
- Code profiling analysis and recommendations
- Analyze results, debug and tune middleware applications
- Guidance on performance test design
- Performance specialist: Database
- Performance analysis and recommendations for SQL, database indexes, configuration, and so on
- Performance specialist: Test and script
- Develop and maintain workload, scripts, test data reporting
- Performance specialist: Caching
- Cache design and implementation
Adopting the performance lifecycle requires project, resource, and process changes to give performance a level of visibility that is tracked and accounted for throughout the project. With this level of change, successful implementation also requires a cultural change of the organization’s mindset on performance. Furthermore, as the priority of performance is often lowered in lieu of deliverables and timeline, sponsorship and support must be present from executives and the senior leadership team to make this a reality.
To help you plan for adopting the performance lifecycle into your business, a sampling of actions that apply at various stages of a project are presented below. Incorporating these actions will help you begin transitioning your project culture away from the traditional approach of including performance as a conclusion to the project testing phase, and toward adopting performance considerations at every project stage.
- Establish key performance and business objectives for your site, such as:
- What do your users expect or require when transacting with your site?
- What are the business and transactional objectives for the site in the next 12-24 months?
- Identify requirements that will have a significant impact on capacity or utilization of system resources. For items that could have a negative impact on performance, negotiate to find out if these are truly must-have items, or if similar objectives can be met with a different (and less performance intensive) approach.
- Table 1. Requirements definition
Not very performance friendly Better Display the entire storage catalog in real time on single page. Present catalog data to users in increments as needed.
Risks from not considering performance during requirements definition:
- Introduction of features that could have negative performance impact without much business benefit.
- Add delay during implementation phases to triage and optimize the performance of the feature.
- Open door for a “reactive” approach to performance and capacity considerations that can take resources and time away from other projects, rather than a “proactive” approach.
- Architect a solution that has a good balance of features and performance considerations.
- Design and build performance test environment. Begin to deploy tools that will be required to support your performance strategy.
- Identify architecture limitations that could challenge a non-functional requirement.
- Table 2. Design
Not very performance friendly Better Inventory information is retrieved via a web service request against a geographically remote service provider. Locally cached inventory information is refreshed automatically when remote data changes.
Risks from not considering performance during design:
- Increased possibility of features consuming higher resource utilization than projected over the capacity plan estimate.
- Application performance and user experience can be adversely affected by third party application interfaces.
- Lack of proper test environment can hamper performance testing capabilities and put test results into question.
- Measure the response time of business critical steps in the development environment.
- Profile code execution to identify patterns against best practices.
- Table 3. Coding and unit testing
Not very performance friendly Better Iterate through items in the shopping cart to calculate applicable promotion. Single calculation of applicable promotion against a list if items.
Risks from not considering performance during coding and unit test:
- Potential of discovering inefficient and resource intensive code late in development, which can often jeopardize key milestones.
- Efforts required for identification and triage of performance issues increase the later they are found. Environments become more complex and are typically under increasing amounts of change control as code gets closer to production, which increases the effort and time required to get to resolution.
- Start with simple, common test cases and gradually go to more complex scenarios and environment configurations.
- Measure the performance of the application as load scales up to projected peak.
- Identify and resolve the root causes of resource constraints and application bottlenecks.
- Table 4. Performance testing
Not root cause Better Raised maximum JVM heap size because JVM crashed due to OutOfMemory condition in a duration test over 6 hours. Find a memory leak that was identified by examining the heap dump after the OutOfMemory crash.
Risks from not executing a well defined performance testing methodology:
- Serious business revenue impact as application breaks down under peak load that was not tested prior to launch.
- Increased deployment and system maintenance costs if additional hardware is viewed as the fix for performance issues.
- Very difficult performance triage required in production environment if performance issues are not caught by testing, and instead are directly impacting users.
- Instrument and monitor performance indicators in the production environment to enable the team to proactively identify and address potential issues before users are affected.
- Foster communication between marketing, performance, and operation teams to be better prepared for promotional events.
- Use the data captured from production to optimize planning for future marketing and promotional activities.
- Table 5. Launch and post launch
Not very performance friendly Better Marketing team sent out an email promotion campaign to 2M registered users with a search page URL as landing page. Landing page is a static page, cached on the edge. The performance team also tested the projected increase in traffic for the event.
- Bad user experiences lead to lost customers.
- Capacity below actual demand, which degrades application performance and user experiences.
The above actions are a sampling of ways you can begin to bring performance considerations into your project, as well as risks to projects that do not adopt lifecycle approach to performance.
The performance characteristics of your site are not just a technical consideration, they are also a business consideration. By adopting a performance lifecycle approach into project implementations, you can balance the business objectives of the site with the appropriate technical implementation to withstand the traffic and pressures of your site.
The best examples of adopting the performance lifecycle begin in the requirements gathering phase and continue through monitoring of actual site performance in production. Design phases include reviews and evaluation of implementation best practices such as code optimization, caching opportunities and the latest performance enhancing technologies. Testing phases include not only functional validation but also validation of the user experience under peak load. By executing these steps, problems can be identified earlier in the development cycle which can improve the efficiency of development and help minimize unnecessary costs.
Finally, in a business climate that constantly evolves to create new and exciting features to increase revenue and remain competitive, your team should embrace the performance lifecycle as part of each release plan.
WebSphere Application Server Performance
Information Center: Caching strategy
Information Center: Example methodology for performance testing phase of a WebSphere Commerce implementation
Best practices and case studies: High Performance On Demand Solutions (HiPODS)
Book: Performance Analysis for Java Websites
IBM developerWorks WebSphere
Keri-Anne Lounsbury is a Delivery Manager in IBM Software Group. Keri-Anne has 15 years of experience in the IT industry with a deep focus on the e-commerce space. Her experiences include leading customer and IBM teams in the delivery of complex WebSphere Commerce projects, as well as helping customers resolve critical issues and prepare sites for peak performance.
Kevin Yu is an application performance specialist with fourteen years’ experience in the IT industry. Kevin has worked on application server, enterprise commerce and database products. He has led multi-disciplinary teams to solve critical customer situations and guided many to achieve new records in transaction volumes and sales on their busiest online shopping days of the year.