How to think about performance
Bill Cary 060001Y279 Visits (1978)
Thinking about the US Healthcare.gov website issues, it reminded me that people don’t always know what “performance” means. When we talk about performance we think it means “slow” or, if we think about our car, maybe we think it means “it works or it doesn’t work”. Well, already we have two different definitions of performance and we haven’t really begun to look deep into it. Having 17 years of experience in the performance business, I know that performance means the end user experience or perception. It doesn’t really matter why something is not what we want it to be, it only matters to us that it is not what we want it to be.
Virtualization – Today, with public and private clouds being the handwriting on the wall for the future of computing, virtualization is a key part of maximizing our resources and responding to demand but virtualization can rob up to 22% of the physical servers performance depending on the vendor and version. Using minimum hardware specifications from the vendor and then virtualizing it could mean your net capability is far below the vendors recommendations. Using swapped memory for virtual machines (VMs) is a bad idea in the same way that it is bad for applications on physical machines. Of the entire cost of an application, memory might just be the worst place to cut corners.
Operating Systems – Applications run on hardware, that is true, but to provide a consistent operating environment, all hardware has operating systems (OS) for applications to run in. Today, the most common OS’s are several flavors of Unix, Linux, and Windows. Some database servers provide their own streamlined OS’s to bypass the overhead required by a typical OS. In the pyramid of reliance, if an application must run in an OS, and the OS, must run on hardware and the hardware must reside on the network, the logic is, the network must perform, the hardware must perform, and the OS must perform. Each OS has its own tuning parameters and it is important to the function of the application that the OS be properly tuned. Since every hardware call will ultimately be passed through the OS, if the OS does not perform well, the application won’t either. The database, the application server, and even the clients browser all run on OS’s so don’t underestimate the impact of poorly tuned OS’s.
Database – If everything the application does is driven by the database (DB), it makes sense that if the DB does not perform, the application will not perform. Things can get a little convoluted here. First, the DB itself must be properly tuned and there are literally hundreds of tuning parameters and other factors that can impact the DB server performance. Things like command compiling, allocated memory to various pools, temp space and many more can result in the DB not saving or returning results in reasonable time. And remember, in environments where many thousand requests may be executed every minute, sub-second response time is required. But other aspects of DB performance include well designed commands to the environment and application settings used to connect to the DB. DBs execute commands using structured query language (SQL). Like most things, there are good and bad ways of doing things. Using wildcards to search for things might provide flexibility but may also preclude the use of indexes which can have a negative impact on the return speed. The balance is to use wildcards where they are needed but not to use them for everything. There are many instances where using the wrong approach in a SQL statement can mean the difference between a 1 second or 5 minute result set.
Application Server – This is a term at is often confused. The Application Server is the publishing product often referred to as middleware. WebSphere, WebLogic, JBoss, and Apache are common application servers. Their function varies but generally provides an environment that can publish multiple instances of an application service (JVM) for load balancing, high availability, and complimentary capability. They are typically also responsible for the relationship of the application to the user directory server, Java Messaging Service (JMS), and other common functions. Tuning the application server including ensuring enough application instances exist to support the user load can have an impact on the performance of the application. User login times or problems with login or poorly configured messaging for integrations can impact the users response time.
The Application – Each application will have settings that interact with other components of the product based on how the product will be used. In the case of Maximo, different versions can use different database releases and each of the database releases use different JDBC parameters that may improve the way they respond. Using Maximo configured for MS SQL Server 2000 with MS SQL Server 2005 may have a very negative impact on the response time. This isn’t an application related performance problem; this is related to proper tuning for the environment. Another example might be where Maximo is shipped for maximum flexibility and simplicity in searching. End users need only enter “B456” to find all records with “A123B456C789”, “B456123”, “C789B456”, etc… This is done by using wildcards on the beginning and end of each search like “%B456%” but this means the indexes will not be used in searching for these values. This usually works fine for small to medium-small databases with under 100 users but as the organization requirement grow, this approach is no longer a good one. Imagine loading 10 million rows and searching each for the value “B456”. Databases are fast but not that fast. In larger environments, wildcard searching should be disabled by default and the users taught how to use wildcards when they need them.
The Client – Oddly enough, this is the topic that encouraged me to write today’s blog. Many of us want to focus on the infrastructure of an application. Anytime something goes wrong in a hosted or cloud environment we think there must be something we can do to make the server faster. Don’t forget my original definition of performance “performance means the end user experience or perception”. In a recent critical situation, I was called in to help solve performance problems. We looked at infrastructure 6 ways to Sunday before the critical question came to the table – had anyone directly spoken to the users about what their problems were. After interviewing several silos of users it became apparent that a number of the problems were really at their desktop. Not enough memory on the client, different browsers like all different versions of IE, Firefox, and Chrome reacting differently, users not knowing the best way to do things (training), poorly designed screens with too many fields, too many clicks to accomplish tasks. Remember again “It doesn’t really matter why something is not what we want it to be, it only matters to us that it is not what we want it to be.” And this means everything from end to end of this solution can be reported as a performance problem.