developerWorks: You're listening to a developerWorks podcast. I'm Scott Laningham. Our guest is Brian Bryson, technology evangelist for IBM Rational Quality Management tools. He joins us to talk about performance testing portal applications. Brian thanks for coming on again.

Bryson: Oh, my pleasure, Scott. My pleasure.

developerWorks: So we're here this time to talk about performance testing of WebSphere® portal applications. But before we dive into that and the specifics of it, could I ask you to just talk about performance testing in general just for a second — the basics. You know, why it's important and how it's done.

Bryson: Sure, yes. We'll begin at the beginning. So without having a formal definition in hand for you, I'd say that basically, performance testing is the process of emulating a large-volume workload on a server and measuring the server's response times. And you do that so that you can gauge the server's ability to handle large volumes of simultaneous requests. Essentially, what you're trying to do is avoid the situation where your Web site crashes due to a high volume of user activity.

developerWorks: Which would be horrible if your idea is to drive traffic and then by doing that you kill it, right?

Bryson: Well, you know, exactly. For some organizations, their Web site is the only way they generate revenue, you know. And for those organizations having the Web site go down, it's like shutting the front door to your business. You know, we all spend so much money just trying to get people to come to our Web sites, but the last thing you want is to have them finally get to the Web site only to find out that's it's really slow or worse, that it's down or it's crashed. I mean, poor performance means lost business. It's as simple as that.

developerWorks: Boy, and I mean what just happened with Skype recently gives and example of how critical it is to some business, doesn't it? I mean that just jumped to mind.

Bryson: Yes, absolutely. That's a perfect example.

developerWorks: Now, given what you just said, it seems kind of ridiculous that performance testing isn't just a given, wouldn't you say?

Bryson: Well, yes. You're right. You know, the risks are high if you're not going to do performance testing. But you know, until recently, performance testing, the process of performance testing, is they're one of those sort of easier said than done activities.

I mean, having done this for so long, I've seen a clear pattern of probably of four cascading problems that really prevent people from performance testing. Well, four plus one, I guess we'll call it. Do we have time for me to run through them for you here?

developerWorks: Absolutely. Let's hear them.

Bryson: OK. So once people accept that they need to performance test — you know and maybe it's a thing like Skype where the whole thing, you know, crashes and they realize that maybe we should have done a little more testing beforehand.

You know, the first thing that you realize is that you need a tool to do the test. You're not going to assemble a thousand people into a room, you know, and get them on a giant con call and then have them all simultaneously type in a URL and hit Enter, you know, on somebody's mark. Not that we haven't seen this but, you know, for most, that's not a reasonable approach. So you go out looking for tools, and the first thing you find out is the tools can be hard to use. You know, you need to understand test scripting and data population techniques and scheduling. And so, it's easier said than done, but it can be done. We've certainly seen it happen.

So, you know, that's kind of your first trouble, but you can get around that. You can clear that hurdle. And you figure out how to use the tools. The next problem you'll encounter is that your system — what you're actually testing is probably not as simple as you thought. Or it's certainly not as simple as it appears to the user. You know, there's security issues, and there's data referencing issues typically masked from your front-end user. Figuring out essentially how to make sure your script actually can emulate your load, be it 100 users or 1,000 or 5,000 or 50,000 users can be really, really tricky.

developerWorks: OK — so I understand the difficult nature of it, and that certainly makes sense. What about some examples of what you're talking about now?

Bryson: Sure. So let's say you're testing an online stock application. And you know, maybe your test is to log on and place your order to buy 100 shares of IBM, for argument's sake.

developerWorks: OK.

Bryson: You know — simple enough from an end-user perspective, but behind the scenes, it's probably a lot trickier or more complicated than you'd see. Likely, when you log on, unbeknownst to you, your browser is assigned some type of security token or security ID, which it will have to send back to the server with every subsequent request — sort of in order to confirm your identity to the server.

Well, our testing methodology — our scripts — work by recording data. So when you go record yourself logging in, your script will actually capture the ID that was generated during the recording session, which is fine until you go to play back that test the next day and your server rejects your subsequent request because you've got a stale security ID. So the solution — it's not a difficult solution, but you need to go into your script and modify it so that it captures the security ID generated from the server, sticks in a variable and then uses that ID for every subsequent request. That is what we call data referencing, or some people call it data correlation.

And the same thing when you buy your stock. The price is always going to be changing. So your script can't hard-code in or simply record the stock price as it was when you created the script. So the bottom line is anytime that you have variable data, visible data like a stock price or invisible data like a security token or a security ID, you need to modify your script to use that data and not to hard-code in the value that was originally recorded. Does that make sense?

developerWorks: Yes. But again, this is why you're doing the talking and I'm asking the questions. That's what I think of in between.

Bryson: Well, like I said, this is easier said than done. But none of these are insurmountable problems. You know, I mean, you can figure out the tool, and there's certainly enough people in your organization that you can figure out how to run your test. And then your test executes, and you get a bunch of reports and you know, bam — that's your next problem. Sometimes you need a Ph.D. in statistics to figure out these reports. A thousand users generates a lot of data, right? So it's tough to interpret. But again, not insurmountable. And if you figure that out how to read those reports, you can find your bottleneck, and you can rush back to the developers and say, "Here is your problem," and I will absolutely guarantee you that the next question becomes ....

developerWorks: "Why did it happen?" right?

Bryson: Exactly. "What's causing the bottleneck? Why are we having this problem? Is it hardware? Did we run out of memory? Did we run out of CPU? Or is it bad software? Is it just poorly performing code?" So, you know, the bottom line is it's tough. You need to figure out the tools, you need to figure out your system, you need to be able to read the reports and interpret them. And if you're lucky enough to find a problem, and you will, you need to be bit of a detective to be able to answer the all-important question of "Why?"

And all of this — you know, remember I said four-plus-one problems. The plus one is the time pressure. You know, the plus one is that most people don't leave enough time to do the performance testing or, if they're smart enough to schedule in a couple good weeks to do the performance testing, they don't leave enough time to fix the problems found by performance testing. So that's why people don't do it.

developerWorks: Now Brian — it sounds like the stuff you're talking about is some of it's a general performance-testing type of thing, but where does the portal angle of this come in? Is that like a whole other level of challenge?

Bryson: Well, the thing with portal is everything I said certainly applies but with portal as soon as you start working in a portal environment the degree of difficulty goes off the charts. So I mean, if you're building portal apps, you're far beyond your Hello World-type applications. You're in the top tier of high-end maximum scalability architectures. But that efficiency — that portal efficiency and scalability — is really achieved at the expense of the performance tester.

The architecture implied or used has some very complicated data referencing. In fact, that same Web page that from a user perspective may often have a unique URL for every visitor — the page looks the same, but it's a different URL. Now that's all part of portal's hyper-efficient caching mechanism, which is great. It means faster response times, especially when you have dynamic data involved. But it means more variability and more variability means more complex tasks.

developerWorks: Yes, I see.

Bryson: Now and that's on the technical side. On the business side, you know, the nature of portal apps is that they're high-visibility. T they aggregate data from across the organization. So if a portal app goes down, everyone looks bad, and it's always the portal's fault. So the thing with performance-testing portals that it's harder technically and it's much riskier from a business perspective.

developerWorks: Now this is where Rational comes into the picture, right?

Bryson: Right. So Rational Performance Tester is our tool for performance testing. But you know, Scott, it's not our first tool. We've been doing performance testing since the early '90s, and I've actually gone through several generations of technology. This might actually be our fourth, maybe fifth generation. It depends on how you account for these things. So with everything we know, all the knowledge we've accumulated over the years, we've really made every effort to make this the easiest and fastest to use performance testing tool.

So a couple of things were done. First off, and probably the most important, is there's no scripting with the tool. All the tasks that you would have had to use scripting or quoting for in the past are now right-click activities or wizard-driven things. We have simplified every aspect of test creation. You don't need to be a Ph.D. in engineering to actually create a test anymore. And you know the biggest challenge, of course, is what I was talking about before, it's the data referencing the data correlation or the variability.

So for data referencing, we've built a lot of intelligence into the tool so that it can do the data referencing for you. So logging on and getting a security ID, Performance Tester will see where that ID is generated and dynamically reference the point where the ID comes in, instead of hard-coding the actual security ID, the value itself. The complex portal URLs. When it's generating these different URLs for every visitor, no problem. We data reference those automatically. And you know that cuts your performance testing script creation time by easily, easily 80 percent. And that's on the front end.

So on the back end our reports are very simple and very clear. In fact, I'm confident that with no training, I can show you a performance test report and you could easily spot the bottleneck. These are very simple things. And then, if you did, we have included the ability to answer the why question. We've got this feature called root-cause analysis that lets you right-click on your reports, to point to the bottleneck and right-click and drill down to determine if the problem was hardware- or software-related or what was the cause.

So if it's hardware, we're monitoring during playback all your system resources from your network utilization to disk and memory to CPU. We're monitoring all that, and we'll correlate it for you on a graph. For software, we will tell you exactly how long it takes to execute every method on every class and even on every tier or your application, and probably more importantly — and this gets a little bit technical — but we will also general a UML sequence diagram, which will show you the sequence of methods that were called to get you right before your bottleneck. So you find your problem and, you know, the question becomes "Why? Why do we have this bottleneck?" Well, you can drill down to that and then you can say, "Well, what happened just before it?" And you can step yourself backwards on this sequence diagram to trace the path to your performance problem.

developerWorks: Wow — that's great. I mean, when you hear all that, the reasons for not testing evaporate pretty quickly with Rational Performance Tester.

Bryson: Yes, yes. I mean we've really — you know, it's a known problem and something that we worked really hard to address. And we've obviously done some optimization here for the portal environment. You know, we fully understand the dynamic URLs. But the way we've done this is we put such a, just a general level of intelligence in the tool that the same technology works in general for, obviously Portal because we've done our optimizations there, but in general for all Web sites, as well as SAP and Oracle and Citrix Systems. There really is no more excuses. You know, the time to test, as we say, is now.

developerWorks: Very nice. Now, how then would someone go about getting started with performance testing portal apps? Where should they go? What should they do?

Bryson: You know, that's always a tough one because it kind of depends on your level of comfort with the process of the tools. I guess we'll begin at the beginning again. The full path for the complete novice would probably start just at our ibm.com Web site, and the reason I point you there is obviously, there's just product information. But what I would recommend is you go check out a demo video of the tool in action. So we have on the Web site a couple three- and four-minute videos that will actually show you, not at a very high level, the process of performance testing.

Now from there, and you can jump in at any point on these, but from there a good idea would be to head over to developerWorks obviously and run through our hosted tutorial. And what's cool about the hosted tutorial is you log on to one of our systems where the product is already installed, as well as a sample app. And so you're given a doc to follow, and the document has step-by-step instructions on how to run your first test. And you simply step one, click here; step two, do this; step three, do that. And you're guided through your first performance test with no installation. The tool is installed, and the app is installed, and then you just go.

developerWorks: And that's in the Rational zone on developerWorks?

Bryson: That's in the Rational zone on developerWorks, exactly. We call it the hosted tutorial for Rational Performance Tester.

developerWorks: OK.

Bryson: And then from there, once you see the video and you actually try it on our system, I guess probably the next thing would be to try it on your system. So, back to developerWorks, you can download the trial of the software, which is a 30-day fully functioning trial, and you can go and run your first performance test on your system with our tool.

Now this is something — I don't know if it actually says this on developerWorks. We can probably go check, but it's something you can do on your own or if you want, you can give us a call and we'll send out one of the field engineers, and they will actually go to your office and help you with the setup and configuration of running your first performance test.

So there's kind of a sequence flow and then supporting you, you know, all the developerWorks stuff. There's articles on developerWorks, as well as probably the last thing I'd mention is the user forum, where you can ask questions and get answers from fellow users and there's actually the product engineers are on there and just general product experts are there. So that's how I'd get going with Rational Performance Tester.

developerWorks: Great information. Brian Bryson, technology evangelist for IBM Rational Quality Management tools. Brian, thanks for your time and expertise. Always good talking to you.

Bryson: Thank you Scott. Always my pleasure. Look forward to the next one.

developerWorks: Check the show notes for this podcast for helpful links related to this topic. You'll also find a transcript of the interview. Find that at ibm.com/developerWorks/podcast. I'm Scott Laningham. Thanks for listening.

Resources

Learn

• Rational Performance Tester
  
  
• (Tutorial) "Hello world: Learn how to discover and analyze performance issues using Rational Performance Tester"
  
  
• Podcast blog
  
  

Get products and technologies

• "Rational Performance Tester trial download"
  
  

About the author

Scott Laningham, host of developerWorks podcasts, was previously editor of developerWorks newsletters. Prior to IBM, he was an award-winning reporter and director for news programming featured on Public Radio International, a freelance writer for the American Communications Foundation and CBS Radio, and a songwriter/musician.

Rate this article

Comments

Back to top

Table of contents

• Resources
• About the author
• Comments

Next steps from IBM

• Try: Rational Performance Tester
• Community: Forum: Rational Performance Testing
• Buy: Rational Performance Tester