are some tools I used from my previous experience:
These two kind of tools
actually run during the unit test area. But, for my team we used the Junit
(actually created a framework) as our QA’s automation testing framework. Since
we are a web service team main workflow is that: call one API and get the result
We may choose some of the automation tools for long time finally put coin on
the JUNIT. Obviously, the code to code framework is the best choice for use.
I first set the Junit/TestNG
as the first tips for the automation tools is that they are really get deep
into the app’s code. No encapsulation, 100% pure code. Driving unit codes as
automation testing is much fun than from the web layer. J
The two tools I found
two years ago, and now everybody are talking them. I prefer selenium than water
because of its great thought that is all the automation codes will run on
different browsers and will show the same result. Compatibility testing is one
of most difficult testing I have met.
to watir, web automation testing in ruby, is actually I used for in my first
project. Long time…
I believe they will
replace the position where QTP now in the web automation area. First of all,
and most important, is they can be programmed in the OO language. Any
experienced DEV or QA can create fancy automation codes with watir/selenium.
Second, they are free, and will be forever. :-)
If you are in the web
services, mostly you will use this tool for your daily work. It’s a java swing
based GUI tools and can be improved by Groovy. And it contains test suit, test
case component like Junit. It's not free actually.
Another web based
automation testing tools.
Adobe has some
automation library created in their flex's source library. The flex monkey
implemented these codes and made itself easy to be used.
above are the tools I can tell u. Automation tools will be the least important
things while you are building your automation codes or frameworks. Good design,
deep understands your business, knows what you can get from automation, and is
the big problem you need to think about.
rather a simple post. Just because automation really depends on the person who
drives them, not the tools!
Performance tuning is science, not art. We
may get the result like "90% line response time is 4s under 100 concurrent
uses’ load" but don't know why. We need to separate the performance result
into serial fields and "look deep into it".
1. Sql Profiling Tools
Above all, SQL profiling tool is one of the
most important performance tuning tools for me. First, we can find the long
duration sql or the single "sql" for each
Reads is too high. Or the sql sequence is
not right (Basically, recommended sequence is "select, insert, update,
delete". This may occur to the db lock.). Second, we also should change
the index (index seek get the best performance for sql server).
For sql server 2005 or 2008, their profiling
tools have integrated into the installation.
There are many other third part tool can
analyze the sql profiling result, such as "jamon". Before using any
of them, one thing should remember is IMPACT, which on the both third part tool
side and your application side.
2. App Profiling Tools
In fact, I can't find a reliable app
profiling tool or framework so far. I have been using "Perf4J" for a
"Perf4j" is a java based app
profiling framework which combined with lots of logging framework such as
log4j, java.common.logging. The concern from my side is that "perf4j"
has done a lot of read time grouping job which may attracted people to use it
(I guess). But this may impact the actual result for the performance itself.
So, be careful using it (especially on the Production env). I use
"System.nanoTime()" to simple the app profiling code and implemented
it into a class.
"Dyntrace" is another strong app
My suggestion is that it is better to write
your own app profiling framework for your use only. We may need more detail
information to indicate the performance issues.
3. Tradition performance tools
I prefer the Jmeter better because of I am
in a web service's team, only need to invoke the API and get response result
correctly. Jmeter is small, easy to understand tool that making performance
more effectible and quickly. This is really important for every performance
engineer duration their daily work. For me, just need to get the response time
as soon as possible and start to turn..
LR is a heavy light and integration tools
but may lead the performance job into different ways. Just as I think
automation are NOT record and play back, we do not spend much time on the
performance scripts maintains.
4. System Monitor Tools
1). jconsole, visualVM (Available in >1.6
These two tools are the big gift we get from
JDK (>1.6). Both are very good at monitoring remote app server's condition.
They will show u the real time jvm's used heap memory, CPU, thread. Usually, I
will find the following points during the performance process:
Thread dump: to indicate the Blocked thread
or most frequently runnable threads
Memory dump: when the memory leak happened,
we need to get the memory dump (jvm snapshot) to find the root cause. I use
"Eclipse Memory Analyze" to analyzer the dump file (*.hprof). Mostly,
the root cause is on the biggest object. But, recently, I find the memory leak
due to too many "small" objects remaining in the jvm and never get
released even a full gc.
The gc log can shows us "warn hit"
before memory leak happened. And gcViewer is a visualize tool to show the gc
log result. Of course, we need more knowledge of how the java garbage
collection's working mechanism. More important is, we need to choose a better
gc way for our production to improve the performance.
3). System real time monitor Tools
We may seldom realize that the important of
monitoring the app server's real time system condition (e.g. CPU, Memory used,
Hard Disk, I/O, Reads,etc). Well, I'm not taking that so important but we may
need this data from the production env. And here are two of tools:
CACI: it's a big heavy tools and not easy to
maintain. Actually, it's a BIG web application, I mean we need install
"python", "mysql" that kind of things then making it
run.I just saw my colleges has set it
run on her performance env and did not get families with that. Much concerns
from my side. One of its main functions shows grouping data the system related
data (CPU,memory, I/O) as a chart. This means in a certain time, it may consume
some app server resource to handle it. The other thing is this tool does not
easy to maintain or deploy even following its guideline.
Hyperic: our company decides to deploy it on
our production side as our monitor tools. Importunely, I did not have time to
take a look at but will, of course, have a try during my performance process.
You may have a try by yourself and tell me how your feeling is. :-)
5. Open Source Web Performance Tools
Two recommended to you if you are in the web
application project. "YSlow" and "HttpWatch". "YSlow"
is the Firefox plug-in and will show up detail response time while browsing the
web side,developed by Steve Sandler. And "HttpWatch" is developed in
Ruby and its main functionality is show each paging loading time. We’ve
integrated it in WATIR and Google Chart. Cool, isn’t?
This is really high level rate performance
tips. Each tip will be a large topic. Performance is science. Every result has
its reason. Group the as more information you can, and split the performance
steps into different level. You may make the simple things simple.