Using the Apache Benchmark Utility
The ab program is provided with the Apache webserver to benchmark a webserver. Since Watson™ Explorer Engine typically runs as a CGI executable within a webserver, it is useful to know the operating performance of the webserver. The ab application is designed to provide information about the performance of your current Apache installation performs, highlighting how many requests per second your Apache installation is capable of serving.
This information is useful for determining the maximum load that your webserver can handle both independent of Watson Explorer Engine and with Watson Explorer Engine running. The ab application can test the webserver while running Watson Explorer Engine so that you can get an accurate benchmark of the performance of the entire application from an end-user perspective.
The ab application is installed as part of the Apache installation package. In many Linux distributions, this application is included in the httpd-tools package. Refer to the Apache documentation for your operating system for information on how to obtain the ab utility.
The following is the help information that is provided by ab:
# ab -h
Usage: ab [options] [http[s]://]hostname[:port]/path
Options are:
-n requests Number of requests to perform
-c concurrency Number of multiple requests to make
-t timelimit Seconds to max. wait for responses
-b windowsize Size of TCP send/receive buffer, in bytes
-p postfile File containing data to POST. Remember also to set -T
-u putfile File containing data to PUT. Remember also to set -T
-T content-type Content-type header for POSTing, eg.
'application/x-www-form-urlencoded'
Default is 'text/plain'
-v verbosity How much troubleshooting info to print
-w Print out results in HTML tables
-i Use HEAD instead of GET
-x attributes String to insert as table attributes
-y attributes String to insert as tr attributes
-z attributes String to insert as td or th attributes
-C attribute Add cookie, eg. 'Apache=1234. (repeatable)
-H attribute Add Arbitrary header line, eg. 'Accept-Encoding: gzip'
Inserted after all normal header lines. (repeatable)
-A attribute Add Basic WWW Authentication, the attributes
are a colon separated username and password.
-P attribute Add Basic Proxy Authentication, the attributes
are a colon separated username and password.
-X proxy:port Proxyserver and port number to use
-V Print version number and exit
-k Use HTTP KeepAlive feature
-d Do not show percentiles served table.
-S Do not show confidence estimators and warnings.
-g filename Output collected data to gnuplot format file.
-e filename Output CSV file with percentages served
-r Don't exit on socket receive errors.
-h Display usage information (this message)
-Z ciphersuite Specify SSL/TLS cipher suite (See openssl ciphers)
-f protocol Specify SSL/TLS protocol (SSL2, SSL3, TLS1, or ALL)
An example of how to load test the Watson Explorer Engine installation:
# ab -n 1000 -c 10 http://localhost/velocity/cgi-bin/query-meta
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.15
Server Hostname: localhost
Server Port: 80
Document Path: /velocity/cgi-bin/query-meta
Document Length: 140256 bytes
Concurrency Level: 10
Time taken for tests: 55.911 seconds
Complete requests: 1000
Failed requests: 65
(Connect: 0, Receive: 0, Length: 65, Exceptions: 0)
Write errors: 0
Total transferred: 140402934 bytes
HTML transferred: 140255934 bytes
Requests per second: 17.89 [#/sec] (mean)
Time per request: 559.107 [ms] (mean)
Time per request: 55.911 [ms] (mean, across all concurrent requests)
Transfer rate: 2452.35 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 1
Processing: 366 558 104.9 541 1350
Waiting: 353 528 104.5 511 1320
Total: 366 558 104.9 541 1350
Percentage of the requests served within a certain time (ms)
50% 541
66% 570
75% 593
80% 605
90% 650
95% 701
98% 941
99% 1059
100% 1350 (longest request)
The above test indicates that the server running this instance of Watson Explorer Engine was able to handle 17.89 requests per second. This is with a default configuration running query-meta and searching the example-metadata collection.
Below is the output of a load test for a search using meta-searching and a search collection:
# ab -n 100 -c 10 "http://localhost/velocity/cgi-bin/query-meta?v:project=query-meta&query=news"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient).....done
Server Software: Apache/2.2.15
Server Hostname: localhost
Server Port: 80
Document Path: /velocity/cgi-bin/query-meta?v:project=query-meta&query=news
Document Length: 124894 bytes
Concurrency Level: 10
Time taken for tests: 70.536 seconds
Complete requests: 100
Failed requests: 99
(Connect: 0, Receive: 0, Length: 99, Exceptions: 0)
Write errors: 0
Total transferred: 12478129 bytes
HTML transferred: 12463429 bytes
Requests per second: 1.42 [#/sec] (mean)
Time per request: 7053.553 [ms] (mean)
Time per request: 705.355 [ms] (mean, across all concurrent requests)
Transfer rate: 172.76 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 0
Processing: 2666 6838 1842.3 6547 13812
Waiting: 2650 6822 1842.5 6533 13798
Total: 2666 6838 1842.3 6547 13812
Percentage of the requests served within a certain time (ms)
50% 6547
66% 7243
75% 7676
80% 8149
90% 8995
95% 10025
98% 13689
99% 13812
100% 13812 (longest request)
You can see that the performance difference is significant when meta-searching is included in the query. The first run was against a search collection with 1000 total requests (10 concurrent) which completed in an average of one half of a second. The second run was against a collection and federated source with only 100 total requests (10 concurrent) which completed in an average of 6.5 seconds. After enabling federation, a 10th of the number of requests completed 13 times longer.