 |
||||
|
|
|
||
|
Contents 
Introduction
Mercury Load Runner
IBM Rational Tester
Segue SilkPerformer
NeoLoad
Specifications
How we tested
Editor's choice
About RMIT |
|
||
|
|
||||
|
|
||||
The IBM Rational Software Development Platform uses an Eclipse-based common development environment that is shared by number of developer-, functional-, and performance-testing applications. All of these products have the same UI, called a workbench, and each product adds functionality to the workbench by contributing plug-ins.
IBM was a little more flexible than Mercury giving us a longer licence and 500 users to throw at our application. A company person performed the installation, which was a little painful. We installed the licence server on the same machine we installed the performance tester. It had us puzzled why we couldn't activate the 500-user licence -- we later worked out that we should have installed the licence server on a different server while pointing the performance tester at the licence server.
Creating a script isn't too difficult but it was a little easier using Mercury or Segue. First we created a project, then a performance test, and then we were able to record a performance test. The application connects to the licence server first and opens up a Web browser -- all you have to do is point and click to record your test. Once you have completed the task you can close the browser and the application generates the script code.
There is a tree-like menu in the centre of the workbench, which displays each of the pages we visited and transactions, as well as an element viewer on the right which details all the elements of the page and includes any variables like logins and passwords.
All variables are highlighted but only text is displayed, not the actual HTML page. Right clicking on the highlighted data allows you to add a data pool so you can vary your test data. The next thing to do is add the test to a schedule to represent a workload. We thought this would have been an easy task but had to search around in order to work out how to do it.
We got a little stuck but Rational offers and excellent wizard that can guide you through the steps. There are tutorials and samples built in as well as Web resources that should get you out of any confusion.
During test execution, you can dynamically throw additional users at the test. This is great, especially during ramp up, and saves you from rerunning tests. We didn't find this feature in any of the other applications. The user interface was good. If you're familiar with Eclipse then you won't have any problems getting a feel for this tool. The layout of the controller wasn't too bad, the only concern was that we had to run our screen at a higher resolution to see everything without having to scroll up and down.
By running the performance schedule we started the load test. We could then view the test summary, page performance and throughput, response times, server health, and more by clicking on the tabs to go from one screen to another.
The page performance graph was the most useful during our test, allowing us to monitor which specific page was taking the longest and in our case, since we placed a bottleneck on our Apache server, it was the main welcome page that recorded the longest delays as the apache server would take on more than 20 connections.
You can't drill down further from the real-time graphs like you can with Load Runner, and we preferred the real-time graphs that were produced by Mercury. To look at the reports you view the completed runs and select up to six different reports -- all can be customised. There is what is called -generic counters" which allow you to add overlays to existing graphs. You can filter data so you only display data with the highest or lowest values and more.
If you want to create some pretty reports to give to your boss you can but this product doesn't do it as well as Mercury and Segue. We also found creating a full-blown report takes a quite a while.
|
| ||||||||||||||||||||||||||
I think you missed iMacros in your review. We use it to test our websites. It's great! It has by far the best support for AJAX and Flash applets and the scripting interface is extremely powerful.