Sunday, March 21, 2010

Load Test Whatever You Want With Apache JMeter

This is the second post about load testing with Apache JMeter, read the first post here: A step by step tutorial about load testing relational databases.

JMeter has lots of Samplers. If you need a sampler that is not provided by JMeter you can write your custom sampler. (custom samplers are called "Java Request" in JMeter terminology)
This post will show you, step by step, how to write a JMeter Java Request.

Step 1: Preparing the development environment

Add these two jar files to the java classpath.
  1. $JMETER_HOME/lib/ext/ApacheJMeter_core.jar
  2. $JMETER_HOME/lib/ext/ApacheJMeter_java.jar
(If you are using Eclipse, add these files as external jar files to the java build path.)

Step 2: Extending AbstractJavaSamplerClient

After setting up the classpath, create a custom sampler by extending AbstractJavaSamplerClient and override the following methods.
public Arguments getDefaultParameters() {...}
public void setupTest(JavaSamplerContext context) {...}
public void teardownTest(JavaSamplerContext context) {...}
public SampleResult runTest(JavaSamplerContext context) {...}
getDefaultParameters
Implement getDefaultParameters if you want initial values for test paramters. JMeter will display the parameters in its Java Request configuration GUI. (See the contents of the red rectangle in the picture below.) Here's an example implementation:
public Arguments getDefaultParameters() {
    Arguments defaultParameters = new Arguments();
    defaultParameters.addArgument("memcached_servers", "localhost:11211");
    defaultParameters.addArgument("username", "testuser");
    defaultParameters.addArgument("password", "testpasswd");
    return defaultParameters;
}
setupTest
This is where you read test parameters and initialize your test client. JMeter calls this method only once for each test thread.

teardownTest
Clean up the mess.

runTest
Write your test logic in this method. JMeter will call runTest method for every execution of test threads. Here is a typical runTest implementation:
@Override
public SampleResult runTest(JavaSamplerContext context) {
    SampleResult result = new SampleResult();
    boolean success = true;
    result.sampleStart();
    //
    // Write your test code here.
    //
    result.sampleEnd();
    result.setSuccessful(success);
    return result;
}
The time elapsed betweed result.sampleStart() and result.sampleEnd() is used to calculate average response time of the application under test.

Step 3: Deploy your custom sampler

When you are done create a jar file (containing your custom sampler) in the $JMETER_HOME/lib/ext/ directory. JMeter will display your java request in the java request configuration page.

You can see the results of your test by adding listeners to your test plan. "A step by step tutorial about load testing relational databases" post shows how to add listeners to test plans.

Sunday, March 14, 2010

Load Testing Relational Databases With JMeter

Apache JMeter is a performance testing tool which is entirely written in Java. Any application that works on request/response model can be load tested with JMeter. A relational database is not an exception: receives sql queries, executes them and returns the results of the execution.

I'am going to show you how easy it is to set up test scenarios with the graphical user interface of JMeter. But before diving into details let's give a shot to basic terms:

Test plan : describes a test scenario
Thread Group : represents users running your test scenario.
Samples : a way of sending request and waiting response. HTTP request, JDBC request, SOAP/XML-RPC request and java object request are examples of samples.
Logic Controller : used to customize the logic that JMeter uses to decide when to send requests
Listeners : receives test results and displays reports.
Timers : cause JMeter to delay a certain amount of time before each request that a thread makes.
Assertions : test that application returns expected responses

Note : This post is not meant to be an alternative documentation for JMeter. JMeter has a great documentation. You can find the details in its User's Manual (http://jakarta.apache.org/jmeter/usermanual/index.html
Suppose we have an application that logs every transaction into a relational database. We are going to create a test plan - step by step - in order to answer the questions below.
  • How many transaction records can be inserted to transaction table in a second?
  • How much time does it take to insert a single transaction record to transaction table?
  • How does number of concurrent threads (users) affects the inserts/secs and average response times?
  • How does number of records affects the insert/secs and average response times?

Step 1

Copy mysql jdbc driver into the lib folder of your JMeter installation. JMeter needs a suitable jdbc driver in the classpath to connect to the database.
Example ~/tools/jakarta-jmeter-2.3.4/lib/mysql-connector-java-5.0.5.jar
We are going to store orders of the customers and the result of the order in the transactions table.
CREATE TABLE transactions (
    id INT NOT NULL AUTO_INCREMENT,
    customer_id INT NOT NULL,
    order_id INT NOT NULL,
    result INT,
    PRIMARY KEY (id)
);

Step 2

Create a test plan and name it "Test MYSQL DB". Then add the following jmeter components to the test plan.
  1. Thread group named 'Database Users'
  2. Sampler of type JDBC Request
  3. Config element of type JDBC Connection Configuration
  4. Three config elements of type Random Variable
  5. Listener of type Summary Report
After adding these components JMeter test plan looks like the following picture.

Step 3

Configure database users. The thread group component simulates the database users.
1. Number of users (threads)
2. How many times a user will send request (loop count). If you select 'Forever', threads will run in a while(true) {...} loop until you decide to stop the test.

Step 4

Configure JDBC connection pool. JDBC Connection Configuration component is used to create jdbc connection pools. Database url, jdbc driver, database user and password are configured with this component. Connection pools are identified by "Variable Name". JDBC Samplers (requests) use this variable name (connection pool name) to pop and push connections. I named the test connection pool as "my db pool"

Step 5

Define random variables that will be used in INSERT statements. In this test I am using three random variables : user id, order id and result. Following picture shows the a random number configuration for user id. Random number generator will give us a random integers between 1 and 1000000. We can refer to generated random number with the name user_id.

Step 6

JDBC Request component is the place where we tell our users (threads) what to do. The name of the pool that was configured in Step 3 "my db pool" will be used as the "variable name bound to pool". All threads will execute prepared statements. User id, order id and result will be generated by the random number configurator (described in Step 5)

Step 7

Now we have our threads configured to insert transaction records to the transactions table. In this last step we will add a Listener of type Summary Report in order to view test results.

The results tells us that 10 concurrent users (threads) working in an infinite loop can insert nearly 3300 rows in our transactions table. And the average time spent for inserting a row is 2 ms. You can also choose "Graph Results" listener to view visual representation of the results.
I created and run a simple DB test plan. I hope you'll find this post helpful. Keep this motto in mind
if you can’t measure it, you can neither manage it nor improve it
Happy testing...