Quick load testing with the Camel Dataset component

A pretty common thing I have to do is investigate issues that only occur when load is put on a system. Instead of hand coding message producers to pummel a Camel route or message broker, I've been using the Camel dataset component. I'm not sure many are aware of it but its an incredibly useful and easy to use tool.

For demonstration's sake let say we want to send 10000 messages in a tight loop to a ActiveMQ broker started on the local machine. First we download ActiveMQ

http://activemq.apache.org/activemq-532-release.html

and then start up ActiveMQ

bin/activemq

Now we set up a simple Camel app that uses the Dataset component in a Spring XML file (source available here):

  #1
  <bean id="myDataSet" class="org.apache.camel.component.dataset.SimpleDataSet">
    <property name="size" value="10000"/>
  </bean>

  #2
  <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
    <property name="brokerURL" value="tcp://localhost:61616"/>
  </bean>

  <camelContext xmlns="http://camel.apache.org/schema/spring">
    #3
    <route>
      <from uri="dataset:myDataSet?produceDelay=-1"/>
      <to uri="activemq:myQueue"/>
    </route>

    #4
    <route>
      <from uri="activemq:myQueue"/>
      <to uri="file:/tmp"/>
    </route>
  </camelContext>

So we first create a SimpleDataSet #1 with a size of 10000. This means that 10000 messages will be sent when we use this is the route. We also need to configure the connection to the ActiveMQ broker #2 that we will be pummeling. Finally, we have a Camel route that sends 10000 messages to a queue named myQueue #3. We also have another route that consumes from the same queue and dumps the message contents to the /tmp directory. You can run this now by issuing mvn camel:run on the command line. You will see output something like:

[pache.camel.spring.Main.main()] DefaultCamelContext            INFO  Apache Camel 2.3.0 (CamelContext: camelContext) started in 1155 millis
[et://myDataSet?produceDelay=-1] et://myDataSet?produceDelay=-1 INFO  Sent: 2000 messages so far. Last group took: 2120 millis which is: 943.396 messages per second. average: 943.396
[et://myDataSet?produceDelay=-1] et://myDataSet?produceDelay=-1 INFO  Sent: 4000 messages so far. Last group took: 1867 millis which is: 1,071.237 messages per second. average: 1,003.261
[et://myDataSet?produceDelay=-1] et://myDataSet?produceDelay=-1 INFO  Sent: 6000 messages so far. Last group took: 2014 millis which is: 993.049 messages per second. average: 999.833
[et://myDataSet?produceDelay=-1] et://myDataSet?produceDelay=-1 INFO  Sent: 8000 messages so far. Last group took: 2399 millis which is: 833.681 messages per second. average: 952.381
[et://myDataSet?produceDelay=-1] et://myDataSet?produceDelay=-1 INFO  Sent: 10000 messages so far. Last group took: 1741 millis which is: 1,148.765 messages per second. average: 986.096

So this allowed us to quickly (1) set up a producer which sent 10000 messages, (2) consume those messages from the same queue and (3) report what the throughput was during the test.

The source for this example is available at http://people.apache.org/~janstey/blog_stuff/camel_dataset/dataset-test.zip

For more information on the dataset component, see the official Apache Camel documentation at http://camel.apache.org/dataset.html