Open-source IoT Platform

Device management, data collection, processing and visualization
for your IoT projects
Learn about Thingsboard

Thursday, February 9, 2017

Load testing of MQTT servers using Java, Maven and Gatling

The rapid growth of IoT market dramatically increased the popularity of MQTT protocol. MQTT is supported by the most popular IoT platforms and is used for data collection, push notifications, real-time messaging and other.

We would like to share our experience with running performance tests against MQTT servers in this article. We are going to concentrate on the tools that we used to generate load on the server. The results of the performance tests for our platform will be published as a separate post.

Gatling

We have chosen Gatling as a framework to run our test due to a number of reasons:

  • Generates beautiful reports out-of-the-box.
  • High performance and low overhead.
  • Easy setup of load scenarios.
  • Scalability of simulations.

Unfortunately Gatling.io framework doesn’t support MQTT protocol out-of-the-box. At the same time Gatling is an open-source framework and we have found an unofficial MQTT plugin.

Gatling MQTT Plugin

Gatling-MQTT plugin was developed by Muneyuki Noguchi and at the moment it is hosted on GitHub under Apache 2.0 License. We have started implementation of Thingsboard performance testing project using Gatling and Gatling-MQTT plugin. Some time later we have realized that the plugin doesn’t support scenarios that we would like to verify and the behaviour of default scenario is not something that we have expected.

The problem with the default scenario of the unofficial Gatling-MQTT plugin is that each time when some data is published, client waits for reply from server and sends MQTT disconnect. So, the message sequence looks like this:

image

This approach is very resource consuming and gives minimal benefits comparing to HTTP or other protocols. This behaviour is normal for HTTP requests-response model, but not for MQTT. The typical MQTT session is maintained for certain time and multiple MQTT publish messages are sent and received between client and MQTT broker. Of course, there are other types of MQTT messages, but they were out of scope for our tests. In our scenario, load testing of the IOT platform must be done in the following way:

image

So once we have connected a device to the IOT platform that acts as an MQTT broker, we will reuse session and publish MQTT messages using the same session. Sure, the session could be recreated at some point, but not every time we would like to publish a message to the server.

In order to support this scenario, we have decided not to implement something new from the scratch, but rather to use Gatling-MQTT plugin as a base and considering fact that this is open-source we are free to modify the software as we wish to meet our needs.

Gatling MQTT Plugin Fork

We have done the fork of Gatling-MQTT plugin, spent some time on investigation how the plugin was constructed, modified it and added Connect, Publish and Disconnect actions as separate steps.

Now we were able to support the expected scenario. The extended version of Gatling-MQTT plugin is located here Extended Gatling-MQTT.

Finally, we were able to implement the scenario that suits our needs. The Gatling simulation below will create separate MQTT session for 10 simulated devices and will send 100 publish messages for each session.

MqttSimulation.scala

class MqttSimulation extends Simulation {
    val mqttConfiguration = mqtt
        // MQTT broker
        .host("tcp://localhost:1883")
    
    val connect = exec(mqtt("connect")
        .connect())
    
    // send 100 publish MQTT messages
    val publish = repeat(100) {
        exec(mqtt("publish")
            // topic: "foo"
            // payload: "Hello"
            // QoS: AT_LEAST_ONCE (1)
            // retain: false
            .publish("foo", "Hello", QoS.AT_LEAST_ONCE, retain = false))
            // 1 seconds pause between sending messages
            .pause(1000 milliseconds)
        }
    
    val disconnect = exec(mqtt("disconnect")
      .disconnect())
    
    val scn = scenario("MQTT Test")
      .exec(connect, publish, disconnect)
    
    setUp(scn
        // linearly connect 10 devices over 1 second 
        // and send 100 publish messages
        .inject(rampUsers(10) over (1 seconds))
    ).protocols(mqttConfiguration)
}

Performance tests project

Our performance test project is hosted on GitHub. It is mainly written in Java and uses Maven as a build tool. Gatling and Gatling-MQTT plugin are written in Scala, and use SBT tool for building sources and running tests. But here, at Thingsboard, we are more Java guys than Scala, that’s why we have implemented custom Java code that connects to the platform, creates a device, warms it up and provides the credentials id string back:

MqttSimulation.scala

// get the device credential ids of the created devices
val deviceCredentialsIds: Array[String] = MqttStressTestTool.createDevices(testParams).asScala.toArray

MqttStressTestTool.java

RestClient restClient = new RestClient(params.getRestApiUrl());
// login to the Thingsboard server
restClient.login(params.getUsername(), params.getPassword());
for (int i = 0; i < params.getDeviceCount(); i++) {
    // create device using REST API
    Device device = restClient.createDevice("Device " + UUID.randomUUID());
    // get credentials from the created device
    DeviceCredentials credentials = restClient.getCredentials(device.getId());
    // store in the array that eventually will be used by Simulation class    
    deviceCredentialsIds.add(credentials.getCredentialsId());
    String[] mqttUrls = params.getMqttUrls();
    String mqttURL = mqttUrls[i % mqttUrls.length];
    MqttStressTestClient client = new MqttStressTestClient(results, mqttURL, credentials.getCredentialsId());
    // connect to the server and do the warm up 
    client.connect().waitForCompletion();
    client.warmUp(data);
    client.disconnect();
}
Thread.sleep(1000);

The list of credential ids is used in MqttSimulation.scala file to do the stress test itself:

// get the device credential ids of the created devices
val deviceCredentialsIds: Array[String] = MqttStressTestTool.createDevices(testParams).asScala.toArray

// provide device credential id as username during the connection phase to the Thingsboard server
val mqttConf = mqtt
    .host(testParams.getMqttUrls.head)
    .userName("${deviceCredentialsId}")

val connect = exec(mqtt("connect").connect())

val publish = repeat(testParams.getPublishTelemetryCount.toInt) {
    exec(mqtt("publish") 
        // publish single message and verify that it was delivered at least once
        .publish("v1/devices/me/telemetry", "{\"temp\":73.2}", QoS.AT_LEAST_ONCE, retain = false))
        .pause(testParams.getPublishTelemetryPause milliseconds)
}

val disconnect = exec(mqtt("disconnect").disconnect())
// create map of device credential ids of the connected devices  
// and use it as a feeder in the scenario
val deviceCredentialsIdsFeeder = deviceCredentialsIds.map( x => {Map("deviceCredentialsId" -> x)})

val scn = scenario("Scenario Name")
    // go over the map and take column deviceCredentialsId as username
    .feed(deviceCredentialsIdsFeeder)
    .exec(connect, publish, disconnect)

setUp(scn
    .inject(rampUsers(deviceCredentialsIds.length) over (1 seconds))
).protocols(mqttConf)

For the guys who prefer Java and Maven there is a maven plugin - gatling-maven-plugin:

<plugin>
    <groupId>io.gatling</groupId>
    <artifactId>gatling-maven-plugin</artifactId>
</plugin>

This plugin finds Simulation files in your project, compiles and runs them. Results will be stored inside of the target folder in a pretty format that you are able to inspect after the run:

image

image

image

To run the tests you simply can type:

mvn clean install gatling:execute

Summary

In general, we have described our approach how we have generated high load on our IoT platform and verified that it provides good results.

We will share the results of Thingsboard IoT platform performance tests in the next post.

Also, we’ll describe code changes, improvements and instances tuning that we have done to achieve processing of more than 1 million MQTT messages per minute. These are basically the first steps for us in the direction of performance testing of the platform, and any feedback regarding this approach is more than welcome.

We hope that the experience provided will help you to verify your solutions against load specifications that you expect to see in the production. Stay tuned by subscribing to our twitter, blog or by starring our project on Github.

Your feedback

If you found this article interesting, please leave your feedback in the comments section, post questions or feature requests on the forum and “star” our project on the github in order to stay tuned for new releases and tutorials.

52 comments :

  1. Excellent post, it has been of great help to me. But perhaps you can help me out with one thing. With the adjusted version created by yourself I have been struggling with using custom feeders that change during every message publication. For example I need to simulate messages that contain the actual date and time of the creation of the message. Any ideas how I can cope with this? I have only been using gatling for one week now so I am still new to it.

    Any help is appreciated

    ReplyDelete
  2. hi Unknown,

    If I got you correctly you are looking for something like this:

    def getMessageBody: io.gatling.core.session.Expression[scala.Predef.String] = {
    val currentTime = System.currentTimeMillis() / 1000
    val result = "{\"tc\":" + currentTime + ", \"temp\":73.2}"
    result
    }

    val publish = repeat(testParams.getPublishTelemetryCount.toInt) {
    exec(mqtt("publish")
    .publish("v1/devices/me/telemetry", getMessageBody, QoS.AT_LEAST_ONCE, retain = false))
    .pause(testParams.getPublishTelemetryPause milliseconds)
    }

    ReplyDelete
  3. That was a great message in my carrier, and It's wonderful commands like mind relaxes with understand words of knowledge by information's Load Test Website

    ReplyDelete
  4. If your life has stopped giving you pleasure due to some unknown reasons than our Escorts Service is the right place it is still not too late to make it exciting. It is just your, who are such buddies, who can make you worth living if you spend a few moments with them. There is no such nudging in availing yourself of their companionship.


    Call Girls Near IGI Airport Delhi
    Escorts Service in Aerocity
    Russian Escorts in Ahmedabad
    Body Massage in Dharuhera
    Body Massage in Bhiwadi
    Massage Parlor in Dharuhera

    ReplyDelete
  5. Hiya very cool web site!! Man ...Superb .. I’ll bookmark your site and take the feeds additionally?
    McAfee is quick and easy to install.
    McAfee programming gives the digital security services to ensure your hardware information.Install mcafee from mcafee.com/activate |
    How to Uninstall or disable McAfee WebAdvisor ?
    How to redeem Redeem McAfee LiveSafe ?
    How to know when my McAfee Subscription Expire ?
    Why do you need McAfee Security ?

    ReplyDelete
  6. This is a smart blog. I mean it. You have so much knowledge about this issue, and so much passion. You also know how to make people rally behind it, obviously from the responses. 한국야동닷컴

    Also feel free to visit may webpage check this link
    국산야동

    ReplyDelete
  7. When I read your article on this topic, the first thought seems profound and difficult. There is also a bulletin board for discussion of articles and photos similar to this topic on my site, but I would like to visit once when I have time to discuss this topic. sòng bạc


    ReplyDelete
  8. It is one of the best information which you are providing. I love your blog because it totally full of informative posts. You are nice one and thanks for sharing. It is very nice and informative blog through which we gain a lot information for our some purpose. you did a great job and i appreciate your effort. Very nice share and keep on sharing. Get best App Developer Dubai service you visit here site for more info.

    ReplyDelete
  9. The setup process for every Canon model is almost similar, however the download through https //ij.start.cannon or http //ij.start.cannon and installation process may differ. All-in-one Canon Inkjet printers are suitable for home, business, school, and others to improve productivity. Depending on your requirement, it offers a type printer including PIXMA, SELPHY, MAXIFY, etc. Some factors need to be in mind while choosing an inkjet printer for you. Later, you can easily set up your Canon printer through drivers from canon.com/ijsetup , wireless connection, USB, and a few components. | ij.start.cannon


    ReplyDelete
  10. Wow, that's what I was looking for, what stuff! Present here on this website, thanks to the admin of this website.. Turkish businessperson visa , You can apply for a Turkish visa online through the e Visa Turkey website. It takes 3-5 minutes to fill in the online application.

    ReplyDelete

  11. This is really an inspiring and helpful article. I am fully satisfied with your effort .There is no need to meet any embassy, especially to apply for Indian visa from Australia. You can easily apply online and easily get your evisa on your email id

    ReplyDelete
  12. I love what you guys do too. Much clever work and reporting! Keep up the good work guys... Turkey visa for Iraq Citizens, Iraq Citizens can be apply online Turkey visa via Turkish visa website.

    ReplyDelete
  13. Good post. I learn something new and challenging on blogs I stumbleupon on a daily basis. 토토사이트

    ReplyDelete
  14. The information which you have provided is very helpful and precise. 바카라사이트닷컴

    ReplyDelete
  15. Absolutely love all your blogs and enjoy reading the story behind each. 파워볼사이트닷컴

    ReplyDelete
  16. Thank you very much for writing this amazing article. I hope you will share more in the future. With the help of the government of India, getting an India tourist visa for US citizens has been easy and convenient done online.

    ReplyDelete
  17. Hey sir, This great article has really piqued my interest. Thank you! How much does an e visa cost for India & how much is an e visa for India? India visa cost depends on your nationality and your visa type.

    ReplyDelete
  18. Thank you.. Get the azerbaijan visas through online e visa application to travel to Azerbaijan. Just follow 3 steps, fill application, upload document and make online payment for Azerbaijan e visa.

    ReplyDelete
  19. Certainly a fantastic piece of work. It has relevant information. Thanks for posting this. Your blog is so interesting and very informative. Thanks sharing. Definitely a great piece of work. Thanks for your work. Get best Freelance Jobs UAE provide good services visit here site for more info from Smart Job.

    ReplyDelete
  20. I trust all the ideas that you have offered for your post. And thank you sir. We have good news for foreign travelers who have been wanting to visit India for a long time. Citizens of 156 countries can plan their travel as per the new visa guideline standards. You can apply now online but firstly do you know India tourist visa covid requirement. You can read tourist visa India covid requirement via our Indian visa blog.

    ReplyDelete
  21. This comment has been removed by the author.

    ReplyDelete
  22. I am impressed by your post. It contains very informative data and i gain a lot information from it. It is very useful for me. Thanks for sharing and keep on sharing that type of posts. It is brilliant blog. I like the way you express information to us. It is very useful for me. You are doing great job and thanks for sharing. Get best SEO Dubai service of marketing you visit here site for more info.

    ReplyDelete
  23. This really goes beyond the comments! I wrote down what I felt while reading the article:)

    스포츠토토
    토토사이트웹
    바카라사이트
    온라인카지노

    ReplyDelete
  24. This comment has been removed by the author.

    ReplyDelete
  25. Great, thank you for this lively article. I hope you will definitely write this type of article for us. and I confirm bookmark your website. e visa to India is one of the most popular and online ways to apply for an Indian Visa without meeting any embassy or consultant.

    ReplyDelete
  26. I have never thought of that. I didn't even dream of it. But with your content, I visualized it in my imagination. This is an excellent event!

    Venom 2 izle

    ReplyDelete
  27. 陸劇線上看 觀眾可以在線觀看台劇、港劇、日劇、韓劇、陸劇、美劇等熱門劇集。 影音視頻播放清晰流暢, 操作界面簡單友好,真正為用戶帶來最舒適順暢的線上觀看體驗 v每天更新海量高清1080P電影免費在線觀看,追劇零時差。內容豐富多元,涵蓋劇集、電影、綜藝、動漫、娛樂等優質影視節目,等熱門 順暢的 视频

    ReplyDelete
  28. 中國人線上看提供熱門韓劇,chinaq, 日劇,陸劇,台劇線上看 台劇線 影,电视剧

    ReplyDelete
  29. 中國人線上看,韓劇, 日劇,陸劇,中國人線上看, 台劇,線上看 漫、娛樂等優質影

    ReplyDelete
  30. 独播库, 独播库,线上看,免费在线观看,电影,电视剧,动漫,视频网站,高清视频,影音視頻播 面簡單友好

    ReplyDelete
  31. Yummy I love lemon ice cream thanks for sharing. This site is full of valuable ideas and significant information which we can share with our family and friends, thanks to the site administrator, you really did a great job. cetak cepat 24 jam jakarta timur it provides good service of design you visit for more details.

    ReplyDelete
  32. Nice informative content. Thanks for sharing this tech post. Keep sharing more tech blogs. DUI Lawyer Arlington VA

    ReplyDelete
  33. Xiaomi Canada has revolutionized the tech market with its innovative and affordable smartphones, gadgets, and smart home devices. Their commitment to delivering top-notch quality and cutting-edge technology is reshaping the Canadian tech landscape. Excited to see what the future holds for Xiaomi in Canada

    ReplyDelete
  34. Your content is both dedicated and brilliant, captivating and enlightening as it showcases a distinct approach to a wide array of subjects. You have become my reliable source for insights and information, nurturing curiosity and promoting learning. I hold great admiration for your exceptional work and eagerly look forward to experiencing more of it!Prevención Violencia Doméstica Acto Nueva Jersey

    ReplyDelete
  35. Sollten bei der Einrichtung Ihres http //fritz.box 7560 login Routers über die Webadresse fritzbox login Probleme auftreten, können Sie sich ganz einfach an unsere erfahrenen Experten wenden.

    ReplyDelete
  36. Load testing MQTT servers using Java, Maven, and Gatling involves simulating a large number of concurrent clients to test the server's performance, scalability, and reliability under various load conditions. This is a great inspiring article. I am pretty much pleased with your good work.You put really very helpful information.
    FLSA Abogado en Virginia

    ReplyDelete
  37. Hello everyone, Navigating international travel can be complex, with e-Visa rejections adding to the challenge. Discover the common reasons behind Turkey e-Visa is Rejected and tips to enhance your application's success.

    ReplyDelete
  38. "ThingsBoard" is an open-source Internet of Things (IoT) platform designed to help users collect, store, and visualize data from IoT devices. Reviews and feedback for ThingsBoard may vary based on users' experiences and specific use cases. Here are some steps to find reviews and information about ThingsBoard.abogado flsa nueva york

    ReplyDelete
  39. Hello! I wanted to express my gratitude for the valuable information shared on your blog. Your dedication to providing in-depth insights is truly commendable. Visa Facilitation Services vfs Saudi arabia streamline visa application processes, ensuring efficiency and convenience for travelers. Explore the benefits and services offered by VFS Saudi Arabia.

    ReplyDelete
  40. The Saudi Arabia visa price can vary depending on various factors, such as the type of visa and the applicant's nationality. For accurate and up-to-date information on Saudi Arabia visa prices, it is advisable to follow our guide Saudi Arabia visa prices.

    ReplyDelete
  41. Hello everyone, How are you all of you ? Navigating the intricacies of Indian visa applications for UK residents has been simplified with the user-friendly interface of www vfsglobal India UK. Discover a smoother and more efficient visa application experience for your journey to India

    ReplyDelete
  42. A thorough manual for testing MQTT servers is available in "Load testing of MQTT servers using Java, Maven, and Gatling". Developer accessibility is ensured by the well-organized and simple-to-follow step-by-step instructions. The combination of Maven and Java demonstrates a powerful method for load testing. The testing procedure is made much more effective by the use of Gatling. Overall, a useful tool for anyone trying to improve the speed of their MQTT server.
    New York Divorce Lawyer Fees

    ReplyDelete
  43. Affordable Divorce Lawyers in Queens New York
    The article discusses load testing of MQTT servers using Java, Maven, and Gatling, a versatile testing framework for IoT applications. It highlights the importance of this approach in ensuring server robustness. The article provides insights into the challenges encountered and unique features of the testing process, providing practical knowledge for developers and engineers in the IoT domain. The article is valuable for understanding the effectiveness of load testing for MQTT servers.

    ReplyDelete
  44. Performing load testing on MQTT servers using Java, Maven, and Gatling allows you to assess the scalability and reliability of your messaging infrastructure, ensuring it can handle real-world demands.
    With this robust combination of tools, you can simulate heavy traffic and analyze how your MQTT server performs under pressure, making it an essential step in optimizing your IoT or messaging systems. ||Monmouth County Reckless Driving Attorney || New Jersey Careless Driving Statute

    ReplyDelete
  45. This tutorial on load testing MQTT servers using Java, Maven, and Gatling offers valuable insights for developers. It provides step-by-step guides and clear code examples, making it accessible to both beginners and professionals. The tutorial addresses a crucial aspect of server performance testing with precision. Best attorney in new york

    ReplyDelete
  46. The article on Load Testing of MQTT Servers using Java, Maven, and Gatling is a valuable resource for developers looking to efficiently assess and optimize their servers' performance. It provides clear step-by-step instructions and empowers readers with the necessary tools and knowledge for effective load testing. The article caters to both beginners and experienced developers, offering a comprehensive tutorial for server performance. The seamless integration of relevant technologies makes it an indispensable reference for enhancing MQTT infrastructure robustness. The article's detailed insights and user-friendly approach simplify the complex process of load testing MQTT servers for developers of all levels.
    Is there No Fault Divorce in New York

    ReplyDelete
  47. "Your comprehensive guide on load testing MQTT servers using Java is a game-changer! The step-by-step instructions coupled with well-explained code snippets make it incredibly easy for both beginners and experienced developers to implement robust load testing for their MQTT infrastructure. The clear and concise explanations not only showcase your expertise but also provide valuable insights into optimizing server performance. Kudos on creating such a valuable resource that undoubtedly empowers the developer community to build and scale resilient MQTT solutions seamlessly. Well done!"

    Domestic Violence Attorney New Jersey

    ReplyDelete
  48. Impressive insights into load testing MQTT servers using Java, Maven, and Gatling! Your innovative approach showcases meticulous attention to detail and a commitment to optimizing IoT platform performance.
    Health Insurance and Divorce in New York

    ReplyDelete
  49. The article provides a detailed guide on load testing of MQTT servers using Java, Maven, and Gatling. It provides step-by-step instructions and code snippets for easy implementation. This robust and efficient approach to testing MQTT servers is crucial for optimizing server performance and ensuring scalability. The article offers valuable insights to master load testing techniques and enhance MQTT server reliability.
    How Much is A Divorce in New York State

    ReplyDelete