The rapid growth of IoT market dramatically increased the popularity of MQTT protocol. MQTT is supported by the most popular IoT platforms and is used for data collection, push notifications, real-time messaging and other.
We would like to share our experience with running performance tests against MQTT servers in this article. We are going to concentrate on the tools that we used to generate load on the server. The results of the performance tests for our platform will be published as a separate post.
Gatling
We have chosen Gatling as a framework to run our test due to a number of reasons:
- Generates beautiful reports out-of-the-box.
- High performance and low overhead.
- Easy setup of load scenarios.
- Scalability of simulations.
Unfortunately Gatling.io framework doesn’t support MQTT protocol out-of-the-box. At the same time Gatling is an open-source framework and we have found an unofficial MQTT plugin.
Gatling MQTT Plugin
Gatling-MQTT plugin was developed by Muneyuki Noguchi and at the moment it is hosted on GitHub under Apache 2.0 License. We have started implementation of Thingsboard performance testing project using Gatling and Gatling-MQTT plugin. Some time later we have realized that the plugin doesn’t support scenarios that we would like to verify and the behaviour of default scenario is not something that we have expected.
The problem with the default scenario of the unofficial Gatling-MQTT plugin is that each time when some data is published, client waits for reply from server and sends MQTT disconnect. So, the message sequence looks like this:
This approach is very resource consuming and gives minimal benefits comparing to HTTP or other protocols. This behaviour is normal for HTTP requests-response model, but not for MQTT. The typical MQTT session is maintained for certain time and multiple MQTT publish messages are sent and received between client and MQTT broker. Of course, there are other types of MQTT messages, but they were out of scope for our tests. In our scenario, load testing of the IOT platform must be done in the following way:
So once we have connected a device to the IOT platform that acts as an MQTT broker, we will reuse session and publish MQTT messages using the same session. Sure, the session could be recreated at some point, but not every time we would like to publish a message to the server.
In order to support this scenario, we have decided not to implement something new from the scratch, but rather to use Gatling-MQTT plugin as a base and considering fact that this is open-source we are free to modify the software as we wish to meet our needs.
Gatling MQTT Plugin Fork
We have done the fork of Gatling-MQTT plugin, spent some time on investigation how the plugin was constructed, modified it and added Connect, Publish and Disconnect actions as separate steps.
Now we were able to support the expected scenario. The extended version of Gatling-MQTT plugin is located here Extended Gatling-MQTT.
Finally, we were able to implement the scenario that suits our needs. The Gatling simulation below will create separate MQTT session for 10 simulated devices and will send 100 publish messages for each session.
MqttSimulation.scala
class MqttSimulation extends Simulation {
val mqttConfiguration = mqtt
// MQTT broker
.host("tcp://localhost:1883")
val connect = exec(mqtt("connect")
.connect())
// send 100 publish MQTT messages
val publish = repeat(100) {
exec(mqtt("publish")
// topic: "foo"
// payload: "Hello"
// QoS: AT_LEAST_ONCE (1)
// retain: false
.publish("foo", "Hello", QoS.AT_LEAST_ONCE, retain = false))
// 1 seconds pause between sending messages
.pause(1000 milliseconds)
}
val disconnect = exec(mqtt("disconnect")
.disconnect())
val scn = scenario("MQTT Test")
.exec(connect, publish, disconnect)
setUp(scn
// linearly connect 10 devices over 1 second
// and send 100 publish messages
.inject(rampUsers(10) over (1 seconds))
).protocols(mqttConfiguration)
}
Performance tests project
Our performance test project is hosted on GitHub. It is mainly written in Java and uses Maven as a build tool. Gatling and Gatling-MQTT plugin are written in Scala, and use SBT tool for building sources and running tests. But here, at Thingsboard, we are more Java guys than Scala, that’s why we have implemented custom Java code that connects to the platform, creates a device, warms it up and provides the credentials id string back:
MqttSimulation.scala
// get the device credential ids of the created devices
val deviceCredentialsIds: Array[String] = MqttStressTestTool.createDevices(testParams).asScala.toArray
MqttStressTestTool.java
RestClient restClient = new RestClient(params.getRestApiUrl());
// login to the Thingsboard server
restClient.login(params.getUsername(), params.getPassword());
for (int i = 0; i < params.getDeviceCount(); i++) {
// create device using REST API
Device device = restClient.createDevice("Device " + UUID.randomUUID());
// get credentials from the created device
DeviceCredentials credentials = restClient.getCredentials(device.getId());
// store in the array that eventually will be used by Simulation class
deviceCredentialsIds.add(credentials.getCredentialsId());
String[] mqttUrls = params.getMqttUrls();
String mqttURL = mqttUrls[i % mqttUrls.length];
MqttStressTestClient client = new MqttStressTestClient(results, mqttURL, credentials.getCredentialsId());
// connect to the server and do the warm up
client.connect().waitForCompletion();
client.warmUp(data);
client.disconnect();
}
Thread.sleep(1000);
The list of credential ids is used in MqttSimulation.scala file to do the stress test itself:
// get the device credential ids of the created devices
val deviceCredentialsIds: Array[String] = MqttStressTestTool.createDevices(testParams).asScala.toArray
// provide device credential id as username during the connection phase to the Thingsboard server
val mqttConf = mqtt
.host(testParams.getMqttUrls.head)
.userName("${deviceCredentialsId}")
val connect = exec(mqtt("connect").connect())
val publish = repeat(testParams.getPublishTelemetryCount.toInt) {
exec(mqtt("publish")
// publish single message and verify that it was delivered at least once
.publish("v1/devices/me/telemetry", "{\"temp\":73.2}", QoS.AT_LEAST_ONCE, retain = false))
.pause(testParams.getPublishTelemetryPause milliseconds)
}
val disconnect = exec(mqtt("disconnect").disconnect())
// create map of device credential ids of the connected devices
// and use it as a feeder in the scenario
val deviceCredentialsIdsFeeder = deviceCredentialsIds.map( x => {Map("deviceCredentialsId" -> x)})
val scn = scenario("Scenario Name")
// go over the map and take column deviceCredentialsId as username
.feed(deviceCredentialsIdsFeeder)
.exec(connect, publish, disconnect)
setUp(scn
.inject(rampUsers(deviceCredentialsIds.length) over (1 seconds))
).protocols(mqttConf)
For the guys who prefer Java and Maven there is a maven plugin - gatling-maven-plugin:
<plugin>
<groupId>io.gatling</groupId>
<artifactId>gatling-maven-plugin</artifactId>
</plugin>
This plugin finds Simulation files in your project, compiles and runs them. Results will be stored inside of the target folder in a pretty format that you are able to inspect after the run:
To run the tests you simply can type:
mvn clean install gatling:execute
Summary
In general, we have described our approach how we have generated high load on our IoT platform and verified that it provides good results.
We will share the results of Thingsboard IoT platform performance tests in the next post.
Also, we’ll describe code changes, improvements and instances tuning that we have done to achieve processing of more than 1 million MQTT messages per minute. These are basically the first steps for us in the direction of performance testing of the platform, and any feedback regarding this approach is more than welcome.
We hope that the experience provided will help you to verify your solutions against load specifications that you expect to see in the production. Stay tuned by subscribing to our twitter, blog or by starring our project on Github.
Excellent post, it has been of great help to me. But perhaps you can help me out with one thing. With the adjusted version created by yourself I have been struggling with using custom feeders that change during every message publication. For example I need to simulate messages that contain the actual date and time of the creation of the message. Any ideas how I can cope with this? I have only been using gatling for one week now so I am still new to it.
ReplyDeleteAny help is appreciated
hi Unknown,
ReplyDeleteIf I got you correctly you are looking for something like this:
def getMessageBody: io.gatling.core.session.Expression[scala.Predef.String] = {
val currentTime = System.currentTimeMillis() / 1000
val result = "{\"tc\":" + currentTime + ", \"temp\":73.2}"
result
}
val publish = repeat(testParams.getPublishTelemetryCount.toInt) {
exec(mqtt("publish")
.publish("v1/devices/me/telemetry", getMessageBody, QoS.AT_LEAST_ONCE, retain = false))
.pause(testParams.getPublishTelemetryPause milliseconds)
}
That was a great message in my carrier, and It's wonderful commands like mind relaxes with understand words of knowledge by information's Load Test Website
ReplyDeleteIf your life has stopped giving you pleasure due to some unknown reasons than our Escorts Service is the right place it is still not too late to make it exciting. It is just your, who are such buddies, who can make you worth living if you spend a few moments with them. There is no such nudging in availing yourself of their companionship.
ReplyDeleteCall Girls Near IGI Airport Delhi
Escorts Service in Aerocity
Russian Escorts in Ahmedabad
Body Massage in Dharuhera
Body Massage in Bhiwadi
Massage Parlor in Dharuhera
The blog was having very informative content and very useful for me. Well done post and keep it up... Thanks for sharing such a Useful info.tableau automation
ReplyDeleteHiya very cool web site!! Man ...Superb .. I’ll bookmark your site and take the feeds additionally?
ReplyDeleteMcAfee is quick and easy to install.
McAfee programming gives the digital security services to ensure your hardware information.Install mcafee from mcafee.com/activate |
How to Uninstall or disable McAfee WebAdvisor ?
How to redeem Redeem McAfee LiveSafe ?
How to know when my McAfee Subscription Expire ?
Why do you need McAfee Security ?
It's really a nice and useful piece of info.
ReplyDeleteSetup your ms office from office.com/setup |
What is the use of Windows 10 media creation tool
How to get microsoft Office 365 for free ?
Do you know you can Install office without account .
How to Create Windows 10 recovery USB
nice
ReplyDeleteI am really impressed with the information you have provided in your post. Looking forward to visiting more.
ReplyDeletemcafee.com/activate | Geek Squad Appointment
hbar coin hangi borsada
ReplyDeletebtcst coin hangi borsada
vet coin hangi borsada
via coin hangi borsada
tron hangi borsada
juventus coin hangi borsada
beta coin hangi borsada
auto coin hangi borsada
mtl coin hangi borsada
Set up Microsoft 365 through microsoft365.com/setup and use Microsoft apps such as Word, PowerPoint, and Excel, online storage, cloud-connected features, and more. If you have a Microsoft account associated with Outlook, OneDrive, Xbox Live, or Skype, you can proceed with Microsoft 365 login
ReplyDelete| microsoft365.com/setup | www.hp.com/go/wirelessprinting
A new purchased Canon printer needs to be set up. First, ensure you’ve installed the latest drivers and software from a valid siteThe wireless and wired models require drivers and software; therefore, each has to undergo one important installation process.
ReplyDeletehttps //ij.start.cannon,
http //ij.start.canon
Visit canon’s official site – ij.start.canon/ts3122 and download appropriate software and drivers one windows PC. Or if you have a CD, install it. ij.start canon is the manufacturer's site to download Canon printer drivers. Install and set up Canon Printer from ij.start.canon and obtain high-quality printing documents straightforwardly.
ReplyDeleteThe setup process for every Canon model is almost similar, however the download through https //ij.start.cannon or http //ij.start.cannon and installation process may differ.
ReplyDeleteYou can easily download the driver according to your model as ij start canon is designed with a user-friendly interface. canonij printer provides an answer for all type of canon printer problems by which you can undoubtedly figure out how to set up from ij.start.canon and improve insight.
ReplyDeleteij.start.canon ,
ij.start canon
This is a smart blog. I mean it. You have so much knowledge about this issue, and so much passion. You also know how to make people rally behind it, obviously from the responses. 한국야동닷컴
ReplyDeleteAlso feel free to visit may webpage check this link
국산야동
I am browsing this website daily and get good facts from here all the time. Aw, this was a really nice post. Kenya welcomes foreigners to the country for various purposes. Obtaining a Kenyan visa is quite convenient and reading about Kenyan visa invitation letter.
ReplyDeleteThanks for sharing this! All the best!
ReplyDeleteWhen I read your article on this topic, the first thought seems profound and difficult. There is also a bulletin board for discussion of articles and photos similar to this topic on my site, but I would like to visit once when I have time to discuss this topic. sòng bạc
ReplyDeleteCanon IJ Network Tool is a toolkit software with the options to keep a check on most of your Canon printer network settings and adjust them according to your requirements. The Canon IJ Network tool will get you through the network settings uninterruptedly. Canon IJ Printer Utility is a complete software package meant to adjust and modify the configurations of your printer device as per the requirement.
ReplyDeleteCanon IJ Network Tool is a toolkit software with the options to keep a check on most of your Canon printer network settings and adjust them according to your requirements.
ReplyDeleteCanon IJ Printer Utility is a complete software package meant to adjust and modify the configurations of your printer device as per the requirement. It is one of the essential utility software offered by Canon to ease your printer configuration alteration using a few clicks.
It is one of the best information which you are providing. I love your blog because it totally full of informative posts. You are nice one and thanks for sharing. It is very nice and informative blog through which we gain a lot information for our some purpose. you did a great job and i appreciate your effort. Very nice share and keep on sharing. Get best App Developer Dubai service you visit here site for more info.
ReplyDeleteThe wireless and wired models require drivers and software; therefore, each has to undergo one important installation process. If your printer model isn’t set up yet, you can get started now.
ReplyDeletehttp //ij.start.canon,
https //ij.start.cannon,
You can easily download the driver according to your model as ij start canon is rinter problems by which you can undoubtedly figure out how to set up from ij.start.canon and improve insight.
ReplyDeleteij.start.canon ,
ij.start canon
The setup process for every Canon model is almost similar, however the download through https //ij.start.cannon or http //ij.start.cannon and installation process may differ. All-in-one Canon Inkjet printers are suitable for home, business, school, and others to improve productivity. Depending on your requirement, it offers a type printer including PIXMA, SELPHY, MAXIFY, etc. Some factors need to be in mind while choosing an inkjet printer for you. Later, you can easily set up your Canon printer through drivers from canon.com/ijsetup , wireless connection, USB, and a few components. | ij.start.cannon
ReplyDeleteWow.. Very informative article thanks for sharing please keep it up.. India visa price is changed in case of emergency. Foreign travelers should check the visa price.
ReplyDeletetiktok jeton hilesi
ReplyDeletetiktok jeton hilesi
referans kimliği nedir
gate güvenilir mi
tiktok jeton hilesi
paribu
btcturk
bitcoin nasıl alınır
yurtdışı kargo
Wow, that's what I was looking for, what stuff! Present here on this website, thanks to the admin of this website.. Turkish businessperson visa , You can apply for a Turkish visa online through the e Visa Turkey website. It takes 3-5 minutes to fill in the online application.
ReplyDeleteDecent data, profitable and phenomenal outline, as offer well done with smart thoughts and ideas, bunches of extraordinary data and motivation, both of which I require, on account of offer such an accommodating data here 토토사이트
ReplyDelete
ReplyDeleteThis is really an inspiring and helpful article. I am fully satisfied with your effort .There is no need to meet any embassy, especially to apply for Indian visa from Australia. You can easily apply online and easily get your evisa on your email id
I love what you guys do too. Much clever work and reporting! Keep up the good work guys... Turkey visa for Iraq Citizens, Iraq Citizens can be apply online Turkey visa via Turkish visa website.
ReplyDeleteIt is my first visit to your blog, and I am very impressed with the articles that you serve. Give adequate knowledge for me. Thank you for sharing useful material. I will be back for the more great post. 먹튀검증사이트 But by chance looking at your post solved my problem! I will leave my blog, so when would you like to visit it?!
ReplyDeleteGood post. I learn something new and challenging on blogs I stumbleupon on a daily basis. 토토사이트
ReplyDeleteThe information which you have provided is very helpful and precise. 바카라사이트닷컴
ReplyDeleteAbsolutely love all your blogs and enjoy reading the story behind each. 파워볼사이트닷컴
ReplyDeleteNice...
ReplyDeleteIt's a very powerful article. I really like this post. Thank you so much for sharing this.
Canon IJ Scan Utility
ij start canon ts3122
Make sure the surface where you are placing the printer is clean and close to Pc or laptop.Check the shipped components with your inkjet printer
ReplyDeletehttp //ij.start.canon,
https //ij.start.cannon
Thank you very much for writing this amazing article. I hope you will share more in the future. With the help of the government of India, getting an India tourist visa for US citizens has been easy and convenient done online.
ReplyDeleteCanon IJ Network Tool is a toolkit software with the options to keep a check on most of your Canon printer network settings and adjust them according to your requirements.
ReplyDeleteCanon IJ Printer Utility is a complete software package meant to adjust and modify the configurations of your printer device as per the requirement. It is one of the essential utility software offered by Canon to ease your printer configuration alteration using a few clicks.
Canon.com/ijsetup/mg2500
is the Canon support link to get you the latest printer drivers and software to set up your PIXMA MG2500 or MG2520.
Visit ij.start canon | ij.start.cannon and find out the best way to download Canon printer drivers. Canon printers are ideal for every situation wherever you need a document, paper, or photo print or even if you wish to scan, fax, and do more.
ReplyDeleteAll-in-one Canon Inkjet printers are suitable for home, business, school, and others to improve productivity. You can easily set up your Canon printer through drivers from Canon.com/ijsetup | canon.come/ijsetup , wireless connection, USB, and a few components.
ij.start canon helps to set up canon printer. It’s the online support platform to download and install canon printer drivers, firmware, and software ij.start.canon , Canon IJ Network Tool
ReplyDeleteinstagram beğeni satın al
ReplyDeleteyurtdışı kargo
seo fiyatları
saç ekimi
dedektör
fantazi iç giyim
sosyal medya yönetimi
farmasi üyelik
mobil ödeme bozdurma
슬롯커뮤니티
ReplyDeletebitcoin nasıl alınır
ReplyDeletetiktok jeton hilesi
youtube abone satın al
gate io güvenilir mi
binance referans kimliği nedir
tiktok takipçi satın al
bitcoin nasıl alınır
mobil ödeme bozdurma
mobil ödeme bozdurma
Hey sir, This great article has really piqued my interest. Thank you! How much does an e visa cost for India & how much is an e visa for India? India visa cost depends on your nationality and your visa type.
ReplyDeleteThank you.. Get the azerbaijan visas through online e visa application to travel to Azerbaijan. Just follow 3 steps, fill application, upload document and make online payment for Azerbaijan e visa.
ReplyDeleteI read your excellent post. After reading this post, i really appreciate your effort and my request is to please share us more post in future. Keep it up. It is one of the best information which you are providing. I love your blog because it totally full of informative posts. You are nice one and thanks for sharing. Get best SEO Company Dubai service of marketing you visit here site for more info.
ReplyDeleteCertainly a fantastic piece of work. It has relevant information. Thanks for posting this. Your blog is so interesting and very informative. Thanks sharing. Definitely a great piece of work. Thanks for your work. Get best Freelance Jobs UAE provide good services visit here site for more info from Smart Job.
ReplyDeleteI trust all the ideas that you have offered for your post. And thank you sir. We have good news for foreign travelers who have been wanting to visit India for a long time. Citizens of 156 countries can plan their travel as per the new visa guideline standards. You can apply now online but firstly do you know India tourist visa covid requirement. You can read tourist visa India covid requirement via our Indian visa blog.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteSmm Panel
ReplyDeleteSMM PANEL
İş İlanları Blog
İnstagram takipçi satın al
Hırdavatçı burada
beyazesyateknikservisi.com.tr
servis
Tiktok para hilesi
I am impressed by your post. It contains very informative data and i gain a lot information from it. It is very useful for me. Thanks for sharing and keep on sharing that type of posts. It is brilliant blog. I like the way you express information to us. It is very useful for me. You are doing great job and thanks for sharing. Get best SEO Dubai service of marketing you visit here site for more info.
ReplyDeleteThis really goes beyond the comments! I wrote down what I felt while reading the article:)
ReplyDelete스포츠토토
토토사이트웹
바카라사이트
온라인카지노
This comment has been removed by the author.
ReplyDeleteGreat, thank you for this lively article. I hope you will definitely write this type of article for us. and I confirm bookmark your website. e visa to India is one of the most popular and online ways to apply for an Indian Visa without meeting any embassy or consultant.
ReplyDeleteI have never thought of that. I didn't even dream of it. But with your content, I visualized it in my imagination. This is an excellent event!
ReplyDeleteVenom 2 izle
陸劇線上看 觀眾可以在線觀看台劇、港劇、日劇、韓劇、陸劇、美劇等熱門劇集。 影音視頻播放清晰流暢, 操作界面簡單友好，真正為用戶帶來最舒適順暢的線上觀看體驗 v每天更新海量高清1080P電影免費在線觀看，追劇零時差。內容豐富多元，涵蓋劇集、電影、綜藝、動漫、娛樂等優質影視節目，等熱門 順暢的 视频
ReplyDelete中國人線上看提供熱門韓劇,chinaq, 日劇,陸劇,台劇線上看 台劇線 影,电视剧
ReplyDelete中國人線上看,韓劇, 日劇,陸劇,中國人線上看, 台劇,線上看 漫、娛樂等優質影
ReplyDelete独播库, 独播库,线上看,免费在线观看,电影,电视剧,动漫,视频网站,高清视频,影音視頻播 面簡單友好
ReplyDeleteYummy I love lemon ice cream thanks for sharing. This site is full of valuable ideas and significant information which we can share with our family and friends, thanks to the site administrator, you really did a great job. cetak cepat 24 jam jakarta timur it provides good service of design you visit for more details.
ReplyDeleteNice informative content. Thanks for sharing this tech post. Keep sharing more tech blogs. DUI Lawyer Arlington VA
ReplyDelete