MQTT (Message Queue Telemetry Transport)

This article is an introduction to the MQTT protocol. MQTT stands for Message Queuing Telemetry Transport, a simple messaging protocol suitable for communication between IoT devices.

MQTT — Message Queuing Telemetry Transport

MQTT (Message Queuing Telemetry Transport) is a messaging protocol for restricted low-bandwidth networks and extremely high-latency IoT devices. Since Message Queuing Telemetry Transport is specialized for low-bandwidth, high-latency environments, it is an ideal protocol for machine-to-machine (M2M) communication.

MQTT works on the publisher / subscriber principle and is operated via a central broker. This means that the sender and receiver have no direct connection. The data sources report their data via a publish and all recipients with interest in certain messages (“marked by the topic”) get the data delivered because they have registered as subscribers.

In IoT , MQTT is used all the way to connecting cloud environments.

MQTT Basic Concepts

In MQTT there are a few basic concepts that you need to understand:

MQTT — Publish/Subscribe

The first concept is the publish and subscribe system. In a publish and subscribe system, a device can publish a message on a topic, or it can be subscribed to a particular topic to receive messages

  • For example Device 1 publishes on a topic.
  • Device 2 is subscribed to the same topic that device 1 is publishing in.
  • So, device 2 receives the message.

MQTT — Messages

Messages are the information that you want to exchange between your devices. It can be a message like a command or data like sensor readings, for example.

MQTT — Topics

Another important concept is the topics. Topics are the way you register interest for incoming messages or how you specify where you want to publish the message.

Topics are represented with strings separated by a forward slash. Each forward slash indicates a topic level. Here’s an example of how you would create a topic for a lamp in your home office:

Note: topics are case-sensitive, which makes these two topics different:

If you would like to turn on a lamp in your home office using MQTT you can imagine the following scenario:

  1. A device publishes “on” and “off” messages on the home/office/lamp topic.
  2. You have a device that controls a lamp (it can be an ESP32, ESP8266, or any other board or device). The ESP32 that controls your lamp, is subscribed to that same topic: home/office/lamp.
  3. So, when a new message is published on that topic, the ESP32 receives the “on” or “off” messages and turns the lamp on or off.

The device that is publishing the messages can be an ESP32, an ESP8266, or an Home Automation controller platform with MQTT support like Node-RED, Home Assistant, Domoticz, or OpenHAB, for example.

MQTT — Broker

The MQTT broker is responsible for receiving all messages, filtering the messages, deciding who is interested in them, and then publishing the message to all subscribed clients.

There are several brokers you can use. In home automation projects, we use the Mosquitto Broker installed on a Raspberry Pi. You can also install the Mosquitto broker on your PC (which is not as convenient as using a Raspberry Pi board, because you have to keep your computer running all the time to keep the MQTT connection between your devices).

Having the Mosquitto broker installed on a Raspberry Pi on your local network allows you to exchange data between your IoT devices that are also connected to that same network.

MQTT Versions

There are two different variants of MQTT and several versions.

  • MQTT v3.1.0 –
  • MQTT v3.1.1 — In Common Use
  • MQTT v5 — Currently Limited use
  • MQTT-SN — See notes later

The original MQTT which was designed in 1999 and has been in use for many years and is designed for TCP/IP networks.

MQTTv3.1.1 is version in common use.

There is very little difference between v3.10 and 3.1.1. Here is a Github page detailing the main differences

Here is the actual Specification MQTT V3.1 and here is a more detailed overview of the MQTT protocol packet structure,.

The latest MQTT version(v5) ,has now been approved (Jan 2018).. You can download the specification here.

If you are wondering what happened to 4 then see here.

For More Information see MQTT v 5.0 New Features Overview

Here is a Github page detailing the main differences between MQTT v3.1.1 and MQTT v5.

How to use MQTT in Android Application

There are lots of MQTT related library are available in market to integrate

Here we will discuss about Hive MQTT Client library.

the HiveMQ MQTT Client library is supported on Android 4.4 (API level 19) and higher versions. According to Google’s statistics this covers more than 95% of all running Android devices.

Independently of the supported API level, you have to grant your app the permission to use internet communication. Here is how you configure the permission in the AndroidManifest.xml:

<manifest>
...
<uses-permission android:name="android.permission.INTERNET"/>
...
</manifest>

Support for Android API levels 24+

As the HiveMQ MQTT Client uses Java 8 language features, you have to specify the following in the app’s build.gradle file:


android {
...
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
packagingOptions {
exclude 'META-INF/INDEX.LIST'
exclude 'META-INF/io.netty.versions.properties'
}
}

Additionally you have to set some proguard rules in the app’s proguard-rules.pro file:

...
-keepclassmembernames class io.netty.** { *; }
-keepclassmembernames class org.jctools.** { *; }
...

Please make sure that the proguard-rules.pro file is referenced in the app’s build.gradle file:


android {
...
buildTypes {
release {
minifyEnabled true
proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
}
...
}
...
}
...

For complete detail of project work, You can check below link:-

https://blog.esper.io/how-to-build-mqtt-into-your-android-app/

Support for Android API levels 19+

The above configuration is enough if your minimum supported Android API level is at least 24 (7.0 Nougat). Targeting lower Android versions requires additional configuration as Android still lacks support for Java 8 APIs like java.util.Optional, java.util.function.* and java.util.concurrent.CompletableFuture on versions below 24.

You can use the Android RetroFix gradle plugin to backport these APIs automatically. This plugin enables you to use the Java 8 APIs even if you have to support lower Android versions.

The following shows how to configure the Android RetroFix plugin in the app’s build.gradle file. You have to add the plugin and the two backport dependencies android-retrostreams and android-retrofuture:

buildscript {
repositories {
google()
gradlePluginPortal()
}
dependencies {
classpath 'gradle.plugin.com.github.sgtsilvio.gradle:android-retrofix:0.3.4'
}
}

apply plugin: 'com.android.application'
apply plugin: 'com.github.sgtsilvio.gradle.android-retrofix'

...

dependencies {
implementation 'com.hivemq:hivemq-mqtt-client:1.2.2'

implementation 'net.sourceforge.streamsupport:android-retrostreams:1.7.1'
implementation 'net.sourceforge.streamsupport:android-retrofuture:1.7.1'
...
}

When you increase the Android API level to 24+ in the future, you will only need to remove the plugin and the backport dependencies. You do not have to change your code. You can check Hive demo with link and other you can check from some below links:-

https://people.utm.my/shaharil/mqtt-android-studio/

https://www.hivemq.com/blog/mqtt-client-library-enyclopedia-paho-android-service/

https://www.alibabacloud.com/help/en/iot-platform/latest/use-the-paho-mqtt-android-client

What are the benefits of using MQTT?

The lightweight properties and minimum overhead of the MQTT protocol architecture help ensure smooth data transfer with low bandwidth and reduce the load on the CPU and RAM. Among MQTT’s advantages over competing protocols are the following:

  • efficient data transmission and quick to implement, due to its being a lightweight protocol;
  • low network usage, due to minimized data packets;
  • efficient distribution of data;
  • successful implementation of remote sensing and control;
  • fast, efficient message delivery;
  • uses small amounts of power, which is good for the connected devices; and
  • optimizes network bandwidth.

What are the drawbacks of MQTT?

Potential downsides to MQTT include the following:

  • MQTT has slower transmit cycles compared to Constrained Application Protocol (CoAP).
  • MQTT’s resource discovery works on flexible topic subscription, whereas CoAP uses a stable resource discovery system.
  • MQTT is unencrypted. Instead, it uses TLS/SSL (Transport Layer Security/Secure Sockets Layer) for security encryption.
  • It is difficult to create a globally scalable MQTT network.
  • Other MQTT challenges relate to security, interoperability and authentication.

Because the MQTT protocol was not designed with security in mind, the protocol has traditionally been used in secure back-end networks for application-specific purposes. MQTT’s topic structure can easily form a huge tree, and there is no clear way to divide a tree into smaller logical domains that can be federated. This makes it difficult to create a globally scalable MQTT network because, as the size of the topic tree grows, the complexity increases.

Another negative aspect of MQTT is its lack of interoperability. Because message payloads are binary, with no information as to how they are encoded, problems can arise — especially in open architectures where different applications from different manufacturers are supposed to work seamlessly with each other.

As touched upon previously, MQTT has minimal authentication features built into the protocol. Usernames and passwords are sent in cleartext, and any form of secure use of MQTT must employ SSL/TLS, which, unfortunately, is not a lightweight protocol.

Authenticating clients with client-side certificates is not a simple process, and there is no way in MQTT to control who owns a topic and who can publish information on it, except using proprietary, out-of-band means. This makes it easy to inject harmful messages into the network, either willfully or by mistake.

Furthermore, there is no way for the message receiver to know who sent the original message unless that information is contained in the actual message. Security features that must be implemented on top of MQTT in a proprietary fashion increase the code footprint and make implementations more difficult.

MQTT protocol applications and use cases

Due to its lightweight properties MQTT works well for applications involving remote monitoring, including the following:

  • synchronization of sensors, such as fire detectors or motion sensors for theft detection, to determine if a hazard is valid;
  • monitoring health parameters using sensors for patients leaving a hospital; and
  • sensors alerting people of danger.

Another application is a text-based messaging application for real-time communication that capitalizes on MQTT’s low data and energy usage. For example, Facebook uses MQTT for its Messenger app, not only because the protocol conserves battery power during mobile phone-to-phone messaging, but also because the protocol enables messages to be delivered efficiently in milliseconds, despite inconsistent internet connections across the globe.

Most major cloud services providers, including Amazon Web Services (AWS), Google Cloud, IBM Cloud and Microsoft Azure, support MQTT.

MQTT is well suited to applications using M2M and IoT devices for purposes such as real-time analytics, preventative maintenance and monitoring in environments, including smart homes, healthcare, logistics, industry and manufacturing.

How is MQTT used in IoT?

Because MQTT clients are small, they require minimal resources and thus can be used on small microcontrollers, according to MQTT.org. To optimize network bandwidth, MQTT headers are small. Plus, MQTT “can scale to connect with millions of IoT devices,” according to the organization.

As a result, MQTT is one of the most commonly used protocols in IoT and IIoT infrastructure — for example, for utilities industries to efficiently transmit data between their services and their customers and devices.

Examples of MQTT use in IoT or IIoT instructure include the following:

  • Smart metering. The MQTT protocol can be used to transmit data with guaranteed message delivery to provide accurate meter readings in real time. This helps make billing more accurate.
  • Gathering ambient sensor data. Sensors used in remote environments are often low-power devices, so MQTT is a good fit for IoT sensor buildouts with lower-priority data transmission needs.
  • Machine health data. Ably, which provides a pub/sub messaging platform, gives the example of a wind turbine requiring “guaranteed delivery of machine health data to local teams even before that information hits a data center.”
  • Billing systems. MQTT helps eliminate duplicate or lost message packets in billing or invoicing.

Whether used for smart metering, fast outage response or other IoT and IIoT applications, MQTT enables resource-constrained IoT devices to send, or publish, information about a specific topic to a server that acts as an MQTT message broker. The broker then pushes the information out to those clients that have subscribed previously to the topic.

To a human, a topic looks like a hierarchical file path. Clients can subscribe to a specific level of a topic’s hierarchy or use a wildcard character to subscribe to multiple levels. The client could be an IoT sensor in the field or an application in a data center that processes IoT data.

As examples, the Carriots, Evrythng and ThingWorx IoT platforms support the MQTT protocol.

Competing protocols

Other transfer protocols that compete with MQTT include the following:

  • Constrained Application Protocol (CoAP). Well suited for IoT, it uses a request/response communication pattern.
  • Advanced Message Queuing Protocol (AMQP). Like MQTT, it uses a publish/subscribe communication pattern.
  • Simple/Streaming Text Oriented Messaging Protocol (STOMP). It is a text-based protocol. However, STOMP does not deal with queues and topics; it uses a send semantic with a destination string.
  • Mosquitto. It is an open source MQTT broker.
  • Simple Media Control Protocol (SMCP). A CoAP stack that is used in embedded environments, is C-based.
  • SSI (Simple Sensor Interface). This is a communications protocol for data transfer between a combination of computers and sensors.
  • Data Distribution Service (DDS). For real-time systems, it is a middleware standard that can directly publish or subscribe communications in real time in embedded systems.

Quality of service levels

QoS refers to an agreement between the sender of a message and the message’s recipient. It acts as a key feature in MQTT, giving the client the ability to choose between three levels of service.

The three different QoS levels determine how the content is managed by the MQTT protocol. Although higher levels of QoS are more reliable, they have more latency and bandwidth requirements, so subscribing clients can specify the highest QoS level they would like to receive.

The simplest QoS level is unacknowledged service. This QoS level uses a PUBLISH packet sequence; the publisher sends a message to the broker one time, and the broker passes the message to subscribers one time. There is no mechanism in place to make sure the message has been received correctly, and the broker does not save the message. This QoS level may also be referred to as at most once, QoS0 or fire and forget.

The second QoS level is acknowledged service. This QoS level uses a PUBLISH/PUBACK packet sequence between the publisher and its broker, as well as between the broker and subscribers. An acknowledgment packet verifies that content has been received, and a retry mechanism will send the original content again if an acknowledgment is not received in a timely manner. This may result in the subscriber receiving multiple copies of the same message. This QoS level may also be referred to as at least once or QoS1.

The third QoS level is assured service. This QoS level delivers the message with two pairs of packets. The first pair is called PUBLISH/PUBREC, and the second pair is called PUBREL/PUBCOMP. The two pairs ensure that, regardless of the number of retries, the message will only be delivered once. This QoS level may also be referred to as exactly once or QoS2.

In the next part of MQTT we will discuss more .

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Abhishek Srivastava

Abhishek Srivastava

Senior Software Engineer | Android | Java | Kotlin|Xamarin Native Android|Flutter