Tutorial: Connecting Devices to the IoT Platform with Node-RED & MQTT

Nod if you ever had problems with connecting devices to an IoT platform. The thing is, there is no such thing as a uniform interoperability standard for the Internet-of-Things industry, which results in developers having to spend extra time going through tutorials and documentation in order to connect their first device.

However, there are tools available like Node-RED that enable rapid prototyping of IoT applications, and fast design / configuration of real-time applications on end devices. It is no coincidence that one can find almost 6,000 scientific papers discussing the usage of Node-RED in industrial environments, including those from Hitachi.

Node-RED is proven to be an efficient IoT solution mapping option for Industrial IoT (IIoT) use cases involving laboratory management, and most kinds of automation. Additionaly Node-RED is widely used to bridge communication protocols in scenarios involving manufacturing, construct, power management automation, and other domains.

In this article, we will explain how to connect devices to the Kaa IoT platform using Node-RED and MQTT.

To start we will explain the MQTT protocol and Node-RED. If you're already familiar with MQTT and Node-RED concepts, you can skip the explanations of MQTT and Node-Red and go straight to the step-by-step tutorial for connecting IoT devices to the Kaa IoT platform using Node-RED.

What is MQTT: Protocol essentials explained

MQTT is a lightweight messaging protocol ideal for limited-bandwidth devices and high-latency environments. MQTT is designed to transmit the smallest amount of data possible and is the most commonly used protocol for connecting and monitoring IoT devices.

MQTT was created by IBM in 1999 as a lightweight, data agnostic application layer protocol for minimal bandwidth and battery drain. Following the widespread adoption of IoT technology, MQTT became open source and is now standardized by OASIS and ISO/IEC.

The latest release of MQTT is 5.0, which contains significant enhancements over the first public MQTT version, MQTT 3.1.

Major components of MQTT

The MQTT architecture includes 2 types of network entities: MQTT client and MQTT broker.

An MQTT client is a device that runs an MQTT library. It can range in size from a tiny controller to a full-fledged server. Below is a ink to list of available MQTT libraries implemented for different programming languages in the MQTT community wiki.

An MQTT broker is a server responsible for collecting messages sent by clients and distribution of those messages to clients that are subscribed to relevant topics.

MQTT brokers are responsible for the following tasks:

• Authenticate and authorize MQTT clients.
• Process messages on behalf of the clients.
• Send messages to other systems for further analysis.
• Handle missed messages and client sessions

An MQTT broker is software that functions in MQTT networks as the central hub. The broker software can be proprietary (EMQS, VerneMQ, Kaa KPC) or open-source (Mosquitto). There are many reliable and robust MQTT brokers available on the market.

Next let us investigate the MQTT communication process in greater detail to understand how MQTT works.

MQTT architecture explained

MQTT is an application layer protocol. The application layer may be implemented by the operating system (embedded OS) or by the firmware.

MQTT must run over a transport protocol that provides ordered, lossless, bi-directional connections. Typically, TCP/IP is used. The data from this layer is encapsulated into TCP streams where lower-layer protocols handle data transmission.

MQTT's control elements are comprised of binary bytes, with a maximum payload of 256 MB. Each MQTT message consists of the following elements:

• Fixed header.
• Variable header (optional).
• Payload.

There are 14 defined MQTT message types. The most common are CONNECT, PUBLISH and SUBSCRIBE. MQTT features a command and command acknowledgment format. Hence whenever a client delivers a command to the broker, the broker responds with an acknowledgment.

MQTT communication uses a publish/subscribe model. MQTT decouples the message-sending client (the publisher) from the message-receiving client/clients (the subscribers). Thus, clients can be either publishers or subscribers.

Publishers are clients that send data to the broker by publishing to a specific MQTT topic. Subscribers are clients that receive data from the broker by subscribing to a particular topic. Information in MQTT communication is organized in a hierarchy of topics that categorize data and enable clients to filter the data they receive. Each topic level is separated by a forward slash (e.g., sensor1/temperature). This structure allows clients to subscribe to a specific topic (e.g., "sensor1/temperature") or to a broader topic (e.g., sensor1/#) to receive data from all subtopics.

Reliable message delivery is guaranteed via MQTTs Quality of Service (QoS) levels:

• QoS 0 – Once (not guaranteed)
• QoS 1 – At least once (guaranteed)
• QoS 2 – Only once (guaranteed)

QoS levels define the guarantee of message delivery based on its purpose, importance, and network conditions. In such a way, the broker can maintain different QoS levels for each device to maximize efficiency and reliability.

Why use MQTT?

Originally, MQTT was used for oil pipeline telemetry systems communication over satellite. Why is MQTT used in IoT? It’s popular within the Internet-of-Things applications because MQTT can consume minimal bandwidth and deliver a fast response time which is important for many IoT applications.

The MQTT protocol is mostly used by the following domains.

• IIoT, or Industrial IoT (automotive, oil and gas, agriculture, etc.)
• IoT solutions (smart homes, smart cities, etc.)
• Biotelemetry (healthcare, natural science)
• Chat and notification services

What makes the MQTT option so attractive for enthusiasts and large enterprises alike? Let’s talk about the benefits of MQTT, and also, let’s bear in mind its drawbacks.

Using MQTT in IoT: Pros and cons

The MQTT protocol has received widespread acceptance in the IoT domain. Its simple architecture that works on top of the TCP/IP protocol made MQTT a real game changer in an industry with a rising number of small, inexpensive, and low-battery products hitting the market.

On the other hand, MQTT has its limitations. So let’s discuss the pros and cons of MQTT in the field of IoT.

Benefits of using MQTT in IoT

• Minimal overhead. MQTT offers push-based instantaneous delivery that eliminates polling and significantly lowers network traffic.
• Flexible & scalable architecture. MQTT excels at many-to-many communication and reduces complexity due to logical data structurization and processing.
• Strong community. MQTT is an open protocol with sufficient documentation. MQTT offers a set of useful libraries for Python, JavaScript, Java, C/C++, and other programming languages.
• Cost-effective development. The open-source MQTT libraries and public brokers reduce the costs of development and speed up the process.

However, we must admit that there are some drawbacks to MQTT you need to consider.

Drawbacks of MQTT in IoT

According to the Analytics for the Internet of Things (IoT) by O’Reilly, one of the main drawbacks of using MQTT in IoT is that MQTT operates over TCP. The reason is that TCP was designed for devices with more memory and processing power than many of the lightweight and battery-constrained IoT devices.

Also, TCP requires handshaking to set up communication links before any messages can be exchanged. Thus, wake-up and communication times can increase, which can affect long-term battery consumption and add points to the memory requirements for IoT devices if speaking of large IoT networks.

That notwithstanding, the MQTT protocol has a lot of advantages to offer in the field of IoT. MQTT is a lightweight protocol for dynamic many-to-many IoT communication. Its simple architecture, straightforward implementation, and flexibility are crucial for the development of the IoT industry.

So, we have discussed the MQTT protocol with its architecture, application, and primary features. And now is the time for us to discuss another great example of an IoT product with a vivid community – Node-RED. What is Node-RED, and who can benefit from using it? Let’s discuss this.

Node-RED: What is it and how does it work?

Node-RED is an open-source flow-based visual programming development tool created by Nick O’Leary and Dave Conway-Jones from IBM. It was born as a proof-of-concept for visualizing mappings between MQTT topics. Since 2016, the Node-RED open-source project has been a part of the JS Foundation that was merged into the OpenJS Foundation in 2019.

Node-RED facilitates connection in the IoT infrastructure, allowing for easy integration of hardware devices, APIs, and online services. You connect different devices and services by connecting nodes in your desired flow.

Node-RED is built on Node.js, an open-source, cross-platform server environment. Node.js uses an event-driven and non-blocking architecture.

That’s why it’s a platform-agnostic tool that runs smoothly both in the cloud and on Raspberry Pi. The range of different environments suitable for Node-RED installation is impressive – from low-cost edge devices to back-end servers, mobile included.

According to the Node-RED GitHub repository, its purpose is to enable low-code programming for event-driven applications. Organizations may use a low-code approach to leverage increased business agility and cost-effectiveness when building custom IoT apps.

Low-code IoT leverages already implemented application building blocks to help companies focus on the business value, not on hiring more developers. The low-code approach enables rapid Node-RED development and faster time-to-market.

Node-RED features a module-based approach. To specify the order of operations to perform, you need to graphically connect predefined modules. Thanks to this approach, non-programmers can define processes or algorithms. This well-proven concept is also used in many business automation, data analysis, and signal processing tools.

Summing up, the Node-RED tool easily handles connecting IoT devices, APIs, and other IoT elements. Thus, Node-RED makes it easier to connect devices to the IoT platform.

Main components of Node-RED

Basic concepts of Node-RED

Nodes

Node-RED offers an extensive number of pre-built nodes, or basic building blocks for your flows. It allows non-developers to automate processes using GUI while also allowing developers to write their custom nodes with JavaScript or other programming languages.

A node is a piece of code that performs a specific function, such as reading data from a sensor or making an API request. Nodes are connected together to form a Node-RED flow, with the output of one node being passed as input to the next.

In such a way, Node-RED makes it possible to build complex low-code IoT applications by combining simple nodes on a screen.

Here are a handful of the most frequently used Node-RED nodes.

Flows

The flow in Node-RED is a set of nodes wired together. The flow structure helps organize sequences of nodes representing separate logical components of your Node-RED app. In the editor, each flow is displayed as an independent tab.

The flow can have a name, description, and flow-based context. Flows are stored in JSON format, a language-independent data format commonly used for data interchange across platforms.

Messages

In Node-RED, messages are objects passed from one node to the next one it is connected to. Basically, messages contain the data that you want to pass to the next node.

The msg object may contain as many attributes as you want. Properties on the object can be added or modified by a node. Some of those attributes are built-in, but you can also create your own. By default, messages in Node-RED have a payload and _msgid properties.

You can find conventions on the message object in the Node-RED wiki on Github.

Context

The context within Node-RED allows specifying global variables that can be accessed by nodes directly rather than from within the message passed to the node.

Context can be defined at 3 different levels.

• Node level: only the node that sets the value has access to it
• Flow level: any node within the flow has access to the value
• Global: any node within the Node-RED instance has access to the value.

Benefits of using Node-RED in IoT

The IoT market is growing around the world, taking its share in almost any domain. Thus, IoT solutions are now connecting devices, managing tasks, analyzing opportunities, and sending information on the service of almost any industry.

Node-RED is a hands-on tool for developers, solution architects, and other stakeholders. It offers ease of use, setup, and deployment. The flow-based programming simplifies visualizing and controlling the maintenance of workflow data and managing sequences of cause-based events.

Let’s briefly summarize the benefits of Node-RED in the Internet of Things below.

• Easy to get started with the pre-built functionality
• Open-source solution with large community support
• Great for heterogeneous environments and constrained hardware
• It can be executed on any device where you can install NodeJS

Node-RED simplifies code development by providing a user with a set of pre-built, easily connectable nodes to configure your data communication and functions from a sensor to its application for visualization and understanding.

Interested in integrating Node-RED into your network? Then dive into the example of Node-RED implementation in IoT below.

How to connect devices to Kaa IoT platform using Node-RED?

In this part, we will show how to connect an IoT device to Node-RED via MQTT and visualize data from it in the Kaa platform.

We will be using a Raspberry Pi with Node-RED installed. Our device will report temperature, humidity, and air pressure from a BME280 sensor. Thanks to the Kaa Web Dashboard, we will create a digital twin of our Raspberry Pi and view telemetry data.

In order to visualize the data from your device in a convenient way, we will be using the Kaa IoT Cloud. It offers hassle-free IoT device management, tracking, and analytics with no IoT development expertise required. You can get a free test drive of the Kaa IoT platform.

What are the 3 steps to connect devices to the IoT platform using Node-RED and Kaa? Let’s dive into the tutorial.

Step 1. Connecting your device

First, we will define our Raspberry Pi in the Kaa Cloud by adding a new device. In the Device Management dashboard in your Kaa Cloud account, we need to specify a token that will help us identify our Raspberry Pi later.

After that, we open the added device page in the Device Management dashboard and copy the version name of our app.

Later, we will need the app version and the token to connect our Raspberry Pi to the cloud.

Now we have a digital twin of our device created in Kaa. So let’s continue with the Raspberry Pi.

We need to connect a BME280 sensor to the Raspberry Pi. You can find more details about that in our tutorial on connecting Raspberry Pi to the Kaa Cloud.

In case you don’t have a sensor, skip the above step and move on. If your sensor is not BME280, it’s not a problem. After connecting our Raspberry Pi to the network, we need to install Node-RED.

                    $ sudo apt install nodejs
                    $ sudo apt install npm
                    $ sudo npm install -g --unsafe-perm node-red
                    

As soon as the installation is over, we can now start Node-RED:

                    $ node-red
                    

When the console reports that Node-RED is running, we open a browser on the Raspberry Pi and go to 127.0.0.1:1880. See the image below.

Step 2. Sending data over MQTT

We need to install a BME280 node. To do that, let’s click on a menu in the top right corner and choose Manage palette. After that, we click on the Install tab and search for bme280.

We need to install node-red-contrib-bme280 node and then import the client flow description from Kaa into Node-RED by clicking Menu → Import .

If you have a BME280 sensor, use this flow description.

If not, use the next flow description.

Copy the matching code, paste it into the field filled in red, and then click Import.

Then, open the Init Global node and change the app version and token accordingly.

To deploy our flows to the server, we click Deploy to go live. Node-RED will run the flow on your Raspberry Pi server, sending the data from your BME280 sensor into the Kaa platform.

Step 3. Visualizing data from your device

We need to enable time series auto extract in the Endpoint Time Series service (EPTS) Kaa component. It transforms raw data samples into well-structured time series and stores them. It also provides API for other services you can integrate.

In the screenshot below, you can find how to enable the time series auto-extraction from data samples.

When enabling this function, you allow Kaa to create a time series for each numeric field it finds in data from your IoT connected devices.

And finally, we proceed to the last step in our tutorial. Now, you can display data from your device in the Kaa IoT platform.

We need to go to the device details page of the recently created endpoint. It can be done by clicking on the corresponding row in the device table on the Devices dashboard. See the data from your Raspberry Pi on the “Device telemetry” widget, as shown in the screenshot below.

Voila, we did it. Now you are able to send data from your Raspberry Pi and BME280 sensor with Node-RED and visualize it in the Kaa UI. For more information, visit our tutorial on connecting Node-RED to Kaa from the Kaa documentation.

Bottom line

Wrapping up, we have seen that the Node-RED visual programming not only works in the Internet of Things but also makes connecting IoT devices so much easier, acting as a shortcut for your IoT solution mapping.

Rapid IoT solution development enabled by Node-RED draws the attention of large enterprises like Siemens, Hitachi, and Samsung. Node-RED offers extensive features, simplicity of use, and thousands of nodes contributed by an enthusiastic community made up of both commercial and home-grown IoT solution architects.

We know from experience that building an end-to-end solution for IoT is a challenging endeavor. We have you covered with the Kaa IoT platform that provides remote device management and analytics for IoT. It has proven its simplicity and efficiency both in enterprise IoT and ‘smart’ projects for personal use.