Node-RED: Lecture 3 – Basic nodes and flows

In this lecture you will get to see a few of the more commonly used nodes and build on some of the things you learnt in previous lectures. You’ll start off with a series of examples based around the popular MQTT protocol that shows how to wire together a set of basic but very useful message processing nodes. Then you’ll briefly look at other ways to get messages into and out of your flows using protocols such as TCP, UDP and websockets.

In this lecture you will get to see a few of the more commonly used nodes and build on some of the things you learnt in previous lectures. You’ll start off with a series of examples based around the popular MQTT protocol that shows how to wire together a set of basic but very useful message processing nodes. Then you’ll briefly look at other ways to get messages into and out of your flows using protocols such as TCP, UDP and websockets.

At the end of this lecture you will have a better understanding of some of the basic nodes used in Node-Red flows. You will also have seen how easy it is to build sophisticated processing flows that take real-world events, process them and generate responses that use popular Internet protocols to communicate the results outside of your flow.

 

We use a cloud hosted version of Node-RED for these lectures called FRED. Sign up for a free account at FRED. Examples in the early lectures will work with other installations of Node-RED, later lectures use nodes you will need to install yourself if you don’t use FRED.

 

Examples

Example 3.1 Receiving JSON via an MQTT message

Example 3.2 Using the switch node to handle a JSON object

Example 3.3 Using a change node to change or manipulate a message payload

Example 3.4 Using the rbe (report by exception) node

Example 3.5 Scaling input with the range node

Example 3.6 Using an mqtt output node to test the flow

Example 3.7 Using Websockets with Node-RED

Example 3.8 Sending TCP requests.

Example 3.1 Receiving JSON via an MQTT message

The following series of examples builds on the mqtt node, which provides a convenient way to take input from an MQTT broker. For those not familiar with MQTT, it is an example of a publish/subscribe system (usually shortened to pub/sub system) which lets sensors publish updates that all delivered to client subscribed to that sensor. MQTT uses a topic model allowing publishers (eg sensors) to create topics and publish data to the topics Equally, others can subscribe to a topic and will receive asynchronous notification of data posted to the topic.

Pub/Sub systems are a great way to connect loosely coupled distributed systems and they map well to typical IoT patterns where devices or things generate events that you want to share. The MQTT protocol, apart from being asynchronous, is also lightweight and doesn’t have as high an overhead as HTTP; which for resource-constrained devices is often an important advantage. MQTT was originally developed in the late 1990s and has been used in a variety of IoT settings. MQTT became an OASIS standard in 2014 and is a standard part of many IoT toolboxes. MQTT actually stands for Message Queueing Telemetry Transport.

To use the mqtt node, you need to have access to a broker. There are a number of free MQTT servers running, for example http://test.mosquitto.org/, or the one that will be used in this lecture, www.hivemq.com. Using the broker address and the topic, you can configure the mqtt input node to subscribe on that topic, causing it to generate a new message whenever new data is published on that topic. The message will contain information on the published data, including the data itself in msg.payload and the MQTT broker topic in msg.topic.

To get you started with the mqtt node, you’ll be using the free mqqt broker hivemq – which is available via (http://www.hivemq.com/showcase/public-mqtt-broker/). Of course you can use any MQTT broker, including your own, if you have installed one.

First, drag and drop an mqtt input node and configure it for the broker. Don’t forget to configure the topic to something unique, in the case of this example we are using noderedlecture/sensor but you should use your own unique topic, i.e. <your name here>/sensor

Figure 3.1 Configuring the mqtt node with broker address and topic

There are many ways you can send mqtt messages to hivemq. You can use their websockets client showcase (http://www.hivemq.com/demos/websocket-client/), the mqtt dashboard (http://www.mqtt-dashboard.com/dashboard) or your own library. You’ll be using their websocket client in this example, so navigate to that page and connect to the broker. You will publish a JSON encoded string to the topic you configured to see both the use of the mqtt node and the json node.

Figure 3.2 Sending an MQTT message (publishing) using the HiveMQ client page.

Since you are sending a JSON string, you will need to parse the message that the mqtt node generates when it receives the MQTT message. To do that, you’ll need to drag and drop a json node and connect it to the mqtt node’s output.

Node-RED’s json node is a sort of convenience function, in that it parses the incoming message and tries to convert it to/from JSON. So if you send it a JSON string, it will convert it to a JavaScript object, and vice versa.

If you wire up the usual debug node to the json node and deploy, then use the HiveMQ dashboard to send the JSON string {“analyze”:false, “value”:10} as shown in Fig 3.2. You will see it printing in the debug tab (Fig 3.3).

 

Figure 3.3 Receiving and parsing an MQTT message sent as a JSON string

If you look closely at the output, you can see that the msg.payload contains an object, which itself has two fields, analyze and value, each with their own values. As you saw in lecture 2, you can access these fields via msg.payload.analyze and msg.payload.value. Let’s take a look at a node that can do that.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-1_mqqtmessages.json

Example 3.2 Using the switch node to handle a JSON object

One of the nice features of having a JSON object is that you can easily act on its properties. A useful node for this is the switch node. Its role is to ‘switch’ or route messages depending on the incoming message properties.  For example, you can check the msg.payload.analyze property and, depending on its value (true/false), decide to route a message to one of the switch node’s outputs.

Drag a switch node and double-click on it. Configure it to evaluate the property “msg.payload.analyze”. If true, send the message to the first output; if false, send it to the second output as shown in in Fig 3.4.

Figure 3.4 Configuring the switch node to route based on a message property

Now you can connect two debug nodes as shown in Fig 3.5 – when you set up multiple outputs for a node, they are numbered from the top, so output 1 is the top output and output 2 is at the bottom in  Fig. 3.5.

Figure 3.5 Connecting a switch node to two debug nodes

If you now go back to the HiveMQ input page and send the MQTT message {“analyze”:true, “value”:6}, you will see that the first (top) output is activated and the incoming messages is routed, or ‘switched’, to output 1. If you send the original message {“analyze”:false, “value”:10}, the switch node will activate output 2 and the original debug node will fire. Hovering the pointer over the debug message will show which debug node is printing out the message as shown in Fig. 3.6.

 

Figure 3.6 Confirming the operation of the switch node

As you can see, this provides you with a built-in Node-RED node that allows you to quickly determine the contents of incoming messages and direct the message through to different parts of the flow depending on the input.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-2_switchnode.json

Example 3.3 Using a change node to change or manipulate a message payload

Another useful node is the change node, which will allow you to change a message payload or add new properties. You can use this node to affect the properties in a message, either by changing existing ones, deleting them or adding new properties.

In this example, you’ll continue with your MQTT theme and see how, now that you have successfully ‘switched’ the message flow based on the incoming MQTT message, you can add a new message property msg.payload.note.

First, let’s drag and drop a change node and connect it to the second output of the switch node (Fig 3.7). As you will remember, this is the output that fires when msg.payload.analyze is set to false.

Figure 3.7 Adding in a change node to set a new message property.

 

Now configure it to set the property msg.payload.note to “this is not being analyzed” as shown in Fig 3.8.

 

Figure 3.8 Using a change node to add a new message property

 

When you receive a message that the switch node sends on the 2nd output, it will be modified to contain a “note” element with the string “this is not being analyzed”. If you deploy and test the flow by sending the MQTT message from HiveMQ, you’ll see the output as shown in Fig 3.9.

 

Figure 3.9 The result of switching and then changing the message

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-3_changenode.json

Example 3.4 Using the rbe (report by exception) node

In this example, you’ll continue your message analysis theme and add nodes to the part of the flow that is used when you determine that the flow should be analyzed. You’ll be using the rbe (report by exception) node which only passes on data if it has changed. You can set it to examine a message payload and either block until a message changes (rbe mode) or when a messages changes by a specified amount (deadband mode). In rbe mode, it works on numbers and strings. In deadband mode, it works on numbers only and uses the configured deadband as a + or – ‘band’, so that the incoming value can fluctuate within a range before it fires.

You’ll start by adding in another change node which you’ll connect to output 1 of the switch node. You’ll then connect an rbe node to the switch node as shown in Fig 3.10.

Let’s connect a change node, and an rbe node like this. To remind us that this output deals with the flag “analyze”, add a comment node and write “Analyze = true”. Comments are useful when writing complex flows.

Figure 3.10 Adding an rbe node to check if our input data has changed by more than 20%

Edit the change node to set the msg.payload to msg.payload.value. This will set the output of this node to the value found in the msg.payload.value element of the input received (Fig 3.11).

Figure 3.11 Using a change node to set payload

Since you want to determine if this value has changed by 20% or more, you’ll need to double-click on the rbe node and configure it to block unless the value changes by more than 20%.

Fig 3.12 Setting the rbe node to check msg.payload.value

To test the flow, deploy this flow and then return to the HiveMQ page and send a series of messages. Firstly, you’ll need to set the analyze value to true so that the switch node sends through a message on output 1. If you use the original message value of 6, this will fail to pass the rbe node. If you then send a second message of value 10, the rbe node will evaluate the difference between 6 and 10, see it is greater than 20%, and send on a message to the final debug node which will print on the debug pane as shown in Fig 3.13.

Fig 3.13 Confirming that 10 is more than 20% larger than 6 using an rbe node.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-4_rbenode.json

Example 3.5 Scaling input with the range node

When dealing with real world input from sensors and other devices, an ability to scale input data is often required. Node-RED provides the scale node to support this and allows you to scale (linearly) an input value. Let’s assume you want to scale your value (originally in the range 0-10) to a range (0-255) when you aren’t doing any analysis. This means we are dealing with the lower part of the flow fired when the switch node evaluates the Analyze property as false.

To do this, select the change node you configured above (set msg.payload) and copy it with ctrl+c, then ctrl+v. Attach a range node as shown in Fig 3.14.

 

Figure 3.14 Scaling input values using the range node.

Double-click on it, and configure it to map the input from 0-10 to 0-255 as shown in Fig 3.15.

The scale node has three options set by the action field. The default will scale according to the mapping given but will happily scale values outside the given ranges, using the same mapping. Scale and limit to target range means that the result will never be outside the range specified within the result range. A third option, Scale and wrap within the target range means that the result will essentially be a “modulo-style” wrap-around within the result range.

Then return to the HiveMQ test page and post {“analyze”:false, “value”:10} as a new MQTT message to the same topic.

Figure 3.15 Setting the scaling ranges for input and output of the scale node

If you return to your Node-RED window, you will see that the debug node associated with the lower part of the flow has fired, showing that the msg.payload.value property that you set as 10 when you published it to MQTT, has been scaled up 255 as shown in Fig 3.16.

Figure 3.16 Final scaled output when analysis is turned off

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-5_rangenode.json

Example 3.6 Using an mqtt output node to test the flow

As an alternative to using the HiveMQ test page to publish on the MQTT topic, we can configure an mqtt output node. This is the mirror of the mqtt input node and allows you to configure an MQTT service and the topic you are publishing on. You can then send the node messages with the exact same JSON string we’ve been sending via the HiveMQ test page.

To try that, drag and drop three inject nodes and an mqtt output node as shown in Fig 3.17, and configure and wire up as shown

Figure 3.17 using an mqtt node to publish on the noderedlecture/sensor topic

You can now test the flow you created to analyze the MQTT message directly from the workspace by clicking on the three inject nodes in sequence. The first will exercise the part of the flow handling the case when analysis is turned off; the 2nd two messages will cause the path with the rbe node to be exercised.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-6_mqqtout.json

Example 3.7 Using Websockets with Node-RED

Websockets are another useful communication capability that is built into Node-RED via the the websocket node. Websockets provide a duplex TCP connection and were designed to allow web browsers and servers to maintain a ‘backchannel’ that could be used to augment traditional HTTP interactions, allowing servers to update web pages without the client making a new pull request.

The websocket node comes in two flavours, input and output, allowing you to listen for incoming data (input) or to send (output) on a websocket. The output version is designed to check to see if the output payload originated at a websocket in a node, in which case it responds to the original sender. Otherwise it will broadcast the payload to all connected websockets.

In addition, both input and output websocket nodes can be configured as either server or client – in server mode they ‘listen on’ a URL, and in client mode they connect to a specified IP address.

To see how the websocket nodes work, you’ll use a public websockets echo server which runs on the public site: (https://www.websocket.org/echo.html).

Drag an inject, websocket in, websocket out and a debug node onto the workspace and connect them as shown in figure 3.18.

Figure 3.18 using websockets to communicate with a Node-RED flow

Configure the inject node to send a string payload of “Hello There” (Fig 3.19)

Figure 3.19 Configuring an inject node to send on a websocket

Configure the websocket nodes to connect to wss://echo.websocket.org as shown in Fig 3.20.

Figure 3.20 Configuring the websocket to send to a public echo server. Do the same for the websocket out node.

Deploy. When you click on the inject node you will see the message printed out as shown in Fig 3.21

 

Figure 3.21 Output from a websocket listening for incoming data

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-7_websockets.json

Example 3.8 Sending TCP requests.

This example shows you how to send TCP requests using the tcp node. In this case you will make an HTTP request following the specifications in (http://tools.ietf.org/html/rfc2616#section-5.1.2).

This example shows the use of the tcp node. It could equally be configured with the udp or http nodes in a similar manner.

To get started, let’s connect an inject, function, tcp request, and debug nodes as shown in Fig 3.22.

Figure 3.22 building a TCP request and sending on a tcp output node

Edit the first function node to add a function that sets the string “GET / HTTP/1.1\r\n\r\nHost: www.google.com” as payload as shown in Fig 3.23.

This string is a standard HTTP request, indicating it is a GET request, the protocol is HTTP 1.1 and the host is www.google.com. The \r\n\r\n is two return/newline pairs which is required in the HTTP protocol.

Figure 3.23 Building a TCP request in a function node

Configure the tcp request node to connect to the www.google.com server, on port 80. Configure it to close the connection after 1 second (1000 ms) as shown in Fig 3.24.

Figure 3.24 Configuring the end-point for a TCP request

The tcp request node response is a buffer and needs to be parsed. Configure the second function node to parse the tcp request node response as shown in Fig 3.25

Figure 3.25. A function node to parse a response buffer into a string

If you deploy the flow and click on inject, you will make a request to Google and will get a TCP response. The debug node will print the response as a string as shown in Fig 3.26.

Figure 3.26 Printing out the response to a well formed HTTP request sent over a TCP connection.

Some of you may be wondering why you need to use a function node to build the HTTP request that we sent over TCP. Why not just use the inject node to input the string? The reason is that the inject node ‘escapes’ the string it uses, causing the return/newline you inserted to be removed. This in turn confuses the receiving server (Google) into not returning a response as it waits for the missing return/newlines. So instead, you build the string in a function node. This is one of those ‘gotchas’ that trips up even experienced Node-RED programmers, so always read the info pane for nodes to make sure you understand any limitations or constraints.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-8_tcp.json

Summary

In this lecture you have seen a series of small examples that use a number of the processing and communication nodes that are available in Node-RED. As you have seen, it is both easy and fast to wire together basic flows that take real-world input, carry out some processing, such as simple data analysis, and return results.

In these examples, you have done little or no coding, but still have been able to build quite complex programs – this is the power of Node-RED.

The next lecture is a quick summary of the basic set of nodes that are available in both vanilla Node-RED as well as the extended set that the FRED service provides. You can read through the lecture to understand the default capabilities, or you can treat the lecture as a reference and use it to look up examples of each node as used in this lecture series.

 

 


About Sense Tecnic: Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures, and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

Author: Rodger Lea

Currently CEO of Internet of Things startup, Sense Tecnic, Dr. Lea has over 25 years experience spanning academic, large corporations and startups. For the last 10 years, he has started or helped start 4 new companies while managing an active research program (University of British Columbia, Canada and Lancaster University, UK) into distributed and ubiquitous computing, the IoT and Smart Cities.

3 thoughts on “Node-RED: Lecture 3 – Basic nodes and flows”

  1. Pingback: Node-RED tutorial
  2. The basic step required to connect to HiveHQ just comes back with “SocketError:Undefined” when I try to connect on the HiveHQ websocket-client page. I have tried different ports, but nothing seems to work?

Comments are closed.