Tutorial: Node-RED dashboards – creating your own UI widget

Creating your own UI widget for a Node-RED dashboard

Node-RED’s dashboard nodes provide a comprehensive set of UI components for building basic dashboards suitable for the Internet of Things (IoT) – offering graphs, gauges, basic text as well as sliders and inputs. However, there will always be situations when you need something custom. The template node is the solution and in this tutorial we’ll show you a quick example of using it to create your own UI widget.

If you want a general introduction to the Node-RED dashboard or to UI techniques for Node-RED, then check out these resources which are part of our lecture series:

The template node

The ui template node is a generic node that takes valid html and Angular/Angular-Material directives and can be used to create a dynamic user interface element.

In this example, we’ll create a simple dashboard element that displays some fake random data mimicking data from a factory machine sensor.

We want our UI element to look as shown on the right of the screen shot below (you can see how to create the 2 line chart in this tutorial)

screen-shot-2016-12-07-at-7-31-32-am

As you can see – our UI widget is named “and more” and has four data columns. These data column show the fake factory machine data and are dynamically updated as new data arrives.

We’ll set up a flow as shown below – the function node will generate the fake data, and the UI template node, which we name Machine status, will format and present the data in the table shown above.

screen-shot-2016-12-07-at-7-21-49-am

If we take a look inside the function node, you can see we use the usual Math.random function to generate some fake data. However, instead of just passing this data in as payload, we insert the data into a JSON structure called factory. Factory has two elements -machine00 and machine01 which in turn have four elements min, max, failProb and name.

screen-shot-2016-12-07-at-7-22-44-am

Once we have set up our function node to create the fake data. We now need to create a UI template node and set it up to display on our dashboard. As you can see below, we assign it to the Home dashboard and use a new group name “and more”. (Details on setting up dashboard groups can be found here)

screen-shot-2016-12-07-at-7-23-09-am

Let’s take a look in more detail at the HTML used to display the data we generate in the function node.

The first <div> sets out the labels for the columns defining them using the <span flex> tag so they will fit the space nicely.

<div layout=”row” layout-align=”start center”>
<span flex>RPM Min</span>
<span flex>RPM Max</span>
<span flex>Failure Prob</span>
<span flex>Machine type</span>
</div>
<div layout=”row” layout-align=”start center” ng-repeat=”machine in msg.payload.factory”>
<span flex style=”color: green”>{{machine.min}}</span>
<span flex style=”color: red”>{{machine.max}}</span>
<span flex style=”color: black”>{{machine.failProb}}%</span>
<span flex style=”color: black”>{{machine.name}}</span>
</div>

The second <div> then uses the ng-repeat command to iterate through the JSON structure in payload and displaying the data in the columns. It works by looking for each element named machine in the factory element. Remember, we named each set of data associated with a machine, machine00, machine01 – so we iterate through those.

For each machine element, we use the sub-elements min, max, failProb and name to fill the data fields. Note how we also use a style directive to set the colour for the text. The result is shown below.

screen-shot-2016-12-07-at-7-22-08-am

This is a simple use of the template node, but it should be enough to get you started. However, the template node is much more sophisticated, it can be used to dynamically set styles (e.g. colours), show images, create clickable elements that generate messages etc – so once you master the basics, you can go on to create some sophisticated UI widgets for your own dashboards.

Tutorial: Node-RED dashboards – multiple lines on a chart

Showing multiple lines on a single chart in Node-RED

This simple tutorial explains how to display multiple lines on a Node-RED chart. We’ll be building on a previous example in our lecture series (Example 7.2 in lecture 7)

As discussed in that lecture, let’s set up a simple flow to generate a random number and display it on a chart.

We use a cloud hosted version of Node-RED for these tutorials called FRED. Sign up for a free account at FRED. You can use this tutorial with a vanilla N0de-RED, eg on a Pi, but make sure you are accessing the correct URL for the dashboard (and not the FRED specific one).

To start, let’s wire up a simple flow that sends a random number between 0 and 99 to a simple chart. For that you’ll need an inject node to repeatedly fire every few seconds, a function node to generate the random number and one of the  node-red-dashboard nodes – in this case the chart node.

screen-shot-2016-10-12-at-4-38-47-pm

Before we look a how the chart node works, let’s configure the inject node to send a timestamp every 5 seconds by setting the payload to timestamp and the repeat field to an interval of 5 seconds.

Screen Shot 2016-05-16 at 12.46.11 PM

This will act as our repeating trigger. Now we need to set up the function node to generate a random number – we’ll use a simple JS math function to do this:

msg.payload = Math.round(Math.random()*100);
return msg;

Screen Shot 2016-05-16 at 12.48.42 PM

This will generate a random number between 0 ~ 99 which is passed to the chart node.

So now let’s take a look at the chart node. When you double click it, you’ll see it’s configuration options:screen-shot-2016-10-12-at-4-46-53-pm

If you click on the button of the Group field, you will be prompted  to configure the tabs of the UI.

screen-shot-2016-10-12-at-4-42-04-pm

The Tab option allows you to specify which tab of the UI page you will see the UI element on – in this case our chart. The default tab is Home – which we are using here. If you select the edit button to the right of the Tab field you can create a new tab and then select that. However, we’ll use the default home for now.

The Name field is the standard Node-RED node name – by default this is chart but you can set it to anything you like.

The Group field allows you to group UI elements – we’ll show you how that works when we add another UI element so let’s use group “Default[Home]” for now – of course, you can use any string you like.

The X-asis field allows you to tell the chart how much data it should store and display – the longer the ‘last‘ filed is set to, the more data is stored and displayed by the chart. Let’s use a short 5 mins which will start to throw away the data that is 5 minutes old.

Lastly the Interpolate field defines how the chart will interpolate values in between actual data values it receives, you can select between linear, step, b-spline and cardinal – which are standard interpolation algorithms. We’ll use the default linear.

Wire these nodes up, hit the deploy button – check that your debug node is showing that random values are showing. Then head over to your default dashboard page to see the results. By default, under FRED, you’ll see your UI at:

https://fred.sensetecnic.com/red/ui/

When you visit that page you’ll see your initial chart as shown below:

screen-shot-2016-10-12-at-4-48-47-pm

As you can see, you get a nice line chart, scaled for your data values (X axis) and the time they arrived (y axis). You can also see “Default” at the top left of the chart, indicating this is UI group “Default” – which is the group name we set in the configuration fields for the chart node.

 

Adding a second line to our chart

Ok, now let’s see how to add another line to the same chart.

If you take a look at the info box for the chart node, you’ll see it mentions that the the msg.topic field can be used to show multiple lines. Essentially the chart node will receive messages, take a look at the msg.topic field, and then append the data to the appropraite line as defined by the topic.

So, to extend our example to handle multiple lines on a single chart, we have to do two things:

  1. update our function node to generate two random numbers and assign these to different messages with different topics
  2. Wire up a second output from the function node to handle the second message

To do that, open up the function node and update the code inside as shown below. As you can see we explicitly create two messages, msg and msg1 (lines 1-2).

At line 4-5 we set msg.topic to ‘Line1’ and a random number is assigned to the payload. We repeat that at line7-8 setting msg1.topic to ‘Line2’ and assigning a different random number.

screen-shot-2016-12-05-at-5-28-56-pm

Both messages are returned using the return message (line 9) – because we are generating two return message, we need to tell the function node that it has two outputs by setting the outputs field to 2 (see above).

More details of generating multiple messages from nodes can be found in the main lectures Example 5.2 in lecture 5.

Once you’ve done this, your flow should look like the screen shot below.

screen-shot-2016-12-05-at-5-23-17-pm

Wire up both outputs from the function node to the chart node and then take a look at the dashboard and the chart. As you can see (shown below) the chart now shows two lines (Line1 and Line2) with different colours. Note we set the dashboard theme to dark to make the two lines clearer.screen-shot-2016-12-05-at-5-24-06-pm

Obviously this example is contrived, in a more realistic example you might   have two sources of data, and so two function nodes delivering messages (rather than the single function node with two outputs).

In addition, you’d be pulling the data from a real source, e.g. a web scrape, an MQTT feed or perhaps a DB. You can see examples of these:

 

Tutorial: SQLite and Node-RED

This tutorial will show you how to make use of an SQLite database using the litedb node on the FRED hosted Node-RED platform to create database tables and store data. Under the hood, the litedb node uses SQLite.  As a very lightweight relational database, SQLite does not need complex setup procedures, making it an ideal database management system to use for embedded systems and rapid prototyping.

The tutorial will cover the basics of:

  • Setting up the dblite (SQLite) node and creating a DB table
  • Generating random data and storing in the SQLite DB
  • Reading from the SQLite DB
  • Graphing the data from the DB using the standard Node-RED UI nodes

Read the full tutorial here

Lecture 7: Node-Red dashboards (Part 3)

Example 7.3 Using a simple JavaScript chart library to build a custom dashboard

For our final example of building Dashboards and UIs we’ll use a generic technique we’ve experimented with before. That’s the web service approach that uses a http node to allow us to accept http requests and return web pages. We used this approach in example 1.3 but didn’t explain the details.

Sticking with the openweathermap node, let’s use it to generate a simple JSON structure that we then visualize using the popular Morris.JS chart library (). The library supports 4 basic chart types line charts, area charts, bar charts and the donut chart. We’ll be using the donut chart but the approach is the same for all the chart types.

The flow is shown below and consists of 4 nodes. The first and final nodes are http input and output nodes that work together to listen for HTTP requests and send HTTP responses. This flow listens for a HTTP request from any source, but let’s assume a regular browser here. When it arrives, it queries the weather using your openweathermap nodethen uses a template node to build a HTTP page using the weather data, and passes that to the http output node which sends back the webpage to the browser.

Screen Shot 2016-05-17 at 2.37.43 PM.png

Figure. A simple web server example to graph weather data using a pi chart

Double click on the http input node and configure it with your URL, e.g. /public/weather. Remember, to access it, you will need to use,

fred.sensetecnic.com/public/{user name}/weather

where {user name} is replaced by your own FRED user name. Obviously /weather is because that’s how you configured it, if you’d used another name, e.g. /public/test, then the URL would end /{user name}/test

When a HTTP request comes in, the http input node creates a message to trigger the openweathermap node, which is the next node in the flow. The openweather node gets the current statistics for the location you configured it for, and then passes those statistics, as a JSON structure in the usual message payload field, to a template node. The html template node is another pre-built node in Node-RED, which, like the function node, allows you to build arbitrary code. However, rather than using JavaScript like the function node, the template node works with text such as HTML.

The HTML code in the template node is shown in the listing below. As you can see, the code is a standard HTML structure with the <head> structure defining the external script libraries that we use. These are all required for the Morris.JS chart library and should be copied exactly.

Note, we are combining all the page elements into the one file (or node) which works for simple web pages but is not considered good practice.

Line 10 sets up a <div> and sets the name and height. Then the morris.js chart code  script begins at line 11 by defining a Morris.Donut element.

Listing  A simple HTML template to display a donut chart of calorie usage

  1. <!doctype html>
  2. <head>
  3.     <title>A Node RED Example</title>
  4.     <link rel=”stylesheet” href=”//cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.css”>
  5.     <script src=”//cdnjs.cloudflare.com/ajax/libs/raphael/2.1.0/raphael-min.js”></script>
  6.     <script src=”//ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js”></script>
  7.     <script src=”//cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.min.js”></script>
  8. </head>
  9. <html>
  10.     <div id=”chart-example” style=”height: 250px;”></div>
  11.     <script>
  12.         Morris.Donut({
  13.            element: ‘chart-example’,
  14.            data: [
  15.              {label: “Temperature (Celcius)”, value: {{payload.tempc}} },
  16.              {label: “Humidity”, value: {{payload.humidity}} },
  17.              {label: “Wind Speed (knts)”, value: {{payload.windspeed}} }
  18.           ]
  19.        });
  20.     </script>
  21. </html>

Lines 15, 16 and 17 define the donut elements, you are using 3 to display the temperature, the humidity and the wind speed. The values from the openweathermap JSON structure can be used directly, i.e. payload.tempc, payload.humidity and payload.windspeed.

Once the template node has generated the HTML file, it passes it as a message to the final node, which is a http response node. This node packages up the HTML as a HTTP response which it sends back to the browser that made the original request.

If you deploy this flow and then point your browser to the URL you configured in the http input node, in your case should be:

https://fred.sensetecnic.com/public/{user name}/weather

You will then see a simple donut chart which you can mouse over to see the temperature, humidity and windspeed for the city you configured in the openweathermap node – all built and served by a simple Node-RED flow!

Screen Shot 2016-05-17 at 2.59.31 PM.png

Figure: A donut chart, served up by Node-RED showing Temperature for the configured city

Summary

In this lecture you have looked at three approaches to building simple dashboards and UIs. The FreeBoard node is a one stop shop allowing you to feed complex data as JSON structures to the Freeboard node and then using the powerful FreeBoard dashboard to configure and show the data.

With the nodered.contrib.ui nodes you have to do a little more work by having to build the UI elements into your flows. This approach gives you more control, and makes the UI elements part of your Node-RED flow, but requires a little more thought and work.

The final approach, using the html template node and a generic JavaScript charting library is the most flexible, but requires you know how to write simple html/css and JavaScript.

Part 1    Part 2


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Lecture 7: Node-RED dashboard (Part2)

Example 7.2 Introduction to the UI nodes from node-red-dashboard


This second example uses the built in dashboard nodes that come with Node-RED. If you are using FRED, make sure you have selected this node set from the add/remove button in the management panel and that you aren’t using the old legacy-ui nodes.

To start, let’s wire up a simple flow that sends a random number between 0 and 99 to a simple chart. For that you’ll need an inject node to repeatedly fire every few seconds, a function node to generate the random number and one of the  node-red-dashboard nodes – in this case the chart node.

screen-shot-2016-10-12-at-4-38-47-pm

Before we look a how the chart node works, let’s configure the inject node to send a timestamp every 5 seconds by setting the payload to timestamp and the repeat field to an interval of 5 seconds.

Screen Shot 2016-05-16 at 12.46.11 PM

This will act as our repeating trigger. Now we need to set up the function node to generate a random number – we’ll use a simple JS math function to do this:

msg.payload = Math.round(Math.random()*100);
return msg;

Screen Shot 2016-05-16 at 12.48.42 PM

This will generate a random number between 0 ~ 99 which is passed to the chart node.

So now let’s take a look at the chart node. When you double click it, you’ll see it’s configuration options:screen-shot-2016-10-12-at-4-46-53-pm

If you click on the button of the Group field, you will be prompted  to configure the tabs of the UI.

screen-shot-2016-10-12-at-4-42-04-pm

The Tab option allows you to specify which tab of the UI page you will see the UI element on – in this case our chart. The default tab is Home – which we are using here. If you select the edit button to the right of the Tab field you can create a new tab and then select that. However, we’ll use the default home for now.

The Name field is the standard Node-RED node name – by default this is chart but you can set it to anything you like.

The Group field allows you to group UI elements – we’ll show you how that works when we add another UI element so let’s use group “Default[Home]” for now – of course, you can use any string you like.

The X-asis field allows you to tell the chart how much data it should store and display – the longer the ‘last‘ filed is set to, the more data is stored and displayed by the chart. Let’s use a short 5 mins which will start to throw away the data that is 5 minutes old.

Lastly the Interpolate field defines how the chart will interpolate values in between actual data values it receives, you can select between linear, step, b-spline and cardinal – which are standard interpolation algorithms. We’ll use the default linear.

Wire these nodes up, hit the deploy button – check that your debug node is showing that random values are showing. Then head over to your default dashboard page to see the results. By default, under FRED, you’ll see your UI at:

https://fred.sensetecnic.com/red/ui/

When you visit that page you’ll see your initial chart as shown below:

screen-shot-2016-10-12-at-4-48-47-pm

As you can see, you get a nice line chart, scaled for your data values (X axis) and the time they arrived (y axis). You can also see “Default” at the top left of the chart, indicating this is UI group “Default” – which is the group name we set in the configuration fields for the chart node.

If you look at the top left of the web page, you can see that we are, by default, on the home tab. If you had created your own tab then when you click the selector top left you’ll get a pull down menu of your tab options:

screen-shot-2016-10-12-at-4-52-40-pm

That was pretty simple, let’s add a few other UI elements to our dashboard. Firstly let’s create a gauge to show the last data value sent. Drag a Gauge node from the UI palette and wire it to the Random Number function node.

Then double click to open up and let’s configure it:

screen-shot-2016-10-12-at-4-56-16-pm

We’ll us the same Tab, home and we’ll also add it to the same group – “Default[Home]”. The Min and Max fields allow you to set the min and max values the gauge will shown. Make sure the max is set to 100 which is the most that the random number function node will generate. You can also change the Colour gradient to show different colours on the widget, but we will leave it as default for now.

Hit deploy and then head over to your dashboard and you’ll see that the chart and gauge are displayed in a group with the chart now showing the last 5 minutes of data and the gauge the latest value.

screen-shot-2016-10-12-at-5-01-16-pm

As a last example, let’s use a couple of the other UI nodes, a slider node and a text node to show the same data on a slider and as a text string.

screen-shot-2016-10-12-at-5-03-24-pm

For these two nodes, configure them to use the same tab – “Home” but use group name “anotherWidget”(You will need to click “Add new UI_group” from the drop down menu of the Group field, and then click the edit button). You will also need to change the min and max value for the slider node to show the correct position of the slider. Deploy those and let’s take a look at your dashboard. As you can see, we now have two widget groups, group “Default” with a chart and a gauge, group “anotherWidget” with a text filed and a slider. Simple eh?

screen-shot-2016-10-12-at-5-05-09-pm

In the dashboard tab beside your debug tab, you can also set the theme and order of the elements. If you don’t see the dashboard tab, click the menu button at top right corner, then select “View” -> “Dashboard”. You can see all the widgets and tabs showing in a tree structure, and you can easily drag the elements to change the orders that they are presented in the dashboard.

screen-shot-2016-10-12-at-5-10-31-pm

Now you’ve got the basics sorted out, play around with the different dashboard elements to build your own dashboards using some real world data – have fun!

Part 1      Part 3

Node-RED: Lecture 7 – Dashboards and UI techniques for Node-RED

In this lecture you will take a look at a few techniques to allow you to visualize data passing through flows. We’ll focus on three approaches, the use of a third party dashboard tool, FreeBoard (Part 1), using the default Dashboard UI nodes provided by default in Node-RED (Part 2) and a general technique using a standard JavaScript charting tool (Part 3).

By the end of the lecture you will have enough knowledge to decide which approach is best for your specific needs and how to quickly get a visualization of your data up in a web browser.

We use a cloud hosted version of Node-RED for these lectures called FRED. Sign up for a free account at FRED.sensetecnic.com. Examples in the early lectures will work with other installations of Node-RED, later lectures use nodes you will need to install yourself if you don’t use FRED.

Example 7.1 Using the FreeBoard dashboarding service

This is a simple example of reading and visualizing data using the FreeBoard node from a Node-RED flow. We’ll be using the Cloud based FRED service as our Node-RED engine and visualizing data from a web weather service . This guide will show you how to:

  • set up an openweathermap node in FRED using the built in node
  • display the data from the openweathermap node using the FreeBoard display node

To begin, head over to openweathermap.org and register for a free account. When you register/login you will be able to access your API Key. You will need your own API key which will look similar to the one below – you will use your own key in Node RED so make a copy.

 

01

Now, head over to FRED and create a new flow by starting with an openweathermap node (you will find this node under “weather”, or by using “filter nodes” to search at the top of the left pane) and connecting it to a debug node like this:

02

 

Now, double click on the openweathermap node to configure it. Enter your API Key from OpenWeatherMap, configure a location you’re interested in, and name your Node:

 

03

Click on Deploy on the top right corner of the Node-RED UI. You will see on the right pane, under “debug” the data that OpenWeatherMap provides us with a variety of information about the location including temperature (in C and K), humidity, windspeed etc:

 

04

As you can see, the data is already in JSON format; so we can use any of the values very easily. Let’s build a dashboard to visualize our data using the  node that allows you to create visualizations very easily. Find the freeboard node in the left pane by either browsing through the nodes or searching for it using “filter nodes”. Add it to the flow. Double click on it to give it a name, like so.

05

 

Click on Deploy. This will prompt the openweathermap to get data and send it to both the debug and freeboard nodes. The freeboard node is quite smart and will do its best to parse the data you send it and figure out how to make it available using the FreeBoard UI. We’ll look at that in a moment. If you inspect the “info” tab in the right panel you will see more information about the “freeboard” node:

06

 

Visit the link provided in the info panel, or at

https://fred.sensetecnic.com/red/freeboard/

This will open a new tab in your browser:

07

This tab will allow you to create visualization in FreeBoards, save them and load them. Let’s create a visualization of our weather data. We first need to add a datasource to our freeboard. Click on “Add” under “DATASOURCES”. Under “type” select the name of the “freeboard” node we configured above. In our case we named it “freeboard”. This will allow us to access ANY data we connect to the “freeboard” node in Node RED.

 

08

 

We will now add a Pane. Click on “ADD PANE”, this will add a new empty pane:

09

 

Now, we will add a “Widget”, click on the “+” (plus) sign in the new pane. Select “Gauge”. Under “DATASOURCE” select the “freeboard” node and select “tempc”. As you can see, all of the different data fields in the openweathermap JSON structure are available for you to visualize. FreeBoard essentially takes the JSON structure you pass it and breaks out any fields. If you can’t see this in the dropdown, go back to Node RED and click on “Deploy” again. So that the “freeboard” node can get some data and store the data it has received. Your configuration should look something like this:

 

10

 

Click on OK. Which will build your widget and add it to the pane you created!

11

 

To save your dashboard click on “save”, and select [pretty] or [minify] which will save it in FRED and will show you the URL to access the Dashboard. You need to save the URL used to access the dashboard, e.g. as a dashboard. From now on you can access your saved dashboard by accessing the saved url, which will be similar to the one below.

https://fred.sensetecnic.com/red/freeboard/#start-16091

Part 2  Part 3

 


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

 

Node-RED: Lecture 6 – Intermediate flows

This lecture will build on the ideas you saw in lecture 5 and focus on examples that explore some of the key concepts from that lecture. The examples in this lecture are a little more complex than previous examples – mainly in the sense that the function nodes are more complex – but are still kept as clean and simple as possible.

This lecture will build on the ideas you saw in lecture 5 and focus on examples that explore some of the key concepts from that lecture. These include the ideas of context, messages and sub-flows. The examples in this lecture are a little more complex than previous examples – mainly in the sense that the function nodes are more complex – but are still kept as clean and simple as possible.

 

We use a cloud hosted version of Node-RED for these lectures called FRED. Sign up for a free account at FRED. Examples in the early lectures will work with other installations of Node-RED, later lectures use nodes you will need to install yourself if you don’t use FRED.

 

Examples

(Click to go directly to the example)

Example 6.1 Retrieving data from a web page

Example 6.2 Counting words in a string

Example 6.3 Using context to generate rolling averages

Example 6.4 Using the context element to share a function and make it accessible to all functions in the canvas

Example 6.5 Defining and using an iterator sub-flow

Example 6.6 Getting earthquake data from an external API and returning it as multiple messages

Example 6.7 Multiple inputs on a function node

Example 6.8 Letting a function node send multiple messages on a single output

Example 6.9 Creating a Blog Site With Node-RED

 

Example 6.1 Retrieving data from a web page

Let’s write a flow that scrapes the latest stock market indices from Yahoo at http://finance.yahoo.com/market-overview/ and format it using a function node. If you inspect the page shown in Figure 6.1 using the Chrome browser’s ‘Inspect Element’, you’ll find that the three indices have the same CSS class called ‘l84’ (that’s a lowercase ‘L’ as in ‘look’). This class can be used to retrieve all three indices with the html node. You’ll set up an inject node to trigger an http request to get the page, then the html node to get the elements with the ‘l84’ class.

Figure 6.1 Stock market index page from Yahoo.

Configure the http node and html nodes as shown in Figure 6.2

Figure 6.2 Configuration of http and html nodes to retrieve stock indices from Yahoo.

Then wire them up as shown in Figure 6.3, with a debug node to show the output.

 

Figure 6.3 Flow to retrieve the market indexes.

The output should contain the latest index values. When you click on the ‘trigger’ inject node, you will see the following in the debug pane:

Figure 6.4 Stock indices as array of strings.

That was easy, but what you really want is an array of JSON name:value pairs that uses the name to describe the values and shows the values as numbers:

Listing 6.1 Desired JSON output of stock indices

  1. [{
  2.     “index”:”S&P”,
  3.     “value”:2096.92
  4.  },{
  5.     “index”:”Dow”,
  6.     “value”:17511.34
  7.  },{
  8.     “index”:”Nasdaq”,
  9.     “value”:5059.35
  10.  }]

Let’s write a function node to format this the way you want it. Take a look at Listing 6.2. Line 1 takes the incoming stock prices and stores them in an array prices and then creates two new arrays, one to hold the output messages and the other to hold the textual name of the three different indices. Lines 5-8 loop through the incoming stock indices and for each one pushes a name:value pair to the output message array. At Line 8, you use the JavaScript replace function to remove all commas from the index values before parsing them as numbers. Let’s call this function format indices.

Listing 6.2 Function node code to format stock indices

  1. var prices = msg.payload;
  2. var newPayload = [];
  3. var priceIndex = [‘S&P’,’Dow’,’Nasdaq’];
  4. for (var i=0; i<prices.length; i++) {
  5.   newPayload.push({
  6.     index:priceIndex[i],
  7.     value:Number(prices[i].replace(/,/g,’’))
  8.   });
  9. }
  10. msg.payload = newPayload;
  11. return msg;

Now wire this into your flow as shown in Figure 6.5.

Figure 6.5 Complete market index flow.

Once you deploy and test, your debug console should output something similar to Figure 6.6. The data now looks good and you can use this format for downstream nodes in your flow.

Figure 6.6 Debug output from new flow.

Example 6.2 Counting words in a string

Next, let’s write a more complex function node that receives some text in a message payload, then outputs multiple messages containing all individual words and the number of times each word was used.

Listing 6.3 Word count function

  1. var outputMsgs = [];
  2. var wordMap = {};
  3. var sentence = msg.payload.replace(/[.,-\/#!$%\^&\*;:{}=\-_`~()]/g,””);
  4. sentence = sentence.replace(/\s{2,}/g,” “);
  5. var words = sentence.split(” “);
  6. for (var i = 0; i < words.length; i++) {
  7.   var lowerCaseWord = words[i].toLowerCase();
  8.   if (!wordMap[lowerCaseWord]) {
  9.     wordMap[lowerCaseWord] = 1;
  10.   } else {
  11.     wordMap[lowerCaseWord] = wordMap[lowerCaseWord] + 1;
  12.   }
  13. }
  14. for (var prop in wordMap) {
  15.   if( wordMap.hasOwnProperty( prop ) ) {
  16.     outputMsgs.push({payload:{word:prop,count:wordMap[prop]}});
  17.   }
  18. }
  19. return [outputMsgs];

In Listing 6.3, the list of output messages and an object to hold the word counts (Lines 1 and 2) is declared. In lines 4 and 5 the payload is ‘cleaned’, removing punctuation and extra spaces using regular expressions. Regular expressions are a very useful tool for text processing; you can learn more about regular expressions by looking at the Mozilla Developers pages for JavaScript (here).  Line 7 splits the text into multiple words, then iterates through the words, creating a mapping of the lowercase version of each word to word count called wordMap in Lines 8-15.  Lines 16 to 18 split the wordMap into multiple messages in the outputMsgs array[1].   Finally, in line 21, an array of this array of messages is returned, sending them all to the first output port one at a time[2].

Let’s wire this up and see it run with some example text.  First, create a function node, and copy in the code above. Call it word count. Then add an inject node and add the following text:

Figure 6.7 Inject node to test word count function node.

Add a debug node and wire up the flow as follows:

Figure 6.8 Test flow for word count function node.

When you click on the inject node, you should see the list of word counts in the debug pane:

Figure 6.9 Debug output from word count function node.

Example 6.3 Using context to generate rolling averages

A special module called context,used to store data between function invocations, is available to function nodes. This can be useful when the function needs to retain state to do its processing. For example, it may be necessary to compute the average value of a sensor’s data readings over a period of time. Listing 6.4 computes a rolling average of the values received over the last 5 seconds, adding the ‘average’ field to the payload when more than 5 seconds have elapsed between received messages.

Listing 6.4 Rolling average function using context

  1. var currentTime = new Date().getTime();
  2. if (!context.lastTime) {
  3.     context.lastTime = currentTime;
  4.     context.sum = msg.payload.value;
  5.     context.count = 1;
  6. }
  7. if (currentTime-context.lastTime > 5000) {
  8.     // calculate average for previous messages
  9.     msg.payload.average = context.sum/context.count;
  10.     // start tracking average again
  11.     context.sum = msg.payload.value;
  12.     context.count = 1;
  13.     context.lastTime = currentTime;
  14. } else {
  15.     context.sum += msg.payload.value;
  16.     context.count +=1;
  17. }
  18. return msg;

Looking at listing 6.4, you first get the current time (line 1).  If there is no lastTime stored in context, you then save the currentTime, reset the sum and count context variables (lines 3-7).

If the currentTime is 5 seconds (5000 milliseconds)  greater than the last time (line 9), you calculate the average of the last values received and include that with the message payload (lines 11-13).  You then reset the sum, count and set lastTime to the currentTime to begin counting and summing up again.

If 5 seconds haven’t elapsed since the last message was received, the counts and sums need to be updated (lines 16-19). You then output the message which includes the latest value and the average if it was calculated for the interval (line 20).

To test this, let’s write another function node, that you’ll call ramp and that also uses context to generate values from 0 to 9 in sequence, as shown in Listing 6.5.

Listing 6.5 Test function to generate sequence of values using context

  1. if (!context.value) {
  2.   context.value = 0;
  3. }
  4. msg.payload = {
  5.   value:context.value
  6. }
  7. context.value +=1;
  8. if (context.value > 9) {
  9.   context.value = 0;
  10. }
  11. return msg;

Let’s configure an inject node to send data every second, as shown in Figure 6.10.

Figure 6.10 Inject node configured to inject a blank payload every second.

Then wire these up as shown in Figure 6.11.

Figure 6.11 Test flow for rolling average function.

The output should look like the one in Figure 6.12.

Figure 6.12 Debug console output showing rolling average.

Example 6.4 Using the context element to share a function and make it accessible to all functions in the canvas

This example shows how to use the context object and its global element to share data across function nodes. The example above used this to store a numerical value. However, one of the great things about JavaScript is that you can assign functions to objects. This allows you to share methods across your canvas without the need to redefine them in each consecutively used function node. The contrived example below simplifies this process for better understanding.

First, connect an inject node, two function nodes and two debug nodes like this:

 

Figure 6.13 Wire up functions for a context example

Let’s edit the first function node and add this code. Set the global element of the context object to have a new element, “hello”, which is an anonymous function that returns the string “Hello There”. Then set the message payload of this function to the return value of this global function. This will print out “Hello There”.

Figure 6.14 Define an anonymous function as a new context element

Now edit the second function node by setting up the message payload of this second function to concatenate the output of the ‘hello’ element (anonymous function) in the global element of the context object with the word “World”:

Figure 6.15 Use the anonymous function defined in another function node

Pressing the button on the inject node shows the output of each of the function nodes:

Figure 6.16 Output of context function sharing example

Despite being very simple, this example shows how easy is to use the context element to set not only variables that contain data, but also functions that can be shared and accessed by several functions in your canvas.

Two important aspects to consider are: firstly, the function setting the global element of the context object must come BEFORE any other function wishing to use that variable or function in the flow. Secondly, in the current version of Node-RED (1.11.0), the context object remains in memory after re-deployment until the Node-RED instance is rebooted. So even after deleting the function nodes that set an elements in the global element, these elements will still be accessible until set to null.

Example 6.5 Defining and using an iterator sub-flow

As mentioned in lecture 5, sub-flows can be used to package up functions into nodes in your node pallette. In this example, you’ll create a sub-flow that processes an array of values in a message payload and outputs a new processed array[3].  An upstream node will supply a message with a payload containing an array of values to be processed. The first output of the iterate node will output each of the input message array payload messages , contained in a message payload in order. The second output will output a message containing a payload with a new array containing processed values. To process data, you can wire up the sub-flow as shown in Figure 6.17, connecting a function node to provide some test data and process each element in the array.

Figure 6.17  Example sub-flow to iterate through list of values in an array payload.

Start by creating a new sub-flow tab. Drag in a function node and add the code from listing 6.6.

Listing 6.6 Iteration function node

  1.   var currentMsg = null, outMessage = null;
  2.   var iState = msg.iState;
  3.   if (!iState) {
  4.      // we received an initial message
  5.      // if the message is not an array, make it one
  6.      if( Object.prototype.toString.call(msg.payload) !== ‘[object Array]’ ) {
  7.            msg.payload = [msg.payload];
  8.      }
  9.      iState = {};
  10.      iState.index = 0;
  11.      iState.inArray = msg.payload;
  12.      iState.outArray = [];
  13.      msg.iState = iState
  14.    } else {
  15.     // save results from the last iteration
  16.      iState.outArray.push(msg.payload)
  17.  }
  18.   //If there are still objects left to iterate goto the next one in the original array
  19.   if (iState.index < iState.inArray.length) {
  20.     currentMsg = msg;
  21.     msg.payload = iState.inArray[iState.index];
  22.   } else {
  23.     currentMsg = null;
  24.     outMessage = msg;
  25.     msg.payload = iState.outArray;
  26.     delete msg.iState;
  27. }
  28. iState.index ++;
  29. return [currentMsg, outMessage];

In lines 1 and 2, several variables are declared.   currentMessage, holds the current input message; outMessage holds the final output message containing our processed array payload and iState is used to make it easier to access the current state of the iteration from the msg.iState property.  While context could be used to manage the function’s state, it’s simpler to hold state in the message, since you may receive new arrays as input before completing the iteration of another array.

In line 4, you check to see if iState exists. If not, assume that the message is a new input message.  Check that the payload is an array, and if not, make it one in lines 7-9. Then generate the iteration state object that includes the current index, input array, output array, and add it to the msg in lines 10-14.

If iState does exist (16-18), you know it’s an iteration message generated by this function node and processed in the loop back.  Push it into the output array to send out when you’re done.

Next, see if you’re done or not. If you need to iterate again (index is less than the length of the input array), set currentMessage to the message and set the payload to thecurrent index (lines 20-22).  When finished iterating, set the currentMessage to null, the outMessage to the message, the payload of the message to the outputArray and delete the iState property that is no longer needed.

Then increment the index and output the messages to the output endpoints. During the iteration, currentMessage is checked to ensure it holds a value, otherwise (when null) the loop terminates.  At the end, outMessage will be non null and sent to the second output.

Now let’s hook up inputs and outputs to our sub-flow as shown in Figure 6.18.  Let’s name this subflow ‘Iterate’.

Figure 6.18 Iterate sub flow.

To test out the flow, generate an array from 1 to 5 as in Listing 6.7, and multiply each element in this array by 5.  To do this, create a new function node called [1,2,3,4,5] and wire it in as shown in Figure 6.17.

Listing 6.7. function node [1,2,3,4,5] from test flow in Figure 6.17

  1. msg.payload = [1,2,3,4,5];
  2. return msg;

Write a second function that is used in each iteration, as in Listing 6.8, calling it times 5, and wire it up to the new subflow output 1 and input.

Listing 6.8 multiply payload times 5

  1. msg.payload = msg.payload * 5;
  2. return msg;

Finally trigger the flow, using an inject node, and add a debug node. After clicking on the inject node, the debug window should display [5,10,15,20,25].

Now that this iteration function is packaged in a sub-flow you can use it in any of your flows by simply dragging it in from the node palette like any other node. It will iterate over any array of integer values and apply the function specified.

Example 6.6 Getting earthquake data from an external API and returning it as multiple messages

This example demonstrates how to get data from an external API and how to separate that data using a function node. We will use data from an external API that provides access to earthquake data which is made available by the US geological survey (USGS). (http://earthquake.usgs.gov/earthquakes/feed/v1.0/geojson.php)

First, let’s wire the flow like this:

 

 

Figure 6.19 Flow to query USDG earthquake data

Let’s edit the http request to get data from the following url as shown below (Fig 6.20):

http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/significant_month.geojson

This data contains significant earthquakes from the last month. The JSON node after the request will allow you to parse the body of the response into an object usable by the following function.

Figure 6.20 Setting up a http request node to query USDG earthquake data

Now edit the function node and add the code shown in Fig 6.21:

This code will create an array (outputMsgs) containing an array of messages built from your data. The for loop goes through the received JSON response and creates a new message with a payload containing lat, lng, value, message and timestamp values (only a few values from the JSON object sent from the server). It will then push this new message object to the outputMsgs array.

It will create a new variable (msg2) with a new message object containing a payload element with the string “Second Output”.

Finally, the function will return an array containing two elements. The first element is an array of messages (outputMsgs); the second element is a single message (msg2). Let’s configure the function to have 2 outputs, to match our returned array.

Figure 6.21 Function node code to query USDG earthquake data

When you click the button on the inject node, you can see that the top output contains an array of messages parsed from the data. This corresponds to the outputMsgs array in the function (See Fig. 6.22).

Figure 6.22 First output from the flow that queries USDG earthquake data

The second output contains the single message (msg2):

 

Figure 6.23 Second output from the flow that queries USDG earthquake data

Example 6.7 Multiple inputs on a function node

Function nodes in Node-RED were designed to process messages as single entities. However, in some cases your functions might depend on two separate data sources. There are many ways to handle these cases in Node RED. The following approach uses the context object in Node-RED and topics to let a function wait for several messages to arrive in order to return. You saw how to set-up and use context data in Example 5.3 in the previous lecture.

Let’s start by connecting two inject nodes, a function node and a debug node like this (Fig 6.24):

Figure 6.24 Setting up a flow to explore multiple inputs to a function node

Let’s edit the function node and add the following code (see Fig 6.25). This code will use the context object in Node-RED and add a data element.

Line 1 initializes the context object. Then the switch statement at line 2 looks for the topic field in the message. It uses this to set the task1 or task2 field of the context.data object. Any other message topic is ignored. Line 16 then checks to see if the function has received messages of both topic types (task1 and task2). If not, the function returns a null message and goes back to wait for another message. Otherwise, line 17 calculates the ratio and outputs it as a message.

Figure 6.25 Function node code to wait for all input before proceeding

Let’s configure the first inject node to return a string payload of “3”, with a topic of “task1”

Figure 6.26 Configuring an inject node with data and a topic used by a context object

You should configure the second input node to return a payload string of “6”, with a topic of “task2” (not shown but similar to Fig 6.26)

You can then deploy the flow. Click on the left button of the “task1:3” inject node. You will see a success message indicating that the string has successfully been injected, but you will not see anything in the debug tab. Click on the left button of the “task2:6” inject node. You will see a success message and the debug tab will show the ratio as expected:

 

Figure 6.27 Exercising the multiple input flow of Example 6.7

Example 6.8 Letting a function node send multiple messages on a single output

Example 5.2 in Lecture 5 demonstrated how to set up and send messages on multiple output nodes. This example shows how to send multiple message, but on the same output, from a single function node. One common scenario is a function node batch processing some data, with the following nodes wanting to have processed data as soon as it becomes available. Another common scenario is the creation of nodes that in turn create counters or pulses to actuate relays.

The following example is based on a flow by Node-RED contributor dceejay. Tthe original flow can be found at http://flows.nodered.org/flow/5c4c0f5a08d4e91ea14d

First, wire together an inject, function, delay and debug nodes:

Figure 6.28 A basic flow to explore multiple output messages

Edit the function node and add the following code (See Fig. 6.29). This code is quite simple; it contains a loop that calls “node.send” 5 times. node.send was introduced in Lecture 5 at the end of example 5.2. It allows a function node to output messages to its output independently of its return value. Note how the function itself returns null. The expected output of this function will be a series of messages with the payloads: 0,1,2,3 and 4.

Figure 6.29 Code to send multiple messages using the node.send() command

To make the example ressemble a counter more closely, you can edit the delay node and configure it to limit the rate of messages to 1 per second, as shown in Fig 6.30.

Figure 6.30 Configure the delay node to limit its output rate to 1 message per second

The flow can now be deployed. Press the button left of the inject node and inspect the debug tab (See Fig 6.31) and you will see the messages arriving every second:

 

Figure 6.31 Output from Example 6.8

 

Example 6.9 Creating a Blog Site With Node-RED

This final example will show you how to build a micro blog service with only a few nodes in Node RED. You’ll be using the MongoDB node as a storage for posts, http nodes to provide end points for the service and the html node to format the micro-blog web page.

We will use a free cloud service called “mongolab”. It allows you to create “sandbox” databases you can use to prototype your applications. Head over to https://mongolab.com. Register for an account. You will then be able to create a database. Click on “Create New” in your home dashboard.

Figure 6.32 Creating a free MongoDB at mongolab

Fill out the form to create a new MongoDB deployment. Select Amazon, Single-node and select the FREE size of 500mb.

Figure 6.33 Choosing the free MongoDB

Scroll down and name your database. We are naming it “mycontent”. Now click on Create new MongoDB deployment:

Figure 6.34 Name the free MongoDB

MongoDB allows you to create “collections” for each database. They are analogous to “tables” in relational databases, each “collection” holds “documents” analogue to “records” in relational databases. In order to begin using your database you need to create a new collection. Select your newly created database in your dashboard and add a new collection. We will name it “posts”.

Figure 6.35 Adding a collection to the  MongoDB

Finally, you will need a user to connect to that database. Add a new database user. We will name it “freduser” but you should use a name that makes sense to you.

Figure 6.36 Adding a user to your MongoDB

Finally, note how this page also gives you important information on how to connect to your MongoDB database. We will need the URI (in our case ds037234.mongolab.com), the port (37234 in our case) and your newly created user and password.

Figure 6.37 Note down the connection info for your new MongoDB

Now, head over to Node-RED and wire up a http-in, mongodb-in, a template, and an http-out node as shown in Fig. 6.38.

Figure 6.38 The basic flow for the micro-blog

Edit the http-in node to accept GET request on the URL “/public/posts” (Fig 6.39)

Figure 6.39 Setting up the http node to accept requests on /public/posts

We will now configure our mongodb in node. Double click on the node and create a new server connection. Here is where you will use the information from your mongolab instance:

Figure 6.40 Configuring the mongodb node using the MongoDB account info

Click on Create/Update and configure your node to use the “posts” collection you created. Configure it to do a “find” operation, which will find all documents in that collection. We will now name it “find posts”.

Figure 6.41 Configuring the mongodb node with the collection and the find operation

Once you have the http node and the mongodb nodes configured you can edit the html template node to handle the data returned by the mongodb node (Listing. 6.9):

 

Listing 6.9 Formatting the blog posts

And add the following code:

  1. <!DOCTYPE html>
  2. <html lang=”en”>
  3.   <head>
  4.   </head>
  5.   <body>
  6.     <h1>My Micro Blog</h1>
  7.     <form action=”posts” method=”post”>
  8.         <label for=”title”>Title</label>
  9.         <input type=”text” class=”form-control” name=”title” placeholder=”Title…” />
  10.         <label for=”post”>Post Content</label>
  11.         <input type=”text” class=”form-control” name=”post” placeholder=”Say something…” />
  12.         <button type=”submit” class=”btn btn-default”>Submit</button>
  13.     </form>
  14.     <div>
  15.         {{#payload}}
  16.         <div class=”post”>
  17.             <h3>{{title}}</h3>
  18.             <p>{{post}}</p>
  19.         </div>
  20.         {{/payload}}
  21.     </div>
  22.   </body>
  23. </html>

This code, while long, is very simple. You will be using Bootstrap to style your website (http://getbootstrap.com/) and provide a responsive layout. If you now visit http://fred.senstecnic.com/public/YOURUSERNAME/posts you will be able to see your new blog site (Fig 6.42). However there is no content yet.

 

Figure 6.42 Output from the micro-blog html template node

 

 

So far, you’ve created the flow needed to display posts. Now you need to create the other side of the blog service, a flow to create posts and save them to your MongoDB collection. Connect an http-in, a function, mongodb-out, and http-out nodes as shown in the bottom part of Fig 6.43.

 

Figure 6.43 Setting up a flow to allow you to post blog entries and store them in MongoDB

Edit the http-in node to accept PUT requests at /public/posts (Fig 6.44)

Figure 6.44 Setting up the end point for posts for the micro blog service

Now let’s edit the function node and add the following code (listing 6.10). This builds a simple http msg.header which your flow will pass to the http node.

 

Listing 6.10: Building a http request for the micro-blog service

  1. msg.headers = {
  2.     “Location” : “http://fred.sensetecnic.com/public/guides/posts”
  3. };
  4. msg.statusCode = 302;
  5. return msg;

 

Note that at line 2 you set the address to be the end point for your service. This will use your user name in place of ‘guides’.

Finally, you will need to edit the mongodb-out node and select your previously configured server. Configure it to use the collection “posts” and an “insert” operation. Make sure to check “only store msg.payload object”, this will make sure we only store the data in the message payload, not the whole message object (fig 6.45).

Figure 6.45 Configuring the mongodb node to store the incoming blog postings

That’s it! You can now go to your url and use your micro blog site where you’ll see something similar to Fig 6.46.

 

Figure 6.46 Screen show of the simple micro-blog web page

 

 

Summary

In this lecture you have seen a number of more complex flows that build on the nodes and techniques introduced in earlier lectures. In particular, you’ve seen how to scrape data from a commercial web site and query web-based APIs, how to use context and message in more complex ways and how to build a simple web service to support a micro-blog. The next lecture will conclude our intermediate section (lectures 4-7) by introducing how to install Node-RED on devices such as the Raspberry Pi. It will show some examples that you can’t run on the cloud-based FRED service (because they require access to the underlying hardware), but which do run on the Pi, and it will discuss some of the issues with flows in devices such as the Pi.

 


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

 


[1] The check for hasOwnProperty() in line 17 is good practice when iterating through the properties of an object using the JavaScript in operator. It ensures you’re getting only properties you’ve added yourselve and none that you’ve (unexpectedly) inherited from the Object base class.

[2] Note that if you returned outputMsgs directly, the function node would attempt to send each message to a different output.

[3] Based on forEach (array iteration & completion) subflow by dhartveld at https://gist.github.com/dhartveld/43501a1b424434de0ffb

Node-RED: Lecture 5 – The Node-RED programming model

In this lecture, you’ll take a more detailed look at Node-RED’s programming model and some of its key concepts and explore the details of the message flow model that underlies Node-RED. Then you’ll dive into how to program your own function nodes using JavaScript and create reusable sub flows to add your own functionality to the set of nodes supplied with Node-RED.

As you’ve seen in previous lectures, Node-RED uses a visual flow-based programming paradigm[1]. This is not unique to Node-RED. Similar approaches have been used in many other domains such as music and multimedia (Max MSP), toys (Lego Mindstorms), enterprise application integration and industrial automation (LabVIEW).  Like these tools, Node-RED makes it easy to switch between design and development for rapid integration, prototyping and development tasks.  So far, you’ve seen how to wire simple flows together and some of the basics of Node-RED programming.

In this lecture, you’ll take a more detailed look at Node-RED’s programming model and some of its key concepts and explore the details of the message flow model that underlies Node-RED. Then you’ll dive into how to program your own function nodes using JavaScript and create reusable sub flows to add your own functionality to the set of nodes supplied with Node-RED.

 

We use a cloud hosted version of Node-RED for these lectures called FRED. Sign up for a free account at FRED. Examples in the early lectures will work with other installations of Node-RED, later lectures use nodes you will need to install yourself if you don’t use FRED.

Examples

Example 5.1 Creating and returning a new message in a function node

Example 5.2 Creating and returning multiple messages in a function node

Example 5.3 Using Context in a function node

Example 5.4 Using Global Context

Example 5.5 Creating a sub-flow

Example 5.6 Packaging Function nodes using sub-flows.

Example 5.7. Creating a Slack Bot to query data

Key Concepts

As you’ve learned so far, when you program with Node-RED, you create flows that are networks of lightweight components, called nodes, that exchange messages along pre-defined connections or wires.  Let’s look at these concepts in a bit more detail.

Flows

Node-RED programs or flows are a collection of nodes wired together to exchange messages.  Under the hood, a flow consists of a list of JavaScript objects that describe the nodes and their configurations, as well as the list of downstream nodes they are connected to, the wires.

Messages

As discussed in Lecture 2, messages passed between nodes in Node-RED are, by convention, JavaScript Objects called msg, consisting of a set of named properties[2].  These messages often contain a msg.payload property with, you guessed it, the payload of the message.  Nodes may attach other properties to a message, which can be used to carry other information onto the next node in the flow. When this happens, these extra properties will be documented in the node documentation that appears in the node info pane when you select a node in the Node-RED workspace.

Messages are the primary data structure used in Node-RED and are, in most cases, the only data that a node has to work with when it is activated. This ensures that a Node-RED flow is conceptually clean and stateless – each node is self-contained, working with input messages and creating output messages.  Apart from the use of context data (see later in this lecture), this means that the effect of a node’s processing is either contained in its output messages, or caused by internal node logic that changes external things such as files, IO pins on the Raspberry Pi or Dropbox files; there are no side effects that could affect the behaviour of other nodes or subsequent calls to the same node.

This is one of the key advantages of a flow-based language. Because nodes are self contained and typically only interact with other nodes using messages, you can be sure that they have no unintended side effects and so can be safely re-used when you create new flows. This ‘safe’ code reuse is exactly what you are doing each time you drag and drop a node onto your workspace.

Nodes

Nodes are the primary building block of Node-RED flows.  When a flow is running, messages are generated, consumed and processed by nodes.  Nodes consist of code that runs in the Node-RED service (javascript .js file), and an HTML file consisting of a description of the node, so that it appears in the node pane with a category, colour, name and an icon, code to configure the node, and help text.  Nodes can have at most one input, and zero or more outputs[3].

During the initialization process, the node is loaded into the Node RED service.  When the browser accesses the Node RED editor, the code for the installed nodes is loaded into the editor page as illustrated in Fig. 5.1.

Figure 5.1 Node RED loads both HTML for the editor and JavaScript for the server from the node packages.

 

As you saw in Lecture 2 (Figure 2.2), there are three core node types:

  • Input nodes – generate messages for downstream nodes.
  • Output nodes – consume messages, for example to send data to an external service or pin on a device, and may generate response messages.
  • Processing nodes – messages that process data in some way, emitting new or modified messages.

In addition to these core types, there are two more categories:

  • Credentials Nodes – these are nodes that hold the credentials used by one or more nodes to connect to an outside system or service such as the Sense Tecnic platform, MQTT or Pushbullet. These are displayed only in the credentials pane, not on the main pane, and are created as needed when you configure a node that requires credentials such as an API key or name and password. Once created, a credentials node can be reused by other nodes of the same type to connect to similar protocols or services. Even when all nodes that use those credentials are deleted from your flow, the credentials node will remain, so it’s a good idea to remove unused credentials nodes when they are no longer needed.
  • User-created nodes – programmable nodes such as function nodes, or sub-flows are nodes created by you to do some custom work or reuse flow segments in other flows.

Wires

Wires define the connections between node input and output endpoints in a flow. They (typically) connect the output endpoints of nodes to inputs of downstream nodes indicating that messages generated by one node should be processed by the connected node next.  Note that it is possible to connect more than one node to an endpoint using wires.  When multiple nodes are connected to an output endpoint, messages are sent to each connected node in turn in the order they were wired to the output.  When more than one node output is connected to an input endpoint, messages from any of those nodes will be processed by the connected node when they arrive.  It is also possible to connect downstream nodes to upstream nodes to form loops.  (You’ll see an example of wiring a node to an upstream node in example 6.5 in Lecture 6).

Context

So far, you have learnt that messages are the only way to get data into and out of nodes. While this is generally true, there is one exception to this rule which is available to function nodes. function nodes have access to a special object called context that is used to hold data in memory that lasts from one message arriving to the next[4]. This is important for nodes that need to maintain an index or count or sum data in messages.  In addition to this local context, a global context context.global is available for sharing data between all of the function nodes of a flow. Some use cases for context will be covered when the function node is discussed in more detail.

Function Nodes

The function node is the ‘Swiss Army knife’ of nodes that you can use when there is no existing node dedicated to your task at hand.  It’s great for doing specialized data processing or formatting for example. As the name implies, a function node exposes a single JavaScript function. Using the function node, you can write your own JavaScript code that runs against the messages passed in and returns zero or more messages to downstream nodes for processing. To write function nodes, you write a JavaScript function using the built-in code editor as shown in Fig. 5.2.

Figure 5.2 The function code editor.

 

If you are already familiar with writing JavaScript, then you can skip the next section, which is a very quick primer on JavaScript, which is used when you are writing your own function nodes.

JavaScript Primer

While we don’t have enough space here to cover JavaScript completely, we’ve written a short primer  for those familiar with other programming languages to get started writing your own nodes. To experiment with JavaScript, you can use the JavaScript console supplied with your favourite browser or startup node.js to run the Read-Eval-Print-Loop (REPL) on your machine.

Read the JavaScript primer

Writing Function Nodes

Let’s get started writing function nodes.  To test these out, you’ll create a simple flow with an inject and a debug node as shown in Fig. 5.3.

Figure 5.3 Simple flow to test our function nodes.

The most simple function node just returns null. When you return null, no message is passed on to downstream nodes and the flow ends here. To create a node that passes the message “as is”, you can simply return the message itself. This is the default code in a function node.

  1. return msg;

Neither of these cases are very interesting, so let’s add some content to the payload. In this example, and the next set of examples that explore writing JavaScript code for the function node, you will use the same flow (i.e that shown in Fig 5.3) and you’ll edit the function node to change the JavaScript using the function node editor as shown in Fig 5.2.

Listing 5.12 Adding some text to the incoming message payload

1 msg.payload += “world”;

2 return msg;

In this example (listing 5.12), a simple string concatenation operator is used to add the string “world” to the incoming message payload (line 1). This updated payload is then sent, using return,  as the output message (line 2) to the next node in the flow.

If you edit the function node you’ve just created and type in the code from listing 5.12 when you deploy and test the flow, you will see in the debug pane the timestamp for the inject node and the text “world” appended.

Obviously, if you now edit the inject node to inject a string instead of the timestamp, and set the injected string to “hello”,you will see “hello world” in the debug pane, when you deploy and test.

Example 5.1 Creating and returning a new message in a function node

Listing 5.13 shows you how to create a new message by defining a new message newMsg, assigning the string “new payload” to the property payload (line 1) and then returning the new message at line 2.

Listing 5.13 Creating and returning a new message

1 var newMsg = { payload: “new payload” };
2 return newMsg;

If you modify the test function node you set up as shown in Figure 5.2 and use the code from Listing 5.13  when you deploy the flow and hit the inject button, the new message you created is output when it arrives at the debug node.

Example 5.2 Creating and returning multiple messages in a function node

The function node also allows you to configure it with multiple outputs. Once you’ve configured multiple outputs, you can send multiple messages, one to each output, using an array.

First, let’s edit the function node to tell Node-RED that the node will have three outputs, using the node configuration window (Fig. 5.4).

Figure 5.4 Setting the number of outputs that a function node provides

Listing 5.14 illustrates how to send a message to one of three different outputs based on a message payload value ‘high’, ‘med’ or ‘low’:

Listing 5.14 Returning message on multiple outputs using an array of messages

  1. if (msg.payload == “high”) {
  2.   return [ msg, null, null ];
  3. } else if (msg.payload == “med”) {
  4.    return [ null, msg, null ];
  5. } else {
  6.    return [null, null, msg];
  7. }

Line 1 checks the incoming payload to see if it is set as “high”. If yes, then it passes on the message on the first output and null messages on the 2nd and 3rd output at line 2 by returning an array with three elements, [msg, null, null]. A “med” message causes a message to be returned on output 2 (line 4). Anything else returns a message on output 3 (line 6).

To try it out, wire it up to three inject nodes and three output nodes as shown in Figure 5.5. Set the different inject nodes to inject a text string of either low, med, high, edit the test function and use listing 5.14, and then deploy. As you select the different inject nodes, you will see the appropriate debug node fire in the debug pane.

Figure 5.5 Flow to test multiple function outputs

It is also possible to send multiple messages from a function node in sequence. To do this, you return an array of messages in the output message array. Listing 5.15 shows how to write a function that returns 10 new messages containing number payloads from 0 to 9. Line 1 creates a new array, msgList, to hold the messages. Lines 2 and 3 loop 10 times adding a new element to the array msgList. Finally, line 5 returns an array, containing 1 element, msgList, which is itself an array of 10 elements. As you saw in the previous example, by returning an array with one element, you are sending a message on the 1st output of the node. However, this time, because the array element actually contains another array msgList, which itself has 10 elements, Node-RED sends each element as a series of sequential messages on output 1.

Listing 5.15 Returning multiple messages on one output using an array of messages

  1. var msgList = [];
  2. for (var i=0; i<10; i++) {
  3.   msgList.push({payload:i});
  4. }
  5. return [msgList];

Node functions

Your function node code has access to a few utility functions from the built-in node module. These allow you to log text to the console or debug output and send messages from callback functions rather than returning them from the function node itself.

  • node.log() – logs messages to the console
  • node.warn() – logs messages to the console and a warning message in the debug pane
  • node.error() – logs to the console and an error in the debug pane.

In FRED, the Node-RED console can be viewed by going to the landing page, then clicking on the “Status” menu in the drop down menu under your name.  Since it can be a bit cumbersome to switch back and forth between the NR editor and the console, you can open the status dialog in a separate tab, then hit the refresh button to see updates to the console.  Since the node.warn and node.error functions also output to the debug pane, it is sometimes useful to utilize these functions for debugging when the console is not available.

In addition to these logging and debugging functions, the node.send() function can be used to send messages to downstream nodes in a callback function, rather than returning the message(s). Listing 5.16 illustrates how to use node.send() in a function node that sends a message after a specific amount of time.

Listing 5.16 Example of how to use node.send in a callback

  1. // send message after 10 second delay
  2. setTimeout(function() {
  3.     node.send(msg);
  4. }, 10000);
  5. return null;

Other modules that are available to function nodes include the following:

  • console – while the Node.js console module is available, node.log() is the preferred method of logging
  • util – the Node.js util module. This module provides useful utilities such as string formatting functions and generating string representations of objects. For more informaton on the util module see the Node.js documentaiton at https://nodejs.org/api/util.html
  • Buffer – the Node.js Buffer module is for handling raw binary data from TCP streams or from files. For more informaiton on the Buffer module, see the Node.js documentation at https://nodejs.org/api/buffer.html.

Example 5.3 Using Context in a function node

A special module called context,used to store data between function invocations, is available to function nodes. This can be useful when the function needs to retain state to do its processing.  For example, let’s use context to count the number of messages that a function node has processed since it was deployed. Let’s create a function node that adds a count property to each message it processes, as in Listing 5.17.

Listing 5.17 Function to count messages.

  1. if (!context.value) {
  2.   context.value = 0;
  3. }
  4. context.value +=1;
  5. msg.count = context.value;
  6. return msg;

It can be wired up, as usual based on that shown in Figure 5.3), to give it a try. Be sure to change the debug node to display the ‘complete msg object’ so you can see your count property. You’ll probably notice that context is reset every time the function node is deployed. If you need to save state between deployments, you’ll need to use some external storage such as a file or a database.

Example 5.4 Using Global Context

In addition to individual function node context, a global context module is available to share context between function nodes.  Let’s use this module to see how accurate a delay node is, as shown in Fig. 5.6.

Figure 5.6 Flow to calculate the time elapsed for a message to go through the delay node.

The code for the function node called save the start time is shown in Listing 5.18.

Listing 5.18 Code to save the start time of the flow in a global context property.

  1. context.global.startTime = new Date().getTime();
  2. return msg;

Another function node, called time elapsed (see Listing 5.19), generates a message containing the time elapsed since a message passe through the save the start time function.

Listing 5.19 Code to calculate the elapsed time.

  1. var currentTime = new Date().getTime();
  2. var timeElapsed = (currentTime – context.global.startTime)/1000;
  3. msg.payload = “Time elapsed is: “+timeElapsed+”s”;
  4. return msg;

Add and configure a delay node to delay the message for 2 seconds, click on the inject node, and wait about 2 seconds.  If all goes well, the debug output should indicate the time elapsed was close to 2 seconds.

The advantage of this approach is that we need not have a message that carries the start time. The disadvantage is that, if another message arrives at save start time before time elapsed, the context.global.startTime variable will be overwritten.

Saving your Function code in a library

Once you have created some useful code for use in a function node, it is often useful to save that code for later reuse. One way to do this would be to cut and paste it into a new function, but to save you time, you can use the function node library that is built into Node-RED.

To save your code in the function library, click on the Book icon in the function node configuration dialog to the right of the name text edit box, and select “Save to Library”.  You can then provide a folder name and a file name to save your code as shown in Fig. 5.7.

Figure 5.7 Save to library dialog

To make use of your code you can create a new function node, click on the book icon, then click on “Open Library…” then choose your saved library code. The code will be loaded in the dialog. If that’s the code you want, click OK, and it will be added to your function node. You can then edit it as needed.

Sub-Flows

Sub-flows are a way to reuse useful flow segments in Node-RED as fully fledged nodes in the node pane that you can drag into your flows. While it’s possible to save flow segments using the flow library (as described at the end of lecture 2), having flow segments in the node pane provides a level of encapsulation and information hiding that importing saved flows doesn’t offer.  Encapsulation means that it is organized into a single item (node) that can be referred to using a single name; information hiding means that the inner workings of the sub-flow are hidden – you can change how the sub-flow does its job and the flows that use it will not change.

They can be created in two ways: either by selecting parts of your flow and creating the sub-flow from the selection, or from scratch. To create one from a selection, select the nodes in the sub-flow, click on the menu > Subflows> Selection to Subflows.  This will immediately create a new, numbered sub-flow in the node pane under the “sub-flows” category.

You can also create a new sub-flow from scratch by clicking on the menu > Subflows > Create Subflow menu. After creating a new sub-flow, or editing an existing sub-flow by double-clicking on it in the node pane, a sub-flow tab will appear in the main pane. Unlike a regular tab, this tab has four buttons in the top left: “edit name”, “+ input”, “+ output” and “delete subflow”. In this pane, you can edit the sub-flow, add nodes by dragging and dropping them to the sub-flow pane, and wire them up as needed. To connect this sub-flow to the flows that use it, you can add a single input node and any number of output nodes, using the +input and +output buttons.

Example 5.5 Creating a sub-flow

Let’s create a sub-flow that switches outputs depending on the sentiment of messages received. This can be used to analyze messages from various sources, including Twitter.  First, create a sub-flow: Menu>Create Subflow.  Drag a Sentiment analysis node to the pane, then drag a Switch node. Wire up the Sentiment flow to the switch and configure the switch as in Fig. 5.8.

Figure 5.8 Switch node configuration for sentiment switch sub-flow

This will output a message to output 1, if the score is negative, to output 2, if it is neutral, and to output 3, if it is positive. Now, add an input and three output nodes. Wire up the input to the sentiment and the output nodes to the switch as in Fig. 5.9.

Figure 5.9. Sentiment switch sub-flow.

To edit the name of the sub-flow, click on ‘edit name’ in the pane and call it something like Sentiment Switch. You will see your new node in the sub-flows section in the node pane.

Figure 5.10 Flow using sentiment switch sub-flow to display positive tweets about hockey and the canucks.

Let’s use this to output only the positive messages about hockey from a Twitter feed.  Add a new tab, drag in a Twitter node, the new Sentiment Switch sub-flow node you created and a debug node, as shown in Fig. 5.10.  Wire up Twitter to the input and debug to the third output node of your new Sentiment Switch.  After a minute or so, hopefully, someone will tweet something positive about hockey, and you’ll see some output in the debug pane!

Example 5.6 Packaging Function nodes using sub-flows.

Sub-flows are also a nice way to package up function nodes so that they appear in your node pallette and can be dragged into your flows for easier reuse. To illustrate this, let’s package up our counter function inside a sub-flow.  To get started, create a new sub-flow tab. Drag in a function node and add the code from Listing 5.17 – the one that adds a counter property using context.

Create a sub-flow as before and add a function node. Add the code from Listing 5.17.  Then hook up inputs and outputs to our sub-flow as shown in Fig. 5.11. Let’s name this subflow ‘Counter’.

Figure 5.11 Counter sub-flow that wraps function node.

 

To test out our new flow, drive it from an inject node and set up a debug node, as shown in Figure 5.12. Be sure to show ‘complete msg object’ in your debug node to see the counter message property that was added.

Figure 5.12 Test for Counter sub-flow.

Now you have a ‘Counter’ sub-flow node you can drag in any time you like and use it in multiple locations in a flow.

 

 

Example 5.7. Creating a Slack Bot to query data

Note: Oct 2016 – the WoTKit is no longer available, we will update this example shortly. However, if you feel confident, you can replace the queries to WoTKit with a query to your favourite sensor platform – don’t forget to update the parse function node to handle the data returned from your sensor platform.

Let’s finish off this lecture with a more complex example that uses one of the extended node set that FRED provides – the slack bot node and queries data from the WoTKit Internet of things (IoT) platform.

WoTKit is a general purpose Internet of Things (IoT) platform that is designed to manage real world objects (things) providing a set of APIs to create virtual representations of things, post data to these and query the data for these things. It runs as a cloud based service and is free to use for FRED users – you can access it using the same login credentials that you use for FRED. Slack is a general purpose messaging app for teams – it’s used by many developer teams and has a rich feature set. One nice feature is the notion of Slack Bots, which are autonomous programs that can be used to perform tasks and send their results on the slack communication channels.

This example will show you how to build a slack integration that allows a slack user to query the WoTKit platform and returns sensor data based on the simple query term.

For this example you will need to create what Slack calls a Slack Bot User (https://api.slack.com/bot-users) that will serve as a virtual user in your Slack application. So go ahead and create new bot (https://my.slack.com/services/new/bot), see Fig 5.13. If you don’t already have a Slack account, you will be guided through the account creation before you can return to the bot service.

Figure 5.13 Creating a new Slack Bot

 

After creating your Slack Bot you will be given an API Token in the form (Fig. 5.14):

 

  1. xoxb-6816691748-vkIsdfsafE1RgF251yjPSm90WAAsadfSDF1

 

Which you should note down as you will need this token when you set up your flow.

Figure 5.14 Slack returns a new API token for access to the Bot

This example will query for data that has been made public in the WoTKit. We will use the aggregated data retrieval resource of the WoTKit API (http://wotkit.readthedocs.org/en/latest/api_v1/api_sensor_data.html#aggregated-data-retrieval).

To do that, you should log into your FRED account and set up a new canvas for the flow.   Drag and drop a Slack Bot In node into your canvas. Double click on it and configure by adding your Slack Bot API Token (Fig. 5.15). By default it will only listen into the #general channel (you can change this).

Figure 5.15 Configuring the Slack node with API token and channel

Now connect a function node and double click on it. Copy and paste the following function (Listing 5.20, Fig. 5.16):

Listing 5.20 A simple function to parse a text based query

  1. var i;
  2. if (msg.payload !== undefined) {
  3.   i = msg.payload.indexOf(‘query ‘);
  4.   if (i != -1) {
  5.     msg.query = msg.payload.slice(i+6);
  6.     return msg;
  7.   }
  8. }
  9. return null;

 

Figure 5.16 Writing some function code to parse for queries on a slack channel

 

All this function does is parse the messages received by our Bot and search for the string “query “. If found, it will slice the string and create a query parameter in the message containing anything after that text. This will become handy in the next step.

Next, connect an HTTP Request node to your function node and configure it to make a GET request to the following url:

  1. http://wotkit.sensetecnic.com/api/data?text={{{query}}}&beforeE=1

This will make a request for data to the aggregated data retrieval API of the WoTKit API. It will search for sensors containing the query text and will limit the number of elements returned to “1”. You will notice the mustache-style bit “{{{query}}}”, this will add the text you just parsed to your query.

Finally, you’ll need to add a new slack bot out node and configure it with your Bot API Token and Channel (Fig 5.17).

Figure 5.17 Configuring the slack bot out node

Now, to test that the flow work, return to your slack home page and either select the bot you created from the panel on the left (in this example you called it wotkit-fred) or just select the general channel. You’ll be presented with an input box, so type in:

1 query weather

The bot will search for the last data element in WoTKit containing that text and return that (Fig 5.18).

 

Figure 5.18 Results of querying the WoTKit for ‘weather’ via your new slack bot

 

You can query any string, feel free to type other query strings (always remembering to start with the keyword query) to see what other sensors are available.

Now you have the basic service working, let’s make our integration better by working with JSON and adding a new function node to parse the JSON returned by the http node and tidy up the data so it is more readable.

Firstly,  double click on the HTTP Request Node and change the return value to “a parsed JSON object” (Fig. 5.19).

Figure 5.19 Configuring the http request node to return a JSON object

Since the http request node is now returning a JSON object, let’s insert a json node into the flow to handle that. Wire the http node to the json node and then add a new function node which you wire to the json node to receive a message, and wire it to the slack bot out node to send the message back to slack.  Double click on the new function node and paste the following code (listing 5.21):

 

Listing 5.21 A function that parses data from the WoTKit

  1. if (msg.payload[0] !== undefined) { //API returns an array
  2.     data = msg.payload[0];
  3.     dataString = JSON.stringify(data);
  4.     msg.payload =
  5.         “The last data found has a value of “+data.value+
  6.         ” and was observed on ” + data.timestamp_iso +
  7.         ” under sensor ” + data.sensor_name+
  8.         “\n\nHere is the raw data:\n\n”+dataString;
  9.     return msg;
  10. } else {
  11.     msg.payload = “Sorry, we couldn’t find any data”
  12.     return msg;
  13. }

Figure 5.20 Javascript code to extract sensor data and meta-data.

This will search for the first element (as our response from the WoTKit is an array of data) and format it. As a bonus it will return a better “not found” message. Your complete flow should look like that shown in Fig 5.21.

Figure 5.21 final flow for example 5.7

 

Now you can make public data queries on your channel receiving both formated and raw data from the WoTKit:

Figure 5.22 Formatted output from our query on the WoTKit

Summary

This lecture reviewed some of the key concepts behind Node-RED and introduced the context and global context objects used to share and maintain state between messages received by function nodes. The JavaScript overview in this lecture should enable you to start programming your own function nodes. Through some additional examples, you’ve seen how function nodes are an important and powerful aspect of Node-Red, allowing you to extend it as needed when a built-in node doesn’t quite do the job. This lecture also gave an introduction to sub-flows, demonstrating how they can be used to reuse flow segments and package up function nodes for easier reuse in your flows. Finally you’ve seen the use of the function node to develop a flow that uses a Slack Bot to make general queries of the public version of the IoT platform WoTKit.

 


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

 

 

 


[1] Node-RED, although often referred to as a Flow based language actually isn’t. The classical definition of flow based languages requires features that Node-RED doesn’t provide.

[2] It is possible for a function or other node to send any JavaScript type to an output endpoint to downstream nodes, but by convention, they should send message objects containing properties.

[3] Interestingly the current version (0.11.1) of Node-RED doesn’t have a maximum number of outputs.

[4] Currently (version 0.11.1 of Node-RED) only function nodes have access to this type of object, but this may be extended to other nodes in future.

Node-RED: Lecture 4 – A tour of the core nodes

This lecture will take a look at the core set of nodes that Node-RED installs by default and then show you the extended set of nodes that the cloud-based Node-RED service – FRED – supports.

This lecture will take a look at the core set of nodes that Node-RED installs by default and then show you the extended set of nodes that the cloud-based Node-RED service – FRED – supports.

For each node, you’ll see a brief overview of its functionality and an indication of which example in this lecture series uses the node, so you can look in more detail at how to work with the node. This lecture is mostly a reference section. However, it is worth looking quickly through the nodes, so that you have an idea of the basic functionality available to you as you begin to craft your own flows.

Node-RED’s default set of nodes

When you install Node-RED for yourself on a device such as a Raspberry Pi or Beagleboard, it starts up with a default set of nodes. There are 8 main categories of nodes in the default install: inputs, outputs, functions, social, storage, analysis, advanced and Raspberry Pi (See Figs 4.1 and 4.2) Let’s take a look at each category in turn.

 

Figure 4.1 The default input, output, function and social media nodes.

Input nodes

There are 7 basic input nodes installed by default. They cover the basic communications mechanisms that IoT applications are likely to use. Ranging from lower-level internet protocols such as UDP and TCP through to the higher-level HTTP and the publish/subscribe MQTT.

Node name Description Examples
inject Injects a timestamp or user-configured text into a message. Can be configured to inject manually, at a set interval, or at specific times (using Cron). Examples 2.1, 3.6, 3.7, 3.8, 5.1-5.4, 6.1-6.8
catch Catches errors thrown by nodes on the same tab. If a node throws an error whilst handling a message, the flow will typically halt. This node can be used to catch those errors returning a message with an error property detailing the error and the source node and type. Examples
mqtt Subscribes to an MQTT broker and listens on a topic, returns any data published on the topic as a new message. Supports Quality of Service levels and last data retention. Examples 3.1-3.5
http Receives HTTP requests, allowing Node-RED to act as a basic web server. HTTP body is delivered as an output message along with any response. Message can contain standard URL-encoded data or JSON. Example 1.3, 5.7
websocket Provides an endpoint for a browser to establish a websocket connection with Node-RED. Offers a duplex connection for browser/server combinations. Example 3.7
tcp Used to accept incoming TCP requests on a specified port or to connect to a remote TCP port. Generates messages containing the TCP data as a single – or stream of – buffer, string or base64 encoded. Examples lecture 7
udp Used to accept incoming UDP packets (or multicast packets) on a specified port. Generates messages containing the UDP data as a BUFFER, string or base64 encoded string. Examples lecture 7
serial in Reads from a serial port on the local device. Can be configured to read buffers, a specific time period or wait for line breaks. Examples lecture 7

Output nodes

The output nodes are essentially the mirror images of the basic set of input nodes and provide a way to send data on the same set of protocols, i.e. mqtt, http, udp etc.

Node name Description Examples
debug Provides a simple way to view messages which are displayed in the debug pane. Can be configured to display just the msg.payload or the entire msg object. Various, examples 2.1, 2.2, 3.1-3.6, 5.1-5.4, 6.1-6.5
mqtt Subscribes to an MQTT broker and posts any data (msg.payload) it receives in incoming messages to a topic. Supports Quality of Service levels and last data retention. Example 3.6
http Sends responses back to HTTP requests received from a HTTP Input node. The response body is determined by msg.payload, and can have headers and status code defined. Examples
websocket Sends msg.payload out on the websocket configured. If msg._session is defined, sends to the origination client, otherwise broadcasts to all connected clients Example 3.7
tcp Replies to a configured TCP port. Can also be used to send to a specific port. Example 3.8
udp Sends a UDP message to the configured host (ip address) and port. Supports broadcast. Like most nodes, configured through UI or message properties. Examples
serial out Sends to the defined serial port. Can be configured to send an optional newline after any message payload. Examples lecture 7

 

Function nodes

The function category contains a variety of nodes that carry out specific processing functions. These range from the simple delay and switch nodes to the programmable function node that can be adapted to almost any programming need.

Node name Description Examples
function Generic programmable function node. Using standard JavaScript, the node can be tailored to carry out complex processing on its input messages generating one or more output messages. Examples 2.1, 2.2, 3.8, 5.1-5.4, 5.7, 6.1-6.8
template Configured with a template (using moustache format) of arbitrary complexity, this node takes an input message containing name:value pairs and inserts into the template. Useful for constructing messages, HTML, config files, etc. Example 1.3
delay A generic node that delays messages by a specific or random time. Can also be configured to throttle a message flow (e.g. 10 msg per sec). Examples 5.6, 6.8
trigger Creates two output messages separated by a configurable time interval whenever an input message is received. Can also be used as a watchdog timer. Example 1.1
comment A simple visual comment configured with title and body. Example 2.1
http request Allows you to construct and send a HTTP request to a specific URL. Method (PUT, GET, etc), headers and payload are all configurable through the UI or programmatically. Examples 1.3, 6.1, 6.6
tcp request A simple TCP request node. It sends the msg.payload to a server tcp port and expects a response. Can be configured to wait for data, wait for a specific character, or return immediately. Example 3.8
switch This node routes messages based on their properties. Properties are configured using the UI and can be a variety of logic (>, <, >= etc) applied to a message property. Examples 3.2-3.5, 5.5
change The change node can be used to set, change or delete properties of incoming messages. A variety of configurable rules allow complex changes including search and replace in the msg.payload Examples  3.3-3.5

 

range A simple scaling node that will map numerical input to a new output. Useful for converting or bounding ranges of input values, e.g. temperature. Undefined for non-numeric data. Example 3.5

 

Node name Description Examples
csv This node parses msg.payload and tries to convert to/from CSV. If it receives a string, it outputs a JavaScript object, and if it receives a JavaScript object, it outputs a CSV string. Examples
html Extracts elements from an html document in msg.payload using a configurable selector (CSS selector syntax). Essentially allows you to parse out the HTML and returns an array of the elements that match. Example 6.1
json This node converts to/from a JSON object. If it receives a JavaScript object, it outputs JSON, and if it receives JSON, it outputs a JavaScript object. Examples 3.1-3.5, 5.7, 6.6
xml This node converts to/from XML format. If it receives a JavaScript object, it outputs an XML string, and if it receives an XML string, it outputs a JavaScript object. Examples
rbe Report By Exception node. Generates a message only when its input is different from the previous input (string or number) or if the input has changed by a configurable amount (deadband mode) – only for numbers. Example 3.4

Social nodes

The basic social media nodes support interaction with email and with Twitter. They enable flows to send or receive email, or to send or receives tweets.

Node name Description Examples
email in Can be configured to repeatedly read from an IMAP server returning new email as it arrives. Sets msg.topic to email subject and either msg.payload to email text body or msg.html if the email is HTML. Example
twitter in Returns tweets as messages. Can be used to search the public or a user’s stream for tweets containing the configured search term or all tweets by specific users or direct messages received by the authenticated user. Example 1.1
email out Sends the incoming message as an email via the configured IMAP server. Topic and recipient all configurable. Will convert binary data to an attachment. Example 1.2
twitter out Tweets the msg.payload on the configured account. Can send direct messages and will send binary data as an image. Example 2.2

 

Storage nodes

The default node set for storage, because it is targeted at devices such as the Raspberry Pi are quite limited and focus on file based storage.

Figure 4.2 The default storage, analysis, advanced and Raspberry Pi nodes.

 

You should note that FRED, because it’s a cloud service, doesn’t support the basic file nodes. Instead it replaces these with a variety of storage nodes ranging from Mongo to Dropbox. However, for completeness the default nodes are covered here, so that you can use them, if you install Node-RED yourself (see lecture 7).

Node name Description Examples
tail Tails (i.e. watches for things to be added) to the configured file. (Linux/Mac ONLY)

This won’t work on Windows file systems, as it relies on the tail -F command.

Examples
file in Reads the specified file and sends the content as msg.payload, and the filename as msg.filename.

The filename can be configured in the node. If left blank, it should be set on msg.filename in an incoming message.

Examples
file Writes msg.payload to the file specified, e.g. to create a log. The filename can be configured in the node. If left blank, it should be set on msg.filename in an incoming message.  The default behaviour is to append to the file. This can be changed to overwrite the file each time; for example, if you want to output a “static” web page or report. Examples

Analysis nodes

Analysis nodes perform standard analyses on incoming messages. In the default node set, the only node provided is the sentiment node, which can be used to try and determine the sentiment of an incoming message based on the words used in the message, for example, an email or tweet.

Node name Description Examples
sentiment The sentiment node analyses the msg.payload and scores the sentiment of the message based on word analysis. It adds a msg.sentiment object that contains the resulting AFINN-111 sentiment score as msg.sentiment.score. Score typically ranges from -5 to +5. Example 5.5

Advanced nodes

A set of miscellaneous nodes offering various types of functionality.

Node name Description Examples
watch Watches a directory or file for changes. You can enter a list of comma-separated directories and/or files. You will need to put quotes “…” around any that contain spaces. On Windows, you must use double backslashes \\ in all directory names.

The full filename of the file that actually changed is put into msg.payload, while a stringified version of the watch list is returned in msg.topic.

msg.file contains just the short filename of the file that changed. msg.type has the type changed, usually file or directory, while msg.size holds the file size in bytes.

Examples
feedparse This node monitors an RSS/atom feed for new entries and delivers new entries as messages. It can be configured to query the feed at a specific interval. Examples
exec Calls out to a system command and provides 3 outputs: stdout, stderr, and return code. By default uses exec(), which calls the command, blocks while waiting for completion, and then returns the complete result in one go, along with any errors. Examples

 

Raspberry Pi nodes

Node name Description Examples
rpi_gpio in Raspberry Pi input node. Generates a msg.payload with either a 0 or 1, depending on the state of the input pin. You may also enable the input pullup resistor or the pulldown resistor.

The msg.topic is set to pi/{the pin number}

Requires the RPi.GPIO python library version 0.5.8 (or better) in order to work.

Note: we are using the actual physical pin numbers on connector P1 as they are easier to locate.

Examples lecture 7
rpi_gpio out Raspberry Pi output node. Expects a msg.payload with either a 0 or 1 (or true or false). Will set the selected physical pin high or low, depending on the value passed in. The initial value of the pin at deploy time can also be set to 0 or 1. When using PWM mode, expects an input value of a number 0 – 100.

Requires the RPi.GPIO Python library version 0.5.8 (or better) in order to work.

Examples 1.1, lecture 7
rpi_mouse Raspberry Pi mouse button node. Generates a msg.payload with either a 1 or 0 when the selected mouse button is pressed and released. Also sets msg.button to the code value, 1 = left, 2 = right, 4 = middle, so you can work out which button or combination was pressed. Examples lecture 7

 

The extended FRED node set

The FRED service adds a number of nodes to the standard default set. These new nodes, which have either been written for FRED or collected from the public repository, have been added, as they offer useful functionality that expands the capabilities of the vanilla node set.

Note: these nodes are available in FRED by installing them using the FRED installation panel. For a quick example of how to install nodes check out this tutorial.

As you will see, the majority of these additional nodes focus on services and capabilities that match well with FRED’s nature, i.e. a cloud-based service. In many cases they focus on using Node-RED for web based integration or to access enterprise level services.

Screen Shot 2015-10-09 at 3.03.41 PM.png

 

Figure 4.3 The extended FRED nodes

 

Extended set of social media nodes

As you can see, the FRED service adds a large number of social media nodes ranging from Pushbullet to Slackbot to Instagram.

Node name Description Examples
pushbullet in Connects to the popular Pushbullet service and receives Pushbullet data items from all your connected devices. Supports data, links and files. Examples
pushbullet out Allows you to send Pushbullet messages to all devices that have the Pushbullet app installed. Examples
XMPP in Receives messages from an XMPP instant messaging server. The buddy field indicates the buddy or room you are receiving from. Presence information is delivered on a second output link. Examples
XMPP out Sends messages to an XMPP instant messaging server. Uses topics to send to rooms (channels) and supports presence notification. Examples
slack A general node that provides a simple way to post on a slack channel specified via its webhook URL.  Can be configured with user name, emoji and to support attachments. Examples
slackbot in A node to use a bot you’ve created on Slack and provides a listener in any channel of which the Slack bot is a member. Outputs the msg.payload as the incoming message. and outputs msg.SlackObj with full Slack message details. Examples 5.7
slackbot out A node to use a bot you’ve created on Slack Sends the msg.payload to Slack based on the Bot API token provided. You can optionally override the destination channel if required – either in the edit dialogue or by setting msg.channel. Examples 5.7
delicious A node to save bookmarks to your Delicious account. The payload should contain the URL to save, and msg.title contains the bookmark name. An optional description field can be set. Examples
pinboard A node to save bookmarks to your Pinboard account. The payload should contain the URL to save, and msg.title contains the bookmark name. An optional description field can be set. Examples
flickr Saves photos to the configured Flickr account. msg.payload requires a Buffer with the image and can set the optional title, description and tag properties. Examples
foursquare Queries your Foursquare account for venues that meet a configurable set of requirements based on your location. Results can be passed back as a single message or a configurable set of messages. Examples
swarm in A node to poll every 15 minutes for Foursquare storm check-ins. Returned as a JSON object. Examples
swarm out Swarm query node that can be used to search for all Swarm check-ins by the authenticated user. Examples
instagram in Queries the configured Instagram account every 15 minutes for new photos which are delivered one per message, either as Buffer objects or URLs. Examples
instagram Same as Instagram in node, except from being triggered by an incoming message. Examples

Extended set of storage nodes

As you can see, the FRED service adds a large number of social media nodes ranging from Pushbullet to Slackbot to Instagram.

Node name Description Examples
amazonS3 watch Amazon S3 watch node. Watches for file events. By default, all file events are reported, but the filename pattern can be supplied to limit the events to files which have full filenames that match the glob pattern. The event messages consist of the full filename in msg.payload property, the filename in msg.file, the event type in msg.event. Examples
amazonS3 in Amazon S3 input node. Downloads content from an Amazon S3 bucket. The bucket name can be specified in the node bucket property or in the msg.bucket property. The name of the file to download is taken from the node filename property or the msg.filename property. The downloaded content is sent as msg.payload property. If the download fails, msg.error will contain an error object. Examples
amazonS3 out Amazon S3 out node. Uploads content to an Amazon S3 bucket. The bucket name can be specified in the node bucket property or in the msg.bucket property. The filename on Amazon S3 is taken from the node filename property or the msg.filename property. The content is taken from either the node localFilename property, the msg.localFilename property or the msg.payload property. Examples
box watch Box is an enterprise version of Dropbox. This node watches for file events on Box. By default all file events are reported, but the filename pattern can be supplied to limit the events to files which have full filenames that match the glob pattern. The event messages consist of the full filename in msg.payload property, the filename in msg.file, the event type in msg.event and the full event entry as returned by the event API in msg.data. Examples
box in Box input node. Downloads content from Box. The filename on Box is taken from the node filename property or the msg.filename property. The content is sent as msg.payload property. Examples
box out Box out node. Uploads content to Box. The filename on Box is taken from the node filename property or the msg.filename property. The content is taken from either the node localFilename property, the msg.localFilename property or the msg.payload property. Examples
dropbox watch Watches for file events on Dropbox.

By default all file events are reported, but the filename pattern can be supplied to limit the events to files which have full filenames that match the glob pattern.

The event messages consist of the full filename in msg.payload property, the filename in msg.file, the event type in msg.event and the dropbox.js API PulledChange object inmsg.data.

Examples
dropbox in Dropbox input node. Downloads content from Dropbox. The filename on Dropbox is taken from the node filename property or the msg.filename property. The downloaded content is sent as msg.payload property. If the download fails, msg.error will contain an error object. Examples
dropbox out Dropbox out node. Uploads content to Dropbox.

The filename on Dropbox is taken from the node filename property or the msg.filename property. You can pass in content either as a filename by setting the localFilename field or msg.localFilename property, or you can pass in content directly using msg.payload.

The file will be uploaded to a directory on Dropbox called Apps/{appname}/{appfolder}, where {appname} and {appfolder} are set when you set up the Dropbox application key and token.

Examples
MongoDB in Calls a MongoDB collection method based on the selected operator.

Find queries a collection using the msg.payload as the query statement as per the .find() function. Count returns a count of the number of documents in a collection or that match a query using the msg.payload as the query statement.

Aggregate provides access to the aggregation pipeline using the msg.payload as the pipeline array.

You can either set the collection method in the node config or on msg.collection. Setting it in the node will override msg.collection.

Examples
MongoDB out A simple MongoDB output node. Can save, insert, update and remove objects from a chosen collection.

Save will update an existing object or insert a new object, if one does not already exist.

Insert will insert a new object.

Update will modify an existing object or objects.

Remove will remove objects that match the query passed in on msg.payload. A blank query will delete all of the objects in the collection.

Examples
mysql Allows basic access to a MySQL database.

This node uses the query operation against the configured database. This does allow both INSERTS and DELETES. msg.topic must hold the query for the database, and the result is returned in msg.payload.

The returned payload will typically be an array of the result rows. If nothing is found for the key, then null is returned.

Examples
postgres A PostgreSql I/O node.

Executes the query specified in msg.payload with optional query parameters in msg.queryParameters. The queryParameters in the query must be specified as $propertyname.

When receiving data from the query, the msg.payload on the output will be a JSON array of the returned records.

Examples

FRED’s IoT nodes

FRED adds a number of nodes to access popular IoT platforms, including Sense Tecnic’s Community edition of its popular IoT platform, WoTKit, and Bug labs’ dweet.io.

IoT platform nodes
wotkit in WoTKit Sensor Input Node which retrieves new data from a WoTKit Sensor. The node generates a message each time new data is received on the sensor within the WoTKit IoT platform. The message.payload contains a map of sensor field names to sensor values. The node is configured with WoTKit user credentials. The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id. Examples
wotkit out WoTKit output node to send data to a registered WoTKit sensor. The node will use the message.payload to create a data object.

The message.payload must contain an object of key-value pairs matching the sensor fields. The node is configured with WoTKit user credentials.The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id.

Examples
wotkit data WoTKit Historical Data Node which retrieves historical data from a WoTKit sensor by either number of elements or relative time before the request. The message.payload contains a map of sensor field names to sensor values. The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id. Examples
wotkit control out The WoTKit Control Output Node provides the ability to send events to the control channel of a registered WoTKit sensor/actuator. This node will use the message.payload object to create a control event. The message.payload must contain an object of key-value pair for each message. The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id. Examples
wotkit control input The WoTKit Control Input Node provides access to the control channel of a registered WoTKit sensor/actuator. When a control event is received by a WoTKit sensor this node will create a message.payload object containing the event. The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id. Examples
dweetio in Listens for messages from Dweet.io

The Thing ID should be globally unique, as they are all public. It is recommended you use a GUID. The Thing ID is set into msg.dweet, and the TimeStamp into msg.created.

Examples
dweetio out Sends the msg.payload to dweet.io

Optionally uses msg.thing to set the Thing ID, if not already set in the properties.

You need to make the Thing ID unique – you are recommended to use a GUID.

Examples

Extended set of analysis nodes

The FRED platform only adds a limited number of analysis nodes as the default set is quite extensive.

Node name Description Examples
smooth A simple but flexible node to provide various functions across several previous values, including max, min, mean, high and low pass filters. Only works on numbers and will fail if it can’t convert the input to a number. Examples
wordpos Analyses msg.payload and classifies the part-of-speech of each word, returning msg.pos as an array of nouns, verbs, adjectives, adverbs, etc.. Examples

 

Extended set of advanced nodes

FRED adds the following advanced nodes.

Node name Description Examples
ping Pings a machine and returns the trip time in MilliSeconds.

Returns false, if no response received within 5 seconds, or if the host is unresolveable.

Default ping is every 20 seconds, but can be configured.

 

Examples
sunrise Uses the suncalc module to generate an output at sunrise and sunset based on a specified location. Several choices of definition of sunrise and sunset are available.

The first output emits a msg.payload of 1 or 0 every minute, depending if in between selected times or not. The second output emits only on the transition between night to day (-> 1) or day to night (-> 0).

Also sets msg.topic to sun and msg.moon to the fraction of the moon between 0 and 1.

Examples

Other nodes

The following nodes are available in FRED and don’t fit into the basic categories that vanilla Node-RED provides. These include nodes to access and control Google services, enterprise nodes such as Salesforce access, nodes for accessing transportation API (e.g. Transport for London) and other miscellaneous nodes for access to weather services, etc.

 

Fig 4.4 FRED’s other nodes

 

Google nodes

FRED supports several Google nodes to interact with a range of Google services.

Node name Description Examples
Google plus Interacts with the Google+ API to get information about people, activities, and comments.

People – Allows you to interact with Google+ profiles. You can get a particular profile, search for a profile, or gather a list of people that have +1’d or reshared an activity.

Activities – Allows you to interact with Google+ activities. You can get a particular activity, search for an activity, or gather a list of activities directly related to a person.

Comments – Allows you to interact with Google+ comments. You can get a particular comment, or gather a list of comments attached to an activity.

Examples
googleplaces A highly flexible node that utilizes the Google Places API in order to find and learn more about local establishments. Search can be based on ranking, radius, price, keywords, language, etc. Examples
Google calendar in A node to watch a calendar and return a message before, at or after the event in the calendar. Configurable to set exactly when the message is generated. Examples
Google calendar out Create an entry in a Google Calendar based on:

payload – either a string to describe the event using quick add format or an object representing the request body for an insert request

calendar – the calendar to which the event is added (optional, defaults to the node calendar property or the user’s primary calendar)

sendNotifications – a Boolean to determine if notifications should be sent to attendees (optional, defaults to false)

Examples

Fitness nodes

The FRED fitness nodes cover a number of popular fitness devices.

Node name Description Examples
strava Get your most recent activity on Strava.

This node returns the most recent activity in the authenticated user’s account whenever it receives a message. Returns the activity, location and time when available.

Examples
Fitbit in Polls Fitbit for new data at regular intervals. The generated messages are determined by the nodetype property (goals, sleep, or badges). Examples
Fitbit activities Retrieves user data from Fitbit and returns a msg.payload determined by nodetype properties (as fitbit in).

The msg.date property may be set to an ISO 8601 format date (e.g. 2014-09-25) to retrieve historical data for activities and sleep log. If no date is supplied, then data for today will be retrieved. In the case of sleep, this is the data for the preceding sleep.

Examples
jawbone The Jawbone Up node can be used to retrieve the workouts completed since the provided time (given as Epoch). This time can be passed in as settings on the node or as the msg.starttime section of the message input. The value set on the node will take precedence over the contents of incoming message. Examples

Weather nodes

FRED adds a set of nodes to give access to a variety of weather services.

Node name Description Examples
openweathermap A node to query the openweathermap.com site for weather information of a city/country or lat/long pair. Two nodes exist: one that is UI-configured and one that can accept the configuration info as an input message. Examples
forecastio A node to query the forecastio.com site for weather information of a lat/long pair. Two nodes exist: one that is UI-configured and one that can accept the configuration info as an input message. Examples
wunderground A node which queries The Weather Underground API for current weather data periodically and returns when a change is detected. Two nodes exist: one that is UI-configured and one that can accept the configuration info as an input message. Examples

Salesforce nodes

A set of experimental SalesForce nodes are available with FRED.

Node name Description Examples
salesforce A set of 5 nodes that interact with the popular salesforce.com service. The nodes can generate SOSL queries, DML statements, subscribe to the salesforce streaming API or parse outbound salesforce message objects. Examples

Transport nodes

Node name Description Examples
TfL bus Get live bus departure/arrival info for London (UK) buses and river buses.

This node enables the user to get bus or river bus arrival information for selected lines arriving at selected stops. The node returns the first vehicle/vessel to arrive at a particular stop. The data is provided by Transport for London.

Examples
TfL underground A node to get the underground line status for the London Underground. It returns a variety of status information for the specified line including overall status, disruption info etc Examples

Formatting nodes

Node name Description Examples
moment moment is a date/time formatter node that takes as input either a JS datetime object or a string that moment can resolve to such. If the input is null, doesn’t exist or is a blank string, the current date/time will be used. This can be used to add a current timestamp to a flow of any kind easily.

Output is a formatted string or a date object onmsg.payload by default, change by setting the output field.

Examples

Summary

In this lecture, you were presented with a summary of the default Node-RED nodes available on installation, and the extended set of nodes that FRED adds. As you have seen, there is a wide variety of nodes allowing you to build complex flows with little or no programming.

You’ve also seen that not all nodes are always available, for example the Raspberry Pi nodes don’t work unless you are actually running Node-RED on a Pi. So it always pays to check that you can run the nodes you need in your environment.

However, given the large number of nodes available, and the fact that the community is creating new nodes on a daily basis, you are likely to be able to find a node that meets your needs. If not, you can fall back on the flexible function node, or even create your own, something that will be discussed in the advanced lectures.


About Sense Tecnic: Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures, and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


 

© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

Node-RED: Lecture 3 – Basic nodes and flows

In this lecture you will get to see a few of the more commonly used nodes and build on some of the things you learnt in previous lectures. You’ll start off with a series of examples based around the popular MQTT protocol that shows how to wire together a set of basic but very useful message processing nodes. Then you’ll briefly look at other ways to get messages into and out of your flows using protocols such as TCP, UDP and websockets.

In this lecture you will get to see a few of the more commonly used nodes and build on some of the things you learnt in previous lectures. You’ll start off with a series of examples based around the popular MQTT protocol that shows how to wire together a set of basic but very useful message processing nodes. Then you’ll briefly look at other ways to get messages into and out of your flows using protocols such as TCP, UDP and websockets.

At the end of this lecture you will have a better understanding of some of the basic nodes used in Node-Red flows. You will also have seen how easy it is to build sophisticated processing flows that take real-world events, process them and generate responses that use popular Internet protocols to communicate the results outside of your flow.

 

We use a cloud hosted version of Node-RED for these lectures called FRED. Sign up for a free account at FRED. Examples in the early lectures will work with other installations of Node-RED, later lectures use nodes you will need to install yourself if you don’t use FRED.

 

Examples

Example 3.1 Receiving JSON via an MQTT message

Example 3.2 Using the switch node to handle a JSON object

Example 3.3 Using a change node to change or manipulate a message payload

Example 3.4 Using the rbe (report by exception) node

Example 3.5 Scaling input with the range node

Example 3.6 Using an mqtt output node to test the flow

Example 3.7 Using Websockets with Node-RED

Example 3.8 Sending TCP requests.

Example 3.1 Receiving JSON via an MQTT message

The following series of examples builds on the mqtt node, which provides a convenient way to take input from an MQTT broker. For those not familiar with MQTT, it is an example of a publish/subscribe system (usually shortened to pub/sub system) which lets sensors publish updates that all delivered to client subscribed to that sensor. MQTT uses a topic model allowing publishers (eg sensors) to create topics and publish data to the topics Equally, others can subscribe to a topic and will receive asynchronous notification of data posted to the topic.

Pub/Sub systems are a great way to connect loosely coupled distributed systems and they map well to typical IoT patterns where devices or things generate events that you want to share. The MQTT protocol, apart from being asynchronous, is also lightweight and doesn’t have as high an overhead as HTTP; which for resource-constrained devices is often an important advantage. MQTT was originally developed in the late 1990s and has been used in a variety of IoT settings. MQTT became an OASIS standard in 2014 and is a standard part of many IoT toolboxes. MQTT actually stands for Message Queueing Telemetry Transport.

To use the mqtt node, you need to have access to a broker. There are a number of free MQTT servers running, for example http://test.mosquitto.org/, or the one that will be used in this lecture, www.hivemq.com. Using the broker address and the topic, you can configure the mqtt input node to subscribe on that topic, causing it to generate a new message whenever new data is published on that topic. The message will contain information on the published data, including the data itself in msg.payload and the MQTT broker topic in msg.topic.

To get you started with the mqtt node, you’ll be using the free mqqt broker hivemq – which is available via (http://www.hivemq.com/showcase/public-mqtt-broker/). Of course you can use any MQTT broker, including your own, if you have installed one.

First, drag and drop an mqtt input node and configure it for the broker. Don’t forget to configure the topic to something unique, in the case of this example we are using noderedlecture/sensor but you should use your own unique topic, i.e. <your name here>/sensor

Figure 3.1 Configuring the mqtt node with broker address and topic

There are many ways you can send mqtt messages to hivemq. You can use their websockets client showcase (http://www.hivemq.com/demos/websocket-client/), the mqtt dashboard (http://www.mqtt-dashboard.com/dashboard) or your own library. You’ll be using their websocket client in this example, so navigate to that page and connect to the broker. You will publish a JSON encoded string to the topic you configured to see both the use of the mqtt node and the json node.

Figure 3.2 Sending an MQTT message (publishing) using the HiveMQ client page.

Since you are sending a JSON string, you will need to parse the message that the mqtt node generates when it receives the MQTT message. To do that, you’ll need to drag and drop a json node and connect it to the mqtt node’s output.

Node-RED’s json node is a sort of convenience function, in that it parses the incoming message and tries to convert it to/from JSON. So if you send it a JSON string, it will convert it to a JavaScript object, and vice versa.

If you wire up the usual debug node to the json node and deploy, then use the HiveMQ dashboard to send the JSON string {“analyze”:false, “value”:10} as shown in Fig 3.2. You will see it printing in the debug tab (Fig 3.3).

 

Figure 3.3 Receiving and parsing an MQTT message sent as a JSON string

If you look closely at the output, you can see that the msg.payload contains an object, which itself has two fields, analyze and value, each with their own values. As you saw in lecture 2, you can access these fields via msg.payload.analyze and msg.payload.value. Let’s take a look at a node that can do that.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-1_mqqtmessages.json

Example 3.2 Using the switch node to handle a JSON object

One of the nice features of having a JSON object is that you can easily act on its properties. A useful node for this is the switch node. Its role is to ‘switch’ or route messages depending on the incoming message properties.  For example, you can check the msg.payload.analyze property and, depending on its value (true/false), decide to route a message to one of the switch node’s outputs.

Drag a switch node and double-click on it. Configure it to evaluate the property “msg.payload.analyze”. If true, send the message to the first output; if false, send it to the second output as shown in in Fig 3.4.

Figure 3.4 Configuring the switch node to route based on a message property

Now you can connect two debug nodes as shown in Fig 3.5 – when you set up multiple outputs for a node, they are numbered from the top, so output 1 is the top output and output 2 is at the bottom in  Fig. 3.5.

Figure 3.5 Connecting a switch node to two debug nodes

If you now go back to the HiveMQ input page and send the MQTT message {“analyze”:true, “value”:6}, you will see that the first (top) output is activated and the incoming messages is routed, or ‘switched’, to output 1. If you send the original message {“analyze”:false, “value”:10}, the switch node will activate output 2 and the original debug node will fire. Hovering the pointer over the debug message will show which debug node is printing out the message as shown in Fig. 3.6.

 

Figure 3.6 Confirming the operation of the switch node

As you can see, this provides you with a built-in Node-RED node that allows you to quickly determine the contents of incoming messages and direct the message through to different parts of the flow depending on the input.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-2_switchnode.json

Example 3.3 Using a change node to change or manipulate a message payload

Another useful node is the change node, which will allow you to change a message payload or add new properties. You can use this node to affect the properties in a message, either by changing existing ones, deleting them or adding new properties.

In this example, you’ll continue with your MQTT theme and see how, now that you have successfully ‘switched’ the message flow based on the incoming MQTT message, you can add a new message property msg.payload.note.

First, let’s drag and drop a change node and connect it to the second output of the switch node (Fig 3.7). As you will remember, this is the output that fires when msg.payload.analyze is set to false.

Figure 3.7 Adding in a change node to set a new message property.

 

Now configure it to set the property msg.payload.note to “this is not being analyzed” as shown in Fig 3.8.

 

Figure 3.8 Using a change node to add a new message property

 

When you receive a message that the switch node sends on the 2nd output, it will be modified to contain a “note” element with the string “this is not being analyzed”. If you deploy and test the flow by sending the MQTT message from HiveMQ, you’ll see the output as shown in Fig 3.9.

 

Figure 3.9 The result of switching and then changing the message

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-3_changenode.json

Example 3.4 Using the rbe (report by exception) node

In this example, you’ll continue your message analysis theme and add nodes to the part of the flow that is used when you determine that the flow should be analyzed. You’ll be using the rbe (report by exception) node which only passes on data if it has changed. You can set it to examine a message payload and either block until a message changes (rbe mode) or when a messages changes by a specified amount (deadband mode). In rbe mode, it works on numbers and strings. In deadband mode, it works on numbers only and uses the configured deadband as a + or – ‘band’, so that the incoming value can fluctuate within a range before it fires.

You’ll start by adding in another change node which you’ll connect to output 1 of the switch node. You’ll then connect an rbe node to the switch node as shown in Fig 3.10.

Let’s connect a change node, and an rbe node like this. To remind us that this output deals with the flag “analyze”, add a comment node and write “Analyze = true”. Comments are useful when writing complex flows.

Figure 3.10 Adding an rbe node to check if our input data has changed by more than 20%

Edit the change node to set the msg.payload to msg.payload.value. This will set the output of this node to the value found in the msg.payload.value element of the input received (Fig 3.11).

Figure 3.11 Using a change node to set payload

Since you want to determine if this value has changed by 20% or more, you’ll need to double-click on the rbe node and configure it to block unless the value changes by more than 20%.

Fig 3.12 Setting the rbe node to check msg.payload.value

To test the flow, deploy this flow and then return to the HiveMQ page and send a series of messages. Firstly, you’ll need to set the analyze value to true so that the switch node sends through a message on output 1. If you use the original message value of 6, this will fail to pass the rbe node. If you then send a second message of value 10, the rbe node will evaluate the difference between 6 and 10, see it is greater than 20%, and send on a message to the final debug node which will print on the debug pane as shown in Fig 3.13.

Fig 3.13 Confirming that 10 is more than 20% larger than 6 using an rbe node.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-4_rbenode.json

Example 3.5 Scaling input with the range node

When dealing with real world input from sensors and other devices, an ability to scale input data is often required. Node-RED provides the scale node to support this and allows you to scale (linearly) an input value. Let’s assume you want to scale your value (originally in the range 0-10) to a range (0-255) when you aren’t doing any analysis. This means we are dealing with the lower part of the flow fired when the switch node evaluates the Analyze property as false.

To do this, select the change node you configured above (set msg.payload) and copy it with ctrl+c, then ctrl+v. Attach a range node as shown in Fig 3.14.

 

Figure 3.14 Scaling input values using the range node.

Double-click on it, and configure it to map the input from 0-10 to 0-255 as shown in Fig 3.15.

The scale node has three options set by the action field. The default will scale according to the mapping given but will happily scale values outside the given ranges, using the same mapping. Scale and limit to target range means that the result will never be outside the range specified within the result range. A third option, Scale and wrap within the target range means that the result will essentially be a “modulo-style” wrap-around within the result range.

Then return to the HiveMQ test page and post {“analyze”:false, “value”:10} as a new MQTT message to the same topic.

Figure 3.15 Setting the scaling ranges for input and output of the scale node

If you return to your Node-RED window, you will see that the debug node associated with the lower part of the flow has fired, showing that the msg.payload.value property that you set as 10 when you published it to MQTT, has been scaled up 255 as shown in Fig 3.16.

Figure 3.16 Final scaled output when analysis is turned off

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-5_rangenode.json

Example 3.6 Using an mqtt output node to test the flow

As an alternative to using the HiveMQ test page to publish on the MQTT topic, we can configure an mqtt output node. This is the mirror of the mqtt input node and allows you to configure an MQTT service and the topic you are publishing on. You can then send the node messages with the exact same JSON string we’ve been sending via the HiveMQ test page.

To try that, drag and drop three inject nodes and an mqtt output node as shown in Fig 3.17, and configure and wire up as shown

Figure 3.17 using an mqtt node to publish on the noderedlecture/sensor topic

You can now test the flow you created to analyze the MQTT message directly from the workspace by clicking on the three inject nodes in sequence. The first will exercise the part of the flow handling the case when analysis is turned off; the 2nd two messages will cause the path with the rbe node to be exercised.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-6_mqqtout.json

Example 3.7 Using Websockets with Node-RED

Websockets are another useful communication capability that is built into Node-RED via the the websocket node. Websockets provide a duplex TCP connection and were designed to allow web browsers and servers to maintain a ‘backchannel’ that could be used to augment traditional HTTP interactions, allowing servers to update web pages without the client making a new pull request.

The websocket node comes in two flavours, input and output, allowing you to listen for incoming data (input) or to send (output) on a websocket. The output version is designed to check to see if the output payload originated at a websocket in a node, in which case it responds to the original sender. Otherwise it will broadcast the payload to all connected websockets.

In addition, both input and output websocket nodes can be configured as either server or client – in server mode they ‘listen on’ a URL, and in client mode they connect to a specified IP address.

To see how the websocket nodes work, you’ll use a public websockets echo server which runs on the public site: (https://www.websocket.org/echo.html).

Drag an inject, websocket in, websocket out and a debug node onto the workspace and connect them as shown in figure 3.18.

Figure 3.18 using websockets to communicate with a Node-RED flow

Configure the inject node to send a string payload of “Hello There” (Fig 3.19)

Figure 3.19 Configuring an inject node to send on a websocket

Configure the websocket nodes to connect to wss://echo.websocket.org as shown in Fig 3.20.

Figure 3.20 Configuring the websocket to send to a public echo server. Do the same for the websocket out node.

Deploy. When you click on the inject node you will see the message printed out as shown in Fig 3.21

 

Figure 3.21 Output from a websocket listening for incoming data

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-7_websockets.json

Example 3.8 Sending TCP requests.

This example shows you how to send TCP requests using the tcp node. In this case you will make an HTTP request following the specifications in (http://tools.ietf.org/html/rfc2616#section-5.1.2).

This example shows the use of the tcp node. It could equally be configured with the udp or http nodes in a similar manner.

To get started, let’s connect an inject, function, tcp request, and debug nodes as shown in Fig 3.22.

Figure 3.22 building a TCP request and sending on a tcp output node

Edit the first function node to add a function that sets the string “GET / HTTP/1.1\r\n\r\nHost: www.google.com” as payload as shown in Fig 3.23.

This string is a standard HTTP request, indicating it is a GET request, the protocol is HTTP 1.1 and the host is www.google.com. The \r\n\r\n is two return/newline pairs which is required in the HTTP protocol.

Figure 3.23 Building a TCP request in a function node

Configure the tcp request node to connect to the www.google.com server, on port 80. Configure it to close the connection after 1 second (1000 ms) as shown in Fig 3.24.

Figure 3.24 Configuring the end-point for a TCP request

The tcp request node response is a buffer and needs to be parsed. Configure the second function node to parse the tcp request node response as shown in Fig 3.25

Figure 3.25. A function node to parse a response buffer into a string

If you deploy the flow and click on inject, you will make a request to Google and will get a TCP response. The debug node will print the response as a string as shown in Fig 3.26.

Figure 3.26 Printing out the response to a well formed HTTP request sent over a TCP connection.

Some of you may be wondering why you need to use a function node to build the HTTP request that we sent over TCP. Why not just use the inject node to input the string? The reason is that the inject node ‘escapes’ the string it uses, causing the return/newline you inserted to be removed. This in turn confuses the receiving server (Google) into not returning a response as it waits for the missing return/newlines. So instead, you build the string in a function node. This is one of those ‘gotchas’ that trips up even experienced Node-RED programmers, so always read the info pane for nodes to make sure you understand any limitations or constraints.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-8_tcp.json

Summary

In this lecture you have seen a series of small examples that use a number of the processing and communication nodes that are available in Node-RED. As you have seen, it is both easy and fast to wire together basic flows that take real-world input, carry out some processing, such as simple data analysis, and return results.

In these examples, you have done little or no coding, but still have been able to build quite complex programs – this is the power of Node-RED.

The next lecture is a quick summary of the basic set of nodes that are available in both vanilla Node-RED as well as the extended set that the FRED service provides. You can read through the lecture to understand the default capabilities, or you can treat the lecture as a reference and use it to look up examples of each node as used in this lecture series.

 

 


About Sense Tecnic: Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures, and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.