JavaScript primer

JavaScript Primer

While we don’t have enough space here to cover JavaScript completely, here is a quick overview for those familiar with other programming languages to get started writing your own nodes. To experiment with JavaScript, you can use the JavaScript console supplied with your favourite browser or startup node.js to run the Read-Eval-Print-Loop (REPL) on your machine.

Variables and Variable Assignment

Variables are declared as follows:

  1. var myFirstVar;

Declarations can be separated by commas on the same line:

  1. var myFirstVar, mySecondVar;

They can be assigned with some initial values using the equals operators.

  1. var myFirstVar = 0;

Types

JavaScript supports various types as follows:

Listing 5.1 The basic types supported by JavaScript

  1. var anInt = 0;
  2. var aFloat = 2.5;
  3. var aBoolean = true;
  4. var aString = “This is a string”;
  5. var anArray = [1,2,3];
  6. var anObjectLiteral = {
  7.     someField : “Some value”
  8. };
  9. var aFunction=function() { return; }; // function
  10. var re = /ab+c/;        // regular expression

You can instantiate an object using the new operator like this:

  1. var aDate = new Date();

Comments

Comments can use the double slash single line or the C-style multiline comment format:

Listing 5.2 Comments in JavaScript

  1. // this is a single line comment
  2. /*
  3.    This is a multi-line comment
  4. */

Basic Operators

The built-in numerical operators include the usual: + (addition), – (subtraction), / (division), * (multiplication) and % (modulo).

Listing 5.3 numerical operations in JavaScript

  1. var a = 6;
  2. a = a + 1; // a is 7
  3. b = b % 2; // b is 3
  4. a++; // a is 8
  5. a–; //a is 7
  6. a += 3; // a is 10

Strings can be concatenated as shown:

Listing 5.4 String concatenation in JavaScript

  1. var aString = “The value of a is ” + a;
  2. var greeting = “What a very “;
  3. greeting += “nice day”;

The typeof operator returns the type of a variable.

  1. typeof aString; // returns “string”

To determine the class of an object, for example to distinguish between arrays and other objects, the Object.toString method can be used as follows:

Listing 5.5 Determining a JavaScript object type

  1. Object.prototype.toString.call(new Date()); // “[object Date]”
  2. Object.prototype.toString.call([1,2,3]); // “[object Array]”
  3. Object.prototype.toString.call({someField:12}); //”[object Object]”
  4. Object.prototype.toString.call(function(){return;}); //”[object Function]”

Conditionals

You can use the comparison operators, < (less than) > (greater than) <= (less than or equal) >= (greater than or equal) == (equals) != (not equals) in if statements as follows:

  1. Listing 5.6 Conditional operators in JavaScript
  1. var a = 1;
  2. var b = null;
  3. if (a > 0) {
  4.   b = “a is greater than zero”;
  5. } else {
  6.   b = “a is zero or less”;
  7. }

To shorten this, one could use the conditional operator as follows:

  1. var b = (a > 0) ? “a is greater than zero” : “a is zero or less”;

Loops

Javascript supports for and while loops:

Listing 5.7 For and while loops in JavaScript

  1. for (var i = 0; i < anArray.length; i++) {
  2.     console.log(myArray[i]);
  3. }
  4. while (x < 10) {
  5.     console.log(x++);
  6. }

Functions

Functions are created like this:

Listing 5.8 Declaring a function in JavaScript

  1. function addOne (x) {
  2.     return x + 1;
  3. }

To call a function, you can call them as follows:

  1. var y = addOne(5);        // y = 6

Or, if the function is a method of an object, for example a Date object:

Listing 5.9 Calling a function in JavaScript

  1. var d = new Date();
  2. var year = d.getMonth();   // returns 0 for January, 1 for February,

Callbacks

Node.js is an event-based framework with only one thread of execution. To allow more than one task to execute while performing I/O or waiting for a timer to expire, callbacks are used extensively. For example, to log ‘done’ to the console after 2 seconds, you can use an anonymous function callback as follows:

  1. setTimeout(function() { console.log(‘done’); }, 2000);

You can use functions to process an array:

Listing 5.10 Arrays in JavaScript

  1. var friends = [“mike”,”roberto”, “rodger”, “ted”, “daniel”];
  2. friends.forEach(function (eachName, index){
  3.   console.log(index + 1 + “. ” + eachName); // 1. mike  2. roberto  3. rodger  4. ted  5. daniel
  4. });

Exception Handling

Exception handling can be used to catch errors in your code. To do so, wrap your code in a try/catch block as shown in Listing 5.11.

Listing 5.11 Handling exceptions in JavaScript

  1. try {
  2.     // try to do something
  3. } catch (e) {
  4.     // handle errors
  5. } finally {
  6.     // this block is always executed regardless of whether there was an exception
  7. }

That’s the end of this JavaScript overview. Those new to JavaScript should now have enough knowledge to get going. Let’s start out by writing some basic function nodes! Don’t forget, if you feel you don’t understand the JavaScript, it’s worthwhile taking a look at one of the many JavaScript primers available on the web.

Lecture 7: Node-Red dashboards (Part 3)

Example 7.3 Using a simple JavaScript chart library to build a custom dashboard

For our final example of building Dashboards and UIs we’ll use a generic technique we’ve experimented with before. That’s the web service approach that uses a http node to allow us to accept http requests and return web pages. We used this approach in example 1.3 but didn’t explain the details.

Sticking with the openweathermap node, let’s use it to generate a simple JSON structure that we then visualize using the popular Morris.JS chart library (). The library supports 4 basic chart types line charts, area charts, bar charts and the donut chart. We’ll be using the donut chart but the approach is the same for all the chart types.

The flow is shown below and consists of 4 nodes. The first and final nodes are http input and output nodes that work together to listen for HTTP requests and send HTTP responses. This flow listens for a HTTP request from any source, but let’s assume a regular browser here. When it arrives, it queries the weather using your openweathermap nodethen uses a template node to build a HTTP page using the weather data, and passes that to the http output node which sends back the webpage to the browser.

Screen Shot 2016-05-17 at 2.37.43 PM.png

Figure. A simple web server example to graph weather data using a pi chart

Double click on the http input node and configure it with your URL, e.g. /public/weather. Remember, to access it, you will need to use,

https://{user name}.fred.sensetecnic.com/api/public/weather

where {user name} is replaced by your own FRED user name. Obviously /weather is because that’s how you configured it, if you’d used another name, e.g. /public/test, then the URL would end /public/test

When a HTTP request comes in, the http input node creates a message to trigger the openweathermap node, which is the next node in the flow. The openweather node gets the current statistics for the location you configured it for, and then passes those statistics, as a JSON structure in the usual message payload field, to a template node. The html template node is another pre-built node in Node-RED, which, like the function node, allows you to build arbitrary code. However, rather than using JavaScript like the function node, the template node works with text such as HTML.

The HTML code in the template node is shown in the listing below. As you can see, the code is a standard HTML structure with the <head> structure defining the external script libraries that we use. These are all required for the Morris.JS chart library and should be copied exactly.

Note, we are combining all the page elements into the one file (or node) which works for simple web pages but is not considered good practice.

Line 10 sets up a <div> and sets the name and height. Then the morris.js chart code  script begins at line 11 by defining a Morris.Donut element.

Listing  A simple HTML template to display a donut chart of calorie usage

  1. <!doctype html>
  2. <head>
  3.     <title>A Node RED Example</title>
  4.     <link rel=”stylesheet” href=”//cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.css”>
  5.     <script src=”//cdnjs.cloudflare.com/ajax/libs/raphael/2.1.0/raphael-min.js”></script>
  6.     <script src=”//ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js”></script>
  7.     <script src=”//cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.min.js”></script>
  8. </head>
  9. <html>
  10.     <div id=”chart-example” style=”height: 250px;”></div>
  11.     <script>
  12.         Morris.Donut({
  13.            element: ‘chart-example’,
  14.            data: [
  15.              {label: “Temperature (Celcius)”, value: {{payload.tempc}} },
  16.              {label: “Humidity”, value: {{payload.humidity}} },
  17.              {label: “Wind Speed (knts)”, value: {{payload.windspeed}} }
  18.           ]
  19.        });
  20.     </script>
  21. </html>

Lines 15, 16 and 17 define the donut elements, you are using 3 to display the temperature, the humidity and the wind speed. The values from the openweathermap JSON structure can be used directly, i.e. payload.tempc, payload.humidity and payload.windspeed.

Once the template node has generated the HTML file, it passes it as a message to the final node, which is a http response node. This node packages up the HTML as a HTTP response which it sends back to the browser that made the original request.

If you deploy this flow and then point your browser to the URL you configured in the http input node, in your case should be:

https://{user name}.fred.sensetecnic.com/api/public/weather

You will then see a simple donut chart which you can mouse over to see the temperature, humidity and windspeed for the city you configured in the openweathermap node – all built and served by a simple Node-RED flow!

Screen Shot 2016-05-17 at 2.59.31 PM.png

Figure: A donut chart, served up by Node-RED showing Temperature for the configured city

Summary

In this lecture you have looked at three approaches to building simple dashboards and UIs. The FreeBoard node is a one stop shop allowing you to feed complex data as JSON structures to the Freeboard node and then using the powerful FreeBoard dashboard to configure and show the data.

With the nodered.contrib.ui nodes you have to do a little more work by having to build the UI elements into your flows. This approach gives you more control, and makes the UI elements part of your Node-RED flow, but requires a little more thought and work.

The final approach, using the html template node and a generic JavaScript charting library is the most flexible, but requires you know how to write simple html/css and JavaScript.

Part 1    Part 2


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Lecture 7: Node-RED dashboard (Part2)

Example 7.2 Introduction to the UI nodes from node-red-dashboard


This second example uses the built in dashboard nodes that come with Node-RED. If you are using FRED, make sure you have selected this node set from the add/remove button in the management panel and that you aren’t using the old legacy-ui nodes.

To start, let’s wire up a simple flow that sends a random number between 0 and 99 to a simple chart. For that you’ll need an inject node to repeatedly fire every few seconds, a function node to generate the random number and one of the  node-red-dashboard nodes – in this case the chart node.

screen-shot-2016-10-12-at-4-38-47-pm

Before we look a how the chart node works, let’s configure the inject node to send a timestamp every 5 seconds by setting the payload to timestamp and the repeat field to an interval of 5 seconds.

Screen Shot 2016-05-16 at 12.46.11 PM

This will act as our repeating trigger. Now we need to set up the function node to generate a random number – we’ll use a simple JS math function to do this:

msg.payload = Math.round(Math.random()*100);
return msg;

Screen Shot 2016-05-16 at 12.48.42 PM

This will generate a random number between 0 ~ 99 which is passed to the chart node.

So now let’s take a look at the chart node. When you double click it, you’ll see it’s configuration options:screen-shot-2016-10-12-at-4-46-53-pm

If you click on the button of the Group field, you will be prompted  to configure the tabs of the UI.

screen-shot-2016-10-12-at-4-42-04-pm

The Tab option allows you to specify which tab of the UI page you will see the UI element on – in this case our chart. The default tab is Home – which we are using here. If you select the edit button to the right of the Tab field you can create a new tab and then select that. However, we’ll use the default home for now.

The Name field is the standard Node-RED node name – by default this is chart but you can set it to anything you like.

The Group field allows you to group UI elements – we’ll show you how that works when we add another UI element so let’s use group “Default[Home]” for now – of course, you can use any string you like.

The X-asis field allows you to tell the chart how much data it should store and display – the longer the ‘last‘ filed is set to, the more data is stored and displayed by the chart. Let’s use a short 5 mins which will start to throw away the data that is 5 minutes old.

Lastly the Interpolate field defines how the chart will interpolate values in between actual data values it receives, you can select between linear, step, b-spline and cardinal – which are standard interpolation algorithms. We’ll use the default linear.

Wire these nodes up, hit the deploy button – check that your debug node is showing that random values are showing. Then head over to your default dashboard page to see the results. By default, under FRED, you’ll see your UI at:

https://{your username}.fred.sensetecnic.com/api/ui/

When you visit that page you’ll see your initial chart as shown below:

screen-shot-2016-10-12-at-4-48-47-pm

As you can see, you get a nice line chart, scaled for your data values (X axis) and the time they arrived (y axis). You can also see “Default” at the top left of the chart, indicating this is UI group “Default” – which is the group name we set in the configuration fields for the chart node.

If you look at the top left of the web page, you can see that we are, by default, on the home tab. If you had created your own tab then when you click the selector top left you’ll get a pull down menu of your tab options:

screen-shot-2016-10-12-at-4-52-40-pm

That was pretty simple, let’s add a few other UI elements to our dashboard. Firstly let’s create a gauge to show the last data value sent. Drag a Gauge node from the UI palette and wire it to the Random Number function node.

Then double click to open up and let’s configure it:

screen-shot-2016-10-12-at-4-56-16-pm

We’ll us the same Tab, home and we’ll also add it to the same group – “Default[Home]”. The Min and Max fields allow you to set the min and max values the gauge will shown. Make sure the max is set to 100 which is the most that the random number function node will generate. You can also change the Colour gradient to show different colours on the widget, but we will leave it as default for now.

Hit deploy and then head over to your dashboard and you’ll see that the chart and gauge are displayed in a group with the chart now showing the last 5 minutes of data and the gauge the latest value.

screen-shot-2016-10-12-at-5-01-16-pm

As a last example, let’s use a couple of the other UI nodes, a slider node and a text node to show the same data on a slider and as a text string.

screen-shot-2016-10-12-at-5-03-24-pm

For these two nodes, configure them to use the same tab – “Home” but use group name “anotherWidget”(You will need to click “Add new UI_group” from the drop down menu of the Group field, and then click the edit button). You will also need to change the min and max value for the slider node to show the correct position of the slider. Deploy those and let’s take a look at your dashboard. As you can see, we now have two widget groups, group “Default” with a chart and a gauge, group “anotherWidget” with a text filed and a slider. Simple eh?

screen-shot-2016-10-12-at-5-05-09-pm

In the dashboard tab beside your debug tab, you can also set the theme and order of the elements. If you don’t see the dashboard tab, click the menu button at top right corner, then select “View” -> “Dashboard”. You can see all the widgets and tabs showing in a tree structure, and you can easily drag the elements to change the orders that they are presented in the dashboard.

screen-shot-2016-10-12-at-5-10-31-pm

Now you’ve got the basics sorted out, play around with the different dashboard elements to build your own dashboards using some real world data – have fun!

Part 1      Part 3


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.

Node-RED: Lecture 7 – Dashboards and UI techniques for Node-RED

In this lecture you will take a look at a few techniques to allow you to visualize data passing through flows. We’ll focus on three approaches, the use of a third party dashboard tool, FreeBoard (Part 1), using the default Dashboard UI nodes provided by default in Node-RED (Part 2) and a general technique using a standard JavaScript charting tool (Part 3).

By the end of the lecture you will have enough knowledge to decide which approach is best for your specific needs and how to quickly get a visualization of your data up in a web browser.

We use a cloud hosted version of Node-RED for these lectures called FRED. Sign up for a free account at FRED.sensetecnic.com. Examples in the early lectures will work with other installations of Node-RED, later lectures use nodes you will need to install yourself if you don’t use FRED.

Example 7.1 Using the FreeBoard dashboarding service

This is a simple example of reading and visualizing data using the FreeBoard node from a Node-RED flow. We’ll be using the Cloud based FRED service as our Node-RED engine and visualizing data from a web weather service . This guide will show you how to:

  • set up an openweathermap node in FRED using the built in node
  • display the data from the openweathermap node using the FreeBoard display node

To begin, head over to openweathermap.org and register for a free account. When you register/login you will be able to access your API Key. You will need your own API key which will look similar to the one below – you will use your own key in Node RED so make a copy.

 

01

Now, head over to FRED and create a new flow by starting with an openweathermap node (you will find this node under “weather”, or by using “filter nodes” to search at the top of the left pane) and connecting it to a debug node like this:

02

 

Now, double click on the openweathermap node to configure it. Enter your API Key from OpenWeatherMap, configure a location you’re interested in, and name your Node:

 

03

Click on Deploy on the top right corner of the Node-RED UI. You will see on the right pane, under “debug” the data that OpenWeatherMap provides us with a variety of information about the location including temperature (in C and K), humidity, windspeed etc:

 

04

As you can see, the data is already in JSON format; so we can use any of the values very easily. Let’s build a dashboard to visualize our data using the  node that allows you to create visualizations very easily. Find the freeboard node in the left pane by either browsing through the nodes or searching for it using “filter nodes”. Add it to the flow. Double click on it to give it a name, like so.

05

 

Click on Deploy. This will prompt the openweathermap to get data and send it to both the debug and freeboard nodes. The freeboard node is quite smart and will do its best to parse the data you send it and figure out how to make it available using the FreeBoard UI. We’ll look at that in a moment. If you inspect the “info” tab in the right panel you will see more information about the “freeboard” node:06Visit the link provided in the info panel, or at

https://{username}.fred.sensetecnic.com/api/freeboard/

This will open a new tab in your browser:

07

This tab will allow you to create visualization in FreeBoards, save them and load them. Let’s create a visualization of our weather data. We first need to add a datasource to our freeboard. Click on “Add” under “DATASOURCES”. Under “type” select the name of the “freeboard” node we configured above. In our case we named it “freeboard”. This will allow us to access ANY data we connect to the “freeboard” node in Node RED.

 

08

 

We will now add a Pane. Click on “ADD PANE”, this will add a new empty pane:

09

 

Now, we will add a “Widget”, click on the “+” (plus) sign in the new pane. Select “Gauge”. Under “DATASOURCE” select the “freeboard” node and select “tempc”. As you can see, all of the different data fields in the openweathermap JSON structure are available for you to visualize. FreeBoard essentially takes the JSON structure you pass it and breaks out any fields. If you can’t see this in the dropdown, go back to Node RED and click on “Deploy” again. So that the “freeboard” node can get some data and store the data it has received. Your configuration should look something like this:

 

10

 

Click on OK. Which will build your widget and add it to the pane you created!

11

 

To save your dashboard click on “save”, and select [pretty] or [minify] which will save it in FRED and will show you the URL to access the Dashboard. You need to save the URL used to access the dashboard, e.g. as a dashboard. From now on you can access your saved dashboard by accessing the saved url, which will be similar to the one below.

https://{username}.fred.sensetecnic.com/api/freeboard/#start-16091

Part 2  Part 3

 


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

 

Node-RED: Lecture 6 – Intermediate flows

This lecture will build on the ideas you saw in lecture 5 and focus on examples that explore some of the key concepts from that lecture. The examples in this lecture are a little more complex than previous examples – mainly in the sense that the function nodes are more complex – but are still kept as clean and simple as possible.

This lecture will build on the ideas you saw in lecture 5 and focus on examples that explore some of the key concepts from that lecture. These include the ideas of context, messages and sub-flows. The examples in this lecture are a little more complex than previous examples – mainly in the sense that the function nodes are more complex – but are still kept as clean and simple as possible.

 

We use a cloud hosted version of Node-RED for these lectures called FRED. Sign up for a free account at FRED. Examples in the early lectures will work with other installations of Node-RED, later lectures use nodes you will need to install yourself if you don’t use FRED.

 

Examples

(Click to go directly to the example)

Example 6.1 Retrieving data from a web page

Example 6.2 Counting words in a string

Example 6.3 Using context to generate rolling averages

Example 6.4 Using the context element to share a function and make it accessible to all functions in the canvas

Example 6.5 Defining and using an iterator sub-flow

Example 6.6 Getting earthquake data from an external API and returning it as multiple messages

Example 6.7 Multiple inputs on a function node

Example 6.8 Letting a function node send multiple messages on a single output

Example 6.9 Creating a Blog Site With Node-RED

 

Example 6.1 Retrieving data from a web page

 

 

Let’s write a flow that scrapes the latest stock market indices from google at http://finance.yahoo.com/market-overview/ and format it using a function node. If you inspect the page shown in Figure 6.1 using the Chrome browser’s ‘Inspect Element’, you’ll find that the three indices have the same td.bld span tag. This tag can be used to retrieve all three indices with the html node. You’ll set up an inject node to trigger an http request to get the page, then the html node to get the elements with the td.bld span tag.

Figure 6.1 Stock market index page from Google.

 

Configure the http node and html nodes as shown in Figure 6.2

Figure 6.2 Configuration of http and html nodes to retrieve stock indices from Yahoo.

Then wire them up as shown in Figure 6.3, with a debug node to show the output.

 

Figure 6.3 Flow to retrieve the market indexes.

The output should contain the latest index values. When you click on the ‘trigger’ inject node, you will see the following in the debug pane:

Figure 6.4 Stock indices as array of strings.

That was easy, but what you really want is an array of JSON name:value pairs that uses the name to describe the values and shows the values as numbers:

Listing 6.1 Desired JSON output of stock indices

  1. [{
  2.     “index”:”S&P”,
  3.     “value”:2096.92
  4.  },{
  5.     “index”:”Dow”,
  6.     “value”:17511.34
  7.  },{
  8.     “index”:”Nasdaq”,
  9.     “value”:5059.35
  10.  }]

Let’s write a function node to format this the way you want it. Take a look at Listing 6.2. Line 1 takes the incoming stock prices and stores them in an array prices and then creates two new arrays, one to hold the output messages and the other to hold the textual name of the three different indices. Lines 5-8 loop through the incoming stock indices and for each one pushes a name:value pair to the output message array. At Line 8, you use the JavaScript replace function to remove all commas from the index values before parsing them as numbers. Let’s call this function format indices.

Listing 6.2 Function node code to format stock indices

  1. msg.headers = msg.originalHeaders;
  2. var prices = msg.payload;
  3. var newPayload = [];
  4. var priceIndex = [‘S&P’,’Dow’,’Nasdaq’];
  5. for ( var i=0; i<prices.length; i++) {
  6.     newPayload.push({
  7.     index:priceIndex[i],
  8.     value:Number(prices[i].replace(/,/g,”))
  9. });
  10. }
  11. msg.payload = newPayload;
  12. return msg;

Now wire this into your flow as shown in Figure 6.5.

Figure 6.5 Complete market index flow.

Once you deploy and test, your debug console should output something similar to Figure 6.6. The data now looks good and you can use this format for downstream nodes in your flow.

Figure 6.6 Debug output from new flow.

Example 6.2 Counting words in a string

Next, let’s write a more complex function node that receives some text in a message payload, then outputs multiple messages containing all individual words and the number of times each word was used.

Listing 6.3 Word count function

  1. var outputMsgs = [];
  2. var wordMap = {};
  3. var sentence = msg.payload.replace(/[.,-\/#!$%\^&\*;:{}=\-_`~()]/g,””);
  4. sentence = sentence.replace(/\s{2,}/g,” “);
  5. var words = sentence.split(” “);
  6. for (var i = 0; i < words.length; i++) {
  7.   var lowerCaseWord = words[i].toLowerCase();
  8.   if (!wordMap[lowerCaseWord]) {
  9.     wordMap[lowerCaseWord] = 1;
  10.   } else {
  11.     wordMap[lowerCaseWord] = wordMap[lowerCaseWord] + 1;
  12.   }
  13. }
  14. for (var prop in wordMap) {
  15.   if( wordMap.hasOwnProperty( prop ) ) {
  16.     outputMsgs.push({payload:{word:prop,count:wordMap[prop]}});
  17.   }
  18. }
  19. return [outputMsgs];

In Listing 6.3, the list of output messages and an object to hold the word counts (Lines 1 and 2) is declared. In lines 4 and 5 the payload is ‘cleaned’, removing punctuation and extra spaces using regular expressions. Regular expressions are a very useful tool for text processing; you can learn more about regular expressions by looking at the Mozilla Developers pages for JavaScript (here).  Line 7 splits the text into multiple words, then iterates through the words, creating a mapping of the lowercase version of each word to word count called wordMap in Lines 8-15.  Lines 16 to 18 split the wordMap into multiple messages in the outputMsgs array[1].   Finally, in line 21, an array of this array of messages is returned, sending them all to the first output port one at a time[2].

Let’s wire this up and see it run with some example text.  First, create a function node, and copy in the code above. Call it word count. Then add an inject node and add the following text:

Figure 6.7 Inject node to test word count function node.

Add a debug node and wire up the flow as follows:

Figure 6.8 Test flow for word count function node.

When you click on the inject node, you should see the list of word counts in the debug pane:

Figure 6.9 Debug output from word count function node.

Example 6.3 Using context to generate rolling averages

A special module called context,used to store data between function invocations, is available to function nodes. This can be useful when the function needs to retain state to do its processing. For example, it may be necessary to compute the average value of a sensor’s data readings over a period of time. Listing 6.4 computes a rolling average of the values received over the last 5 seconds, adding the ‘average’ field to the payload when more than 5 seconds have elapsed between received messages.

Listing 6.4 Rolling average function using context

  1. var currentTime = new Date().getTime();
  2. if (!context.lastTime) {
  3.     context.lastTime = currentTime;
  4.     context.sum = msg.payload.value;
  5.     context.count = 1;
  6. }
  7. if (currentTime-context.lastTime > 5000) {
  8.     // calculate average for previous messages
  9.     msg.payload.average = context.sum/context.count;
  10.     // start tracking average again
  11.     context.sum = msg.payload.value;
  12.     context.count = 1;
  13.     context.lastTime = currentTime;
  14. } else {
  15.     context.sum += msg.payload.value;
  16.     context.count +=1;
  17. }
  18. return msg;

Looking at listing 6.4, you first get the current time (line 1).  If there is no lastTime stored in context, you then save the currentTime, reset the sum and count context variables (lines 3-7).

If the currentTime is 5 seconds (5000 milliseconds)  greater than the last time (line 9), you calculate the average of the last values received and include that with the message payload (lines 11-13).  You then reset the sum, count and set lastTime to the currentTime to begin counting and summing up again.

If 5 seconds haven’t elapsed since the last message was received, the counts and sums need to be updated (lines 16-19). You then output the message which includes the latest value and the average if it was calculated for the interval (line 20).

To test this, let’s write another function node, that you’ll call ramp and that also uses context to generate values from 0 to 9 in sequence, as shown in Listing 6.5.

Listing 6.5 Test function to generate sequence of values using context

  1. if (!context.value) {
  2.   context.value = 0;
  3. }
  4. msg.payload = {
  5.   value:context.value
  6. }
  7. context.value +=1;
  8. if (context.value > 9) {
  9.   context.value = 0;
  10. }
  11. return msg;

Let’s configure an inject node to send data every second, as shown in Figure 6.10.

Figure 6.10 Inject node configured to inject a blank payload every second.

Then wire these up as shown in Figure 6.11.

Figure 6.11 Test flow for rolling average function.

The output should look like the one in Figure 6.12.

Figure 6.12 Debug console output showing rolling average.

Example 6.4 Using the context element to share a function and make it accessible to all functions in the canvas

This example shows how to use the context object and its global element to share data across function nodes. The example above used this to store a numerical value. However, one of the great things about JavaScript is that you can assign functions to objects. This allows you to share methods across your canvas without the need to redefine them in each consecutively used function node. The contrived example below simplifies this process for better understanding.

First, connect an inject node, two function nodes and two debug nodes like this:

 

Figure 6.13 Wire up functions for a context example

Let’s edit the first function node and add this code. Set the global element of the context object to have a new element, “hello”, which is an anonymous function that returns the string “Hello There”. Then set the message payload of this function to the return value of this global function. This will print out “Hello There”.

Figure 6.14 Define an anonymous function as a new context element

Now edit the second function node by setting up the message payload of this second function to concatenate the output of the ‘hello’ element (anonymous function) in the global element of the context object with the word “World”:

Figure 6.15 Use the anonymous function defined in another function node

Pressing the button on the inject node shows the output of each of the function nodes:

Figure 6.16 Output of context function sharing example

Despite being very simple, this example shows how easy is to use the context element to set not only variables that contain data, but also functions that can be shared and accessed by several functions in your canvas.

Two important aspects to consider are: firstly, the function setting the global element of the context object must come BEFORE any other function wishing to use that variable or function in the flow. Secondly, in the current version of Node-RED (1.11.0), the context object remains in memory after re-deployment until the Node-RED instance is rebooted. So even after deleting the function nodes that set an elements in the global element, these elements will still be accessible until set to null.

Example 6.5 Defining and using an iterator sub-flow

As mentioned in lecture 5, sub-flows can be used to package up functions into nodes in your node pallette. In this example, you’ll create a sub-flow that processes an array of values in a message payload and outputs a new processed array[3].  An upstream node will supply a message with a payload containing an array of values to be processed. The first output of the iterate node will output each of the input message array payload messages , contained in a message payload in order. The second output will output a message containing a payload with a new array containing processed values. To process data, you can wire up the sub-flow as shown in Figure 6.17, connecting a function node to provide some test data and process each element in the array.

Figure 6.17  Example sub-flow to iterate through list of values in an array payload.

Start by creating a new sub-flow tab. Drag in a function node and add the code from listing 6.6.

Listing 6.6 Iteration function node

  1.   var currentMsg = null, outMessage = null;
  2.   var iState = msg.iState;
  3.   if (!iState) {
  4.      // we received an initial message
  5.      // if the message is not an array, make it one
  6.      if( Object.prototype.toString.call(msg.payload) !== ‘[object Array]’ ) {
  7.            msg.payload = [msg.payload];
  8.      }
  9.      iState = {};
  10.      iState.index = 0;
  11.      iState.inArray = msg.payload;
  12.      iState.outArray = [];
  13.      msg.iState = iState
  14.    } else {
  15.     // save results from the last iteration
  16.      iState.outArray.push(msg.payload)
  17.  }
  18.   //If there are still objects left to iterate goto the next one in the original array
  19.   if (iState.index < iState.inArray.length) {
  20.     currentMsg = msg;
  21.     msg.payload = iState.inArray[iState.index];
  22.   } else {
  23.     currentMsg = null;
  24.     outMessage = msg;
  25.     msg.payload = iState.outArray;
  26.     delete msg.iState;
  27. }
  28. iState.index ++;
  29. return [currentMsg, outMessage];

In lines 1 and 2, several variables are declared.   currentMessage, holds the current input message; outMessage holds the final output message containing our processed array payload and iState is used to make it easier to access the current state of the iteration from the msg.iState property.  While context could be used to manage the function’s state, it’s simpler to hold state in the message, since you may receive new arrays as input before completing the iteration of another array.

In line 4, you check to see if iState exists. If not, assume that the message is a new input message.  Check that the payload is an array, and if not, make it one in lines 7-9. Then generate the iteration state object that includes the current index, input array, output array, and add it to the msg in lines 10-14.

If iState does exist (16-18), you know it’s an iteration message generated by this function node and processed in the loop back.  Push it into the output array to send out when you’re done.

Next, see if you’re done or not. If you need to iterate again (index is less than the length of the input array), set currentMessage to the message and set the payload to thecurrent index (lines 20-22).  When finished iterating, set the currentMessage to null, the outMessage to the message, the payload of the message to the outputArray and delete the iState property that is no longer needed.

Then increment the index and output the messages to the output endpoints. During the iteration, currentMessage is checked to ensure it holds a value, otherwise (when null) the loop terminates.  At the end, outMessage will be non null and sent to the second output.

Now let’s hook up inputs and outputs to our sub-flow as shown in Figure 6.18.  Let’s name this subflow ‘Iterate’.

Figure 6.18 Iterate sub flow.

To test out the flow, generate an array from 1 to 5 as in Listing 6.7, and multiply each element in this array by 5.  To do this, create a new function node called [1,2,3,4,5] and wire it in as shown in Figure 6.17.

Listing 6.7. function node [1,2,3,4,5] from test flow in Figure 6.17

  1. msg.payload = [1,2,3,4,5];
  2. return msg;

Write a second function that is used in each iteration, as in Listing 6.8, calling it times 5, and wire it up to the new subflow output 1 and input.

Listing 6.8 multiply payload times 5

  1. msg.payload = msg.payload * 5;
  2. return msg;

Finally trigger the flow, using an inject node, and add a debug node. After clicking on the inject node, the debug window should display [5,10,15,20,25].

Now that this iteration function is packaged in a sub-flow you can use it in any of your flows by simply dragging it in from the node palette like any other node. It will iterate over any array of integer values and apply the function specified.

Example 6.6 Getting earthquake data from an external API and returning it as multiple messages

This example demonstrates how to get data from an external API and how to separate that data using a function node. We will use data from an external API that provides access to earthquake data which is made available by the US geological survey (USGS). (http://earthquake.usgs.gov/earthquakes/feed/v1.0/geojson.php)

First, let’s wire the flow like this:

 

 

Figure 6.19 Flow to query USDG earthquake data

Let’s edit the http request to get data from the following url as shown below (Fig 6.20):

http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/significant_month.geojson

This data contains significant earthquakes from the last month. The JSON node after the request will allow you to parse the body of the response into an object usable by the following function.

Figure 6.20 Setting up a http request node to query USDG earthquake data

Now edit the function node and add the code shown in Fig 6.21:

This code will create an array (outputMsgs) containing an array of messages built from your data. The for loop goes through the received JSON response and creates a new message with a payload containing lat, lng, value, message and timestamp values (only a few values from the JSON object sent from the server). It will then push this new message object to the outputMsgs array.

It will create a new variable (msg2) with a new message object containing a payload element with the string “Second Output”.

Finally, the function will return an array containing two elements. The first element is an array of messages (outputMsgs); the second element is a single message (msg2). Let’s configure the function to have 2 outputs, to match our returned array.

Figure 6.21 Function node code to query USDG earthquake data

When you click the button on the inject node, you can see that the top output contains an array of messages parsed from the data. This corresponds to the outputMsgs array in the function (See Fig. 6.22).

Figure 6.22 First output from the flow that queries USDG earthquake data

The second output contains the single message (msg2):

 

Figure 6.23 Second output from the flow that queries USDG earthquake data

Example 6.7 Multiple inputs on a function node

Function nodes in Node-RED were designed to process messages as single entities. However, in some cases your functions might depend on two separate data sources. There are many ways to handle these cases in Node RED. The following approach uses the context object in Node-RED and topics to let a function wait for several messages to arrive in order to return. You saw how to set-up and use context data in Example 5.3 in the previous lecture.

Let’s start by connecting two inject nodes, a function node and a debug node like this (Fig 6.24):

Figure 6.24 Setting up a flow to explore multiple inputs to a function node

Let’s edit the function node and add the following code (see Fig 6.25). This code will use the context object in Node-RED and add a data element.

Line 1 initializes the context object. Then the switch statement at line 2 looks for the topic field in the message. It uses this to set the task1 or task2 field of the context.data object. Any other message topic is ignored. Line 16 then checks to see if the function has received messages of both topic types (task1 and task2). If not, the function returns a null message and goes back to wait for another message. Otherwise, line 17 calculates the ratio and outputs it as a message.

Figure 6.25 Function node code to wait for all input before proceeding

Let’s configure the first inject node to return a string payload of “3”, with a topic of “task1”

Figure 6.26 Configuring an inject node with data and a topic used by a context object

You should configure the second input node to return a payload string of “6”, with a topic of “task2” (not shown but similar to Fig 6.26)

You can then deploy the flow. Click on the left button of the “task1:3” inject node. You will see a success message indicating that the string has successfully been injected, but you will not see anything in the debug tab. Click on the left button of the “task2:6” inject node. You will see a success message and the debug tab will show the ratio as expected:

 

Figure 6.27 Exercising the multiple input flow of Example 6.7

Example 6.8 Letting a function node send multiple messages on a single output

Example 5.2 in Lecture 5 demonstrated how to set up and send messages on multiple output nodes. This example shows how to send multiple message, but on the same output, from a single function node. One common scenario is a function node batch processing some data, with the following nodes wanting to have processed data as soon as it becomes available. Another common scenario is the creation of nodes that in turn create counters or pulses to actuate relays.

The following example is based on a flow by Node-RED contributor dceejay. Tthe original flow can be found at http://flows.nodered.org/flow/5c4c0f5a08d4e91ea14d

First, wire together an inject, function, delay and debug nodes:

Figure 6.28 A basic flow to explore multiple output messages

Edit the function node and add the following code (See Fig. 6.29). This code is quite simple; it contains a loop that calls “node.send” 5 times. node.send was introduced in Lecture 5 at the end of example 5.2. It allows a function node to output messages to its output independently of its return value. Note how the function itself returns null. The expected output of this function will be a series of messages with the payloads: 0,1,2,3 and 4.

Figure 6.29 Code to send multiple messages using the node.send() command

To make the example ressemble a counter more closely, you can edit the delay node and configure it to limit the rate of messages to 1 per second, as shown in Fig 6.30.

Figure 6.30 Configure the delay node to limit its output rate to 1 message per second

The flow can now be deployed. Press the button left of the inject node and inspect the debug tab (See Fig 6.31) and you will see the messages arriving every second:

 

Figure 6.31 Output from Example 6.8

 

Example 6.9 Creating a Blog Site With Node-RED

This final example will show you how to build a micro blog service with only a few nodes in Node RED. You’ll be using the MongoDB node as a storage for posts, http nodes to provide end points for the service and the html node to format the micro-blog web page.

We will use a free cloud service called “mongolab”. It allows you to create “sandbox” databases you can use to prototype your applications. Head over to https://mongolab.com. Register for an account. You will then be able to create a database. Click on “Create New” in your home dashboard.

Figure 6.32 Creating a free MongoDB at mongolab

Fill out the form to create a new MongoDB deployment. Select Amazon, Single-node and select the FREE size of 500mb.

Figure 6.33 Choosing the free MongoDB

Scroll down and name your database. We are naming it “mycontent”. Now click on Create new MongoDB deployment:

Figure 6.34 Name the free MongoDB

MongoDB allows you to create “collections” for each database. They are analogous to “tables” in relational databases, each “collection” holds “documents” analogue to “records” in relational databases. In order to begin using your database you need to create a new collection. Select your newly created database in your dashboard and add a new collection. We will name it “posts”.

Figure 6.35 Adding a collection to the  MongoDB

Finally, you will need a user to connect to that database. Add a new database user. We will name it “freduser” but you should use a name that makes sense to you.

Figure 6.36 Adding a user to your MongoDB

Finally, note how this page also gives you important information on how to connect to your MongoDB database. We will need the URI (in our case ds037234.mongolab.com), the port (37234 in our case) and your newly created user and password.

Figure 6.37 Note down the connection info for your new MongoDB

Now, head over to Node-RED and wire up a http-in, mongodb-in, a template, and an http-out node as shown in Fig. 6.38.

Figure 6.38 The basic flow for the micro-blog

Edit the http-in node to accept GET request on the URL “/public/posts” (Fig 6.39)

Figure 6.39 Setting up the http node to accept requests on /public/posts

We will now configure our mongodb in node. Double click on the node and create a new server connection. Here is where you will use the information from your mongolab instance:

Figure 6.40 Configuring the mongodb node using the MongoDB account info

Click on Create/Update and configure your node to use the “posts” collection you created. Configure it to do a “find” operation, which will find all documents in that collection. We will now name it “find posts”.

Figure 6.41 Configuring the mongodb node with the collection and the find operation

Once you have the http node and the mongodb nodes configured you can edit the html template node to handle the data returned by the mongodb node (Listing. 6.9):

 

Listing 6.9 Formatting the blog posts

And add the following code:

  1. <!DOCTYPE html>
  2. <html lang=”en”>
  3.   <head>
  4.   </head>
  5.   <body>
  6.     <h1>My Micro Blog</h1>
  7.     <form action=”posts” method=”post”>
  8.         <label for=”title”>Title</label>
  9.         <input type=”text” class=”form-control” name=”title” placeholder=”Title…” />
  10.         <label for=”post”>Post Content</label>
  11.         <input type=”text” class=”form-control” name=”post” placeholder=”Say something…” />
  12.         <button type=”submit” class=”btn btn-default”>Submit</button>
  13.     </form>
  14.     <div>
  15.         {{#payload}}
  16.         <div class=”post”>
  17.             <h3>{{title}}</h3>
  18.             <p>{{post}}</p>
  19.         </div>
  20.         {{/payload}}
  21.     </div>
  22.   </body>
  23. </html>

This code, while long, is very simple. You will be using Bootstrap to style your website (http://getbootstrap.com/) and provide a responsive layout. If you now visit http://{your user name}.fred.senstecnic.com/api/public/posts you will be able to see your new blog site (Fig 6.42). However there is no content yet.

 

Figure 6.42 Output from the micro-blog html template node

 

 

So far, you’ve created the flow needed to display posts. Now you need to create the other side of the blog service, a flow to create posts and save them to your MongoDB collection. Connect an http-in, a function, mongodb-out, and http-out nodes as shown in the bottom part of Fig 6.43.

 

Figure 6.43 Setting up a flow to allow you to post blog entries and store them in MongoDB

Edit the http-in node to accept PUT requests at /public/posts (Fig 6.44)

Figure 6.44 Setting up the end point for posts for the micro blog service

Now let’s edit the function node and add the following code (listing 6.10). This builds a simple http msg.header which your flow will pass to the http node.

 

Listing 6.10: Building a http request for the micro-blog service

  1. msg.headers = {
  2.     “Location” : “https://{your username}.fred.sensetecnic.com/api/public/posts”
  3. };
  4. msg.statusCode = 302;
  5. return msg;

 

Note that at line 2 you set the address to be the end point for your service. This will use your user name in place of ‘guides’.

Finally, you will need to edit the mongodb-out node and select your previously configured server. Configure it to use the collection “posts” and an “insert” operation. Make sure to check “only store msg.payload object”, this will make sure we only store the data in the message payload, not the whole message object (fig 6.45).

Figure 6.45 Configuring the mongodb node to store the incoming blog postings

That’s it! You can now go to your url and use your micro blog site where you’ll see something similar to Fig 6.46.

 

Figure 6.46 Screen show of the simple micro-blog web page

 

 

Summary

In this lecture you have seen a number of more complex flows that build on the nodes and techniques introduced in earlier lectures. In particular, you’ve seen how to scrape data from a commercial web site and query web-based APIs, how to use context and message in more complex ways and how to build a simple web service to support a micro-blog. The next lecture will conclude our intermediate section (lectures 4-7) by introducing how to install Node-RED on devices such as the Raspberry Pi. It will show some examples that you can’t run on the cloud-based FRED service (because they require access to the underlying hardware), but which do run on the Pi, and it will discuss some of the issues with flows in devices such as the Pi.

 


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

 


[1] The check for hasOwnProperty() in line 17 is good practice when iterating through the properties of an object using the JavaScript in operator. It ensures you’re getting only properties you’ve added yourselve and none that you’ve (unexpectedly) inherited from the Object base class.

[2] Note that if you returned outputMsgs directly, the function node would attempt to send each message to a different output.

[3] Based on forEach (array iteration & completion) subflow by dhartveld at https://gist.github.com/dhartveld/43501a1b424434de0ffb

Node-RED: Lecture 5 – The Node-RED programming model

In this lecture, you’ll take a more detailed look at Node-RED’s programming model and some of its key concepts and explore the details of the message flow model that underlies Node-RED. Then you’ll dive into how to program your own function nodes using JavaScript and create reusable sub flows to add your own functionality to the set of nodes supplied with Node-RED.

As you’ve seen in previous lectures, Node-RED uses a visual flow-based programming paradigm[1]. This is not unique to Node-RED. Similar approaches have been used in many other domains such as music and multimedia (Max MSP), toys (Lego Mindstorms), enterprise application integration and industrial automation (LabVIEW).  Like these tools, Node-RED makes it easy to switch between design and development for rapid integration, prototyping and development tasks.  So far, you’ve seen how to wire simple flows together and some of the basics of Node-RED programming.

In this lecture, you’ll take a more detailed look at Node-RED’s programming model and some of its key concepts and explore the details of the message flow model that underlies Node-RED. Then you’ll dive into how to program your own function nodes using JavaScript and create reusable sub flows to add your own functionality to the set of nodes supplied with Node-RED.

 

We use a cloud hosted version of Node-RED for these lectures called FRED. Sign up for a free account at FRED. Examples in the early lectures will work with other installations of Node-RED, later lectures use nodes you will need to install yourself if you don’t use FRED.

Examples

Example 5.1 Creating and returning a new message in a function node

Example 5.2 Creating and returning multiple messages in a function node

Example 5.3 Using Context in a function node

Example 5.4 Using Global Context

Example 5.5 Creating a sub-flow

Example 5.6 Packaging Function nodes using sub-flows.

Example 5.7. Creating a Slack Bot to query data

Key Concepts

As you’ve learned so far, when you program with Node-RED, you create flows that are networks of lightweight components, called nodes, that exchange messages along pre-defined connections or wires.  Let’s look at these concepts in a bit more detail.

Flows

Node-RED programs or flows are a collection of nodes wired together to exchange messages.  Under the hood, a flow consists of a list of JavaScript objects that describe the nodes and their configurations, as well as the list of downstream nodes they are connected to, the wires.

Messages

As discussed in Lecture 2, messages passed between nodes in Node-RED are, by convention, JavaScript Objects called msg, consisting of a set of named properties[2].  These messages often contain a msg.payload property with, you guessed it, the payload of the message.  Nodes may attach other properties to a message, which can be used to carry other information onto the next node in the flow. When this happens, these extra properties will be documented in the node documentation that appears in the node info pane when you select a node in the Node-RED workspace.

Messages are the primary data structure used in Node-RED and are, in most cases, the only data that a node has to work with when it is activated. This ensures that a Node-RED flow is conceptually clean and stateless – each node is self-contained, working with input messages and creating output messages.  Apart from the use of context data (see later in this lecture), this means that the effect of a node’s processing is either contained in its output messages, or caused by internal node logic that changes external things such as files, IO pins on the Raspberry Pi or Dropbox files; there are no side effects that could affect the behaviour of other nodes or subsequent calls to the same node.

This is one of the key advantages of a flow-based language. Because nodes are self contained and typically only interact with other nodes using messages, you can be sure that they have no unintended side effects and so can be safely re-used when you create new flows. This ‘safe’ code reuse is exactly what you are doing each time you drag and drop a node onto your workspace.

Nodes

Nodes are the primary building block of Node-RED flows.  When a flow is running, messages are generated, consumed and processed by nodes.  Nodes consist of code that runs in the Node-RED service (javascript .js file), and an HTML file consisting of a description of the node, so that it appears in the node pane with a category, colour, name and an icon, code to configure the node, and help text.  Nodes can have at most one input, and zero or more outputs[3].

During the initialization process, the node is loaded into the Node RED service.  When the browser accesses the Node RED editor, the code for the installed nodes is loaded into the editor page as illustrated in Fig. 5.1.

Figure 5.1 Node RED loads both HTML for the editor and JavaScript for the server from the node packages.

 

As you saw in Lecture 2 (Figure 2.2), there are three core node types:

  • Input nodes – generate messages for downstream nodes.
  • Output nodes – consume messages, for example to send data to an external service or pin on a device, and may generate response messages.
  • Processing nodes – messages that process data in some way, emitting new or modified messages.

In addition to these core types, there are two more categories:

  • Credentials Nodes – these are nodes that hold the credentials used by one or more nodes to connect to an outside system or service such as the Sense Tecnic platform, MQTT or Pushbullet. These are displayed only in the credentials pane, not on the main pane, and are created as needed when you configure a node that requires credentials such as an API key or name and password. Once created, a credentials node can be reused by other nodes of the same type to connect to similar protocols or services. Even when all nodes that use those credentials are deleted from your flow, the credentials node will remain, so it’s a good idea to remove unused credentials nodes when they are no longer needed.
  • User-created nodes – programmable nodes such as function nodes, or sub-flows are nodes created by you to do some custom work or reuse flow segments in other flows.

Wires

Wires define the connections between node input and output endpoints in a flow. They (typically) connect the output endpoints of nodes to inputs of downstream nodes indicating that messages generated by one node should be processed by the connected node next.  Note that it is possible to connect more than one node to an endpoint using wires.  When multiple nodes are connected to an output endpoint, messages are sent to each connected node in turn in the order they were wired to the output.  When more than one node output is connected to an input endpoint, messages from any of those nodes will be processed by the connected node when they arrive.  It is also possible to connect downstream nodes to upstream nodes to form loops.  (You’ll see an example of wiring a node to an upstream node in example 6.5 in Lecture 6).

Context

So far, you have learnt that messages are the only way to get data into and out of nodes. While this is generally true, there is one exception to this rule which is available to function nodes. function nodes have access to a special object called context that is used to hold data in memory that lasts from one message arriving to the next[4]. This is important for nodes that need to maintain an index or count or sum data in messages.  In addition to this local context, a global context context.global is available for sharing data between all of the function nodes of a flow. Some use cases for context will be covered when the function node is discussed in more detail.

Function Nodes

The function node is the ‘Swiss Army knife’ of nodes that you can use when there is no existing node dedicated to your task at hand.  It’s great for doing specialized data processing or formatting for example. As the name implies, a function node exposes a single JavaScript function. Using the function node, you can write your own JavaScript code that runs against the messages passed in and returns zero or more messages to downstream nodes for processing. To write function nodes, you write a JavaScript function using the built-in code editor as shown in Fig. 5.2.

Figure 5.2 The function code editor.

 

If you are already familiar with writing JavaScript, then you can skip the next section, which is a very quick primer on JavaScript, which is used when you are writing your own function nodes.

JavaScript Primer

While we don’t have enough space here to cover JavaScript completely, we’ve written a short primer  for those familiar with other programming languages to get started writing your own nodes. To experiment with JavaScript, you can use the JavaScript console supplied with your favourite browser or startup node.js to run the Read-Eval-Print-Loop (REPL) on your machine.

Read the JavaScript primer

Writing Function Nodes

Let’s get started writing function nodes.  To test these out, you’ll create a simple flow with an inject and a debug node as shown in Fig. 5.3.

Figure 5.3 Simple flow to test our function nodes.

The most simple function node just returns null. When you return null, no message is passed on to downstream nodes and the flow ends here. To create a node that passes the message “as is”, you can simply return the message itself. This is the default code in a function node.

  1. return msg;

Neither of these cases are very interesting, so let’s add some content to the payload. In this example, and the next set of examples that explore writing JavaScript code for the function node, you will use the same flow (i.e that shown in Fig 5.3) and you’ll edit the function node to change the JavaScript using the function node editor as shown in Fig 5.2.

Listing 5.12 Adding some text to the incoming message payload

1 msg.payload += “world”;

2 return msg;

In this example (listing 5.12), a simple string concatenation operator is used to add the string “world” to the incoming message payload (line 1). This updated payload is then sent, using return,  as the output message (line 2) to the next node in the flow.

If you edit the function node you’ve just created and type in the code from listing 5.12 when you deploy and test the flow, you will see in the debug pane the timestamp for the inject node and the text “world” appended.

Obviously, if you now edit the inject node to inject a string instead of the timestamp, and set the injected string to “hello”,you will see “hello world” in the debug pane, when you deploy and test.

Example 5.1 Creating and returning a new message in a function node

Listing 5.13 shows you how to create a new message by defining a new message newMsg, assigning the string “new payload” to the property payload (line 1) and then returning the new message at line 2.

Listing 5.13 Creating and returning a new message

1 var newMsg = { payload: “new payload” };
2 return newMsg;

If you modify the test function node you set up as shown in Figure 5.2 and use the code from Listing 5.13  when you deploy the flow and hit the inject button, the new message you created is output when it arrives at the debug node.

Example 5.2 Creating and returning multiple messages in a function node

The function node also allows you to configure it with multiple outputs. Once you’ve configured multiple outputs, you can send multiple messages, one to each output, using an array.

First, let’s edit the function node to tell Node-RED that the node will have three outputs, using the node configuration window (Fig. 5.4).

Figure 5.4 Setting the number of outputs that a function node provides

Listing 5.14 illustrates how to send a message to one of three different outputs based on a message payload value ‘high’, ‘med’ or ‘low’:

Listing 5.14 Returning message on multiple outputs using an array of messages

  1. if (msg.payload == “high”) {
  2.   return [ msg, null, null ];
  3. } else if (msg.payload == “med”) {
  4.    return [ null, msg, null ];
  5. } else {
  6.    return [null, null, msg];
  7. }

Line 1 checks the incoming payload to see if it is set as “high”. If yes, then it passes on the message on the first output and null messages on the 2nd and 3rd output at line 2 by returning an array with three elements, [msg, null, null]. A “med” message causes a message to be returned on output 2 (line 4). Anything else returns a message on output 3 (line 6).

To try it out, wire it up to three inject nodes and three output nodes as shown in Figure 5.5. Set the different inject nodes to inject a text string of either low, med, high, edit the test function and use listing 5.14, and then deploy. As you select the different inject nodes, you will see the appropriate debug node fire in the debug pane.

Figure 5.5 Flow to test multiple function outputs

It is also possible to send multiple messages from a function node in sequence. To do this, you return an array of messages in the output message array. Listing 5.15 shows how to write a function that returns 10 new messages containing number payloads from 0 to 9. Line 1 creates a new array, msgList, to hold the messages. Lines 2 and 3 loop 10 times adding a new element to the array msgList. Finally, line 5 returns an array, containing 1 element, msgList, which is itself an array of 10 elements. As you saw in the previous example, by returning an array with one element, you are sending a message on the 1st output of the node. However, this time, because the array element actually contains another array msgList, which itself has 10 elements, Node-RED sends each element as a series of sequential messages on output 1.

Listing 5.15 Returning multiple messages on one output using an array of messages

  1. var msgList = [];
  2. for (var i=0; i<10; i++) {
  3.   msgList.push({payload:i});
  4. }
  5. return [msgList];

Node functions

Your function node code has access to a few utility functions from the built-in node module. These allow you to log text to the console or debug output and send messages from callback functions rather than returning them from the function node itself.

  • node.log() – logs messages to the console
  • node.warn() – logs messages to the console and a warning message in the debug pane
  • node.error() – logs to the console and an error in the debug pane.

In FRED, the Node-RED console can be viewed by going to the landing page, then clicking on the “Status” menu in the drop down menu under your name.  Since it can be a bit cumbersome to switch back and forth between the NR editor and the console, you can open the status dialog in a separate tab, then hit the refresh button to see updates to the console.  Since the node.warn and node.error functions also output to the debug pane, it is sometimes useful to utilize these functions for debugging when the console is not available.

In addition to these logging and debugging functions, the node.send() function can be used to send messages to downstream nodes in a callback function, rather than returning the message(s). Listing 5.16 illustrates how to use node.send() in a function node that sends a message after a specific amount of time.

Listing 5.16 Example of how to use node.send in a callback

  1. // send message after 10 second delay
  2. setTimeout(function() {
  3.     node.send(msg);
  4. }, 10000);
  5. return null;

Other modules that are available to function nodes include the following:

  • console – while the Node.js console module is available, node.log() is the preferred method of logging
  • util – the Node.js util module. This module provides useful utilities such as string formatting functions and generating string representations of objects. For more informaton on the util module see the Node.js documentaiton at https://nodejs.org/api/util.html
  • Buffer – the Node.js Buffer module is for handling raw binary data from TCP streams or from files. For more informaiton on the Buffer module, see the Node.js documentation at https://nodejs.org/api/buffer.html.

Example 5.3 Using Context in a function node

A special module called context,used to store data between function invocations, is available to function nodes. This can be useful when the function needs to retain state to do its processing.  For example, let’s use context to count the number of messages that a function node has processed since it was deployed. Let’s create a function node that adds a count property to each message it processes, as in Listing 5.17.

Listing 5.17 Function to count messages.

  1. if (!context.value) {
  2.   context.value = 0;
  3. }
  4. context.value +=1;
  5. msg.count = context.value;
  6. return msg;

It can be wired up, as usual based on that shown in Figure 5.3), to give it a try. Be sure to change the debug node to display the ‘complete msg object’ so you can see your count property. You’ll probably notice that context is reset every time the function node is deployed. If you need to save state between deployments, you’ll need to use some external storage such as a file or a database.

Example 5.4 Using Global Context

In addition to individual function node context, a global context module is available to share context between function nodes.  Let’s use this module to see how accurate a delay node is, as shown in Fig. 5.6.

Figure 5.6 Flow to calculate the time elapsed for a message to go through the delay node.

The code for the function node called save the start time is shown in Listing 5.18.

Listing 5.18 Code to save the start time of the flow in a global context property.

  1. context.global.startTime = new Date().getTime();
  2. return msg;

Another function node, called time elapsed (see Listing 5.19), generates a message containing the time elapsed since a message passe through the save the start time function.

Listing 5.19 Code to calculate the elapsed time.

  1. var currentTime = new Date().getTime();
  2. var timeElapsed = (currentTime – context.global.startTime)/1000;
  3. msg.payload = “Time elapsed is: “+timeElapsed+”s”;
  4. return msg;

Add and configure a delay node to delay the message for 2 seconds, click on the inject node, and wait about 2 seconds.  If all goes well, the debug output should indicate the time elapsed was close to 2 seconds.

The advantage of this approach is that we need not have a message that carries the start time. The disadvantage is that, if another message arrives at save start time before time elapsed, the context.global.startTime variable will be overwritten.

Saving your Function code in a library

Once you have created some useful code for use in a function node, it is often useful to save that code for later reuse. One way to do this would be to cut and paste it into a new function, but to save you time, you can use the function node library that is built into Node-RED.

To save your code in the function library, click on the Book icon in the function node configuration dialog to the right of the name text edit box, and select “Save to Library”.  You can then provide a folder name and a file name to save your code as shown in Fig. 5.7.

Figure 5.7 Save to library dialog

To make use of your code you can create a new function node, click on the book icon, then click on “Open Library…” then choose your saved library code. The code will be loaded in the dialog. If that’s the code you want, click OK, and it will be added to your function node. You can then edit it as needed.

Sub-Flows

Sub-flows are a way to reuse useful flow segments in Node-RED as fully fledged nodes in the node pane that you can drag into your flows. While it’s possible to save flow segments using the flow library (as described at the end of lecture 2), having flow segments in the node pane provides a level of encapsulation and information hiding that importing saved flows doesn’t offer.  Encapsulation means that it is organized into a single item (node) that can be referred to using a single name; information hiding means that the inner workings of the sub-flow are hidden – you can change how the sub-flow does its job and the flows that use it will not change.

They can be created in two ways: either by selecting parts of your flow and creating the sub-flow from the selection, or from scratch. To create one from a selection, select the nodes in the sub-flow, click on the menu > Subflows> Selection to Subflows.  This will immediately create a new, numbered sub-flow in the node pane under the “sub-flows” category.

You can also create a new sub-flow from scratch by clicking on the menu > Subflows > Create Subflow menu. After creating a new sub-flow, or editing an existing sub-flow by double-clicking on it in the node pane, a sub-flow tab will appear in the main pane. Unlike a regular tab, this tab has four buttons in the top left: “edit name”, “+ input”, “+ output” and “delete subflow”. In this pane, you can edit the sub-flow, add nodes by dragging and dropping them to the sub-flow pane, and wire them up as needed. To connect this sub-flow to the flows that use it, you can add a single input node and any number of output nodes, using the +input and +output buttons.

Example 5.5 Creating a sub-flow

Let’s create a sub-flow that switches outputs depending on the sentiment of messages received. This can be used to analyze messages from various sources, including Twitter.  First, create a sub-flow: Menu>Create Subflow.  Drag a Sentiment analysis node to the pane, then drag a Switch node. Wire up the Sentiment flow to the switch and configure the switch as in Fig. 5.8.

Figure 5.8 Switch node configuration for sentiment switch sub-flow

This will output a message to output 1, if the score is negative, to output 2, if it is neutral, and to output 3, if it is positive. Now, add an input and three output nodes. Wire up the input to the sentiment and the output nodes to the switch as in Fig. 5.9.

Figure 5.9. Sentiment switch sub-flow.

To edit the name of the sub-flow, click on ‘edit name’ in the pane and call it something like Sentiment Switch. You will see your new node in the sub-flows section in the node pane.

Figure 5.10 Flow using sentiment switch sub-flow to display positive tweets about hockey and the canucks.

Let’s use this to output only the positive messages about hockey from a Twitter feed.  Add a new tab, drag in a Twitter node, the new Sentiment Switch sub-flow node you created and a debug node, as shown in Fig. 5.10.  Wire up Twitter to the input and debug to the third output node of your new Sentiment Switch.  After a minute or so, hopefully, someone will tweet something positive about hockey, and you’ll see some output in the debug pane!

Example 5.6 Packaging Function nodes using sub-flows.

Sub-flows are also a nice way to package up function nodes so that they appear in your node pallette and can be dragged into your flows for easier reuse. To illustrate this, let’s package up our counter function inside a sub-flow.  To get started, create a new sub-flow tab. Drag in a function node and add the code from Listing 5.17 – the one that adds a counter property using context.

Create a sub-flow as before and add a function node. Add the code from Listing 5.17.  Then hook up inputs and outputs to our sub-flow as shown in Fig. 5.11. Let’s name this subflow ‘Counter’.

Figure 5.11 Counter sub-flow that wraps function node.

 

To test out our new flow, drive it from an inject node and set up a debug node, as shown in Figure 5.12. Be sure to show ‘complete msg object’ in your debug node to see the counter message property that was added.

Figure 5.12 Test for Counter sub-flow.

Now you have a ‘Counter’ sub-flow node you can drag in any time you like and use it in multiple locations in a flow.

 

 

Example 5.7. Creating a Slack Bot to query data

Let’s finish off this lecture with a more complex example that uses one of the extended node set that FRED provides – the slack bot node and queries data from the Openweathermap platform.

Openweathermap is a platform that provides weather forecast data from all over the world. By using the APIs of the website, we are able to query any location in Slack. You will need to setup a developer account on Openweathermap in order to use the openweathermap node on FRED.

This example will show you how to build a Slack integration that allows a slack user to query the openweathermap platform and returns weather data based on the simple query term.

For this example you will need to create what Slack calls a Slack Bot User (https://api.slack.com/bot-users) that will serve as a virtual user in your Slack application. So go ahead and create new bot (https://my.slack.com/services/new/bot), see Fig 5.13. If you don’t already have a Slack account, you will be guided through the account creation before you can return to the bot service.

Figure 5.13 Creating a new Slack Bot

 

After creating your Slack Bot you will be given an API Token in the form (Fig. 5.14):

 

  1. xoxb-6816691748-vkIsdfsafE1RgF251yjPSm90WAAsadfSDF1

Which you should note down as you will need this token when you set up your flow.

Figure 5.14 Slack returns a new API token for access to the Bot

Next, you will need to have the openweathermap node installed on your FRED instance. Simply install openweathermap node in “Add or Remove Nodes” from the side bar. 

To do that, you should log into your FRED account and set up a new canvas for the flow. Drag and drop a Slack Bot In node into your canvas. Double click on it and configure by adding your Slack Bot API Token (Fig. 5.15). By default it will only listen into the #general channel (you can change this).

Figure 5.15 Configuring the Slack node with API token and channel

Now connect a function node and double click on it. You will need to set the function node with 2 outputs. Copy and paste the following function (Listing 5.20, Fig. 5.16):

Listing 5.20 A simple function to parse a text-based query

var query = msg.payload.split(",");
if (query.length == 2){
 msg.location={
 city:query[0],
 country:query[1]
 }
 return [msg, null];
} else {
 msg.payload="invalid search";
 return [null,msg] 
}

Figure 5.16 Writing some function code to parse for queries on a slack channel

All this function does is parse the messages received by our Bot. If the message contains two words and the words are separated by “,”, it will slice the string and create a query parameter in the output message. This will become handy in the next step.

Next, connect an openweathermap node to your function node and don’t forget to add the API key in the configuration tab.

Then, you need to add a function node after the openweathermap node to parse the output response.

Simply add in the following code inside the function node:

var newMsg = msg;
newMsg.payload="The city you asked was "+
msg.location.city+", "+msg.location.country+", "+ 
"the weather is "+msg.payload.weather+" with "+ 
msg.payload.detail+". "+ 
"The temperature is "+msg.payload.tempc+"C right now.";
return newMsg;

Finally, you’ll need to add a new slack bot out node and configure it with your Bot API Token and Channel. The final flow will look like this (Fig 5.17).

Figure 5.17 Configuring the slack bot out node

Now, to test that the flow work, return to your slack home page and either select the bot you created from the panel on the left. You’ll be presented with an input box, so type in:

1 Vancouver, Canada

The bot will query weather data from openweathermap and parse the weather forecast data back to the slack bot (Fig 5.18).

Figure 5.18 Results of querying ‘weather’ via your new slack bot

You are welcome to developer your own query features to find out all other possible functions you can have with FRED!

 

Summary

This lecture reviewed some of the key concepts behind Node-RED and introduced the context and global context objects used to share and maintain state between messages received by function nodes. The JavaScript overview in this lecture should enable you to start programming your own function nodes. Through some additional examples, you’ve seen how function nodes are an important and powerful aspect of Node-Red, allowing you to extend it as needed when a built-in node doesn’t quite do the job. This lecture also gave an introduction to sub-flows, demonstrating how they can be used to reuse flow segments and package up function nodes for easier reuse in your flows. Finally you’ve seen the use of the function node to develop a flow that uses a Slack Bot to make general queries to get data from public APIs.

 


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

 

 

 


[1] Node-RED, although often referred to as a Flow based language actually isn’t. The classical definition of flow based languages requires features that Node-RED doesn’t provide.

[2] It is possible for a function or other node to send any JavaScript type to an output endpoint to downstream nodes, but by convention, they should send message objects containing properties.

[3] Interestingly the current version (0.11.1) of Node-RED doesn’t have a maximum number of outputs.

[4] Currently (version 0.11.1 of Node-RED) only function nodes have access to this type of object, but this may be extended to other nodes in future.

Node-RED: Lecture 4 – A tour of the core nodes

This lecture will take a look at the core set of nodes that Node-RED installs by default and then show you the extended set of nodes that the cloud-based Node-RED service – FRED – supports.

This lecture will take a look at the core set of nodes that Node-RED installs by default and then show you the extended set of nodes that the cloud-based Node-RED service – FRED – supports.

For each node, you’ll see a brief overview of its functionality and an indication of which example in this lecture series uses the node, so you can look in more detail at how to work with the node. This lecture is mostly a reference section. However, it is worth looking quickly through the nodes, so that you have an idea of the basic functionality available to you as you begin to craft your own flows.

Node-RED’s default set of nodes

When you install Node-RED for yourself on a device such as a Raspberry Pi or Beagleboard, it starts up with a default set of nodes. There are 8 main categories of nodes in the default install: inputs, outputs, functions, social, storage, analysis, advanced and Raspberry Pi (See Figs 4.1 and 4.2) Let’s take a look at each category in turn.

 

Figure 4.1 The default input, output, function and social media nodes.

Input nodes

There are 7 basic input nodes installed by default. They cover the basic communications mechanisms that IoT applications are likely to use. Ranging from lower-level internet protocols such as UDP and TCP through to the higher-level HTTP and the publish/subscribe MQTT.

Node name Description Examples
inject Injects a timestamp or user-configured text into a message. Can be configured to inject manually, at a set interval, or at specific times (using Cron). Examples 2.1, 3.6, 3.7, 3.8, 5.1-5.4, 6.1-6.8
catch Catches errors thrown by nodes on the same tab. If a node throws an error whilst handling a message, the flow will typically halt. This node can be used to catch those errors returning a message with an error property detailing the error and the source node and type. Examples
mqtt Subscribes to an MQTT broker and listens on a topic, returns any data published on the topic as a new message. Supports Quality of Service levels and last data retention. Examples 3.1-3.5
http Receives HTTP requests, allowing Node-RED to act as a basic web server. HTTP body is delivered as an output message along with any response. Message can contain standard URL-encoded data or JSON. Example 1.3, 5.7
websocket Provides an endpoint for a browser to establish a websocket connection with Node-RED. Offers a duplex connection for browser/server combinations. Example 3.7
tcp Used to accept incoming TCP requests on a specified port or to connect to a remote TCP port. Generates messages containing the TCP data as a single – or stream of – buffer, string or base64 encoded. Examples lecture 7
udp Used to accept incoming UDP packets (or multicast packets) on a specified port. Generates messages containing the UDP data as a BUFFER, string or base64 encoded string. Examples lecture 7
serial in Reads from a serial port on the local device. Can be configured to read buffers, a specific time period or wait for line breaks. Examples lecture 7

Output nodes

The output nodes are essentially the mirror images of the basic set of input nodes and provide a way to send data on the same set of protocols, i.e. mqtt, http, udp etc.

Node name Description Examples
debug Provides a simple way to view messages which are displayed in the debug pane. Can be configured to display just the msg.payload or the entire msg object. Various, examples 2.1, 2.2, 3.1-3.6, 5.1-5.4, 6.1-6.5
mqtt Subscribes to an MQTT broker and posts any data (msg.payload) it receives in incoming messages to a topic. Supports Quality of Service levels and last data retention. Example 3.6
http Sends responses back to HTTP requests received from a HTTP Input node. The response body is determined by msg.payload, and can have headers and status code defined. Examples
websocket Sends msg.payload out on the websocket configured. If msg._session is defined, sends to the origination client, otherwise broadcasts to all connected clients Example 3.7
tcp Replies to a configured TCP port. Can also be used to send to a specific port. Example 3.8
udp Sends a UDP message to the configured host (ip address) and port. Supports broadcast. Like most nodes, configured through UI or message properties. Examples
serial out Sends to the defined serial port. Can be configured to send an optional newline after any message payload. Examples lecture 7

 

Function nodes

The function category contains a variety of nodes that carry out specific processing functions. These range from the simple delay and switch nodes to the programmable function node that can be adapted to almost any programming need.

Node name Description Examples
function Generic programmable function node. Using standard JavaScript, the node can be tailored to carry out complex processing on its input messages generating one or more output messages. Examples 2.1, 2.2, 3.8, 5.1-5.4, 5.7, 6.1-6.8
template Configured with a template (using moustache format) of arbitrary complexity, this node takes an input message containing name:value pairs and inserts into the template. Useful for constructing messages, HTML, config files, etc. Example 1.3
delay A generic node that delays messages by a specific or random time. Can also be configured to throttle a message flow (e.g. 10 msg per sec). Examples 5.6, 6.8
trigger Creates two output messages separated by a configurable time interval whenever an input message is received. Can also be used as a watchdog timer. Example 1.1
comment A simple visual comment configured with title and body. Example 2.1
http request Allows you to construct and send a HTTP request to a specific URL. Method (PUT, GET, etc), headers and payload are all configurable through the UI or programmatically. Examples 1.3, 6.1, 6.6
tcp request A simple TCP request node. It sends the msg.payload to a server tcp port and expects a response. Can be configured to wait for data, wait for a specific character, or return immediately. Example 3.8
switch This node routes messages based on their properties. Properties are configured using the UI and can be a variety of logic (>, <, >= etc) applied to a message property. Examples 3.2-3.5, 5.5
change The change node can be used to set, change or delete properties of incoming messages. A variety of configurable rules allow complex changes including search and replace in the msg.payload Examples  3.3-3.5

 

range A simple scaling node that will map numerical input to a new output. Useful for converting or bounding ranges of input values, e.g. temperature. Undefined for non-numeric data. Example 3.5

 

Node name Description Examples
csv This node parses msg.payload and tries to convert to/from CSV. If it receives a string, it outputs a JavaScript object, and if it receives a JavaScript object, it outputs a CSV string. Examples
html Extracts elements from an html document in msg.payload using a configurable selector (CSS selector syntax). Essentially allows you to parse out the HTML and returns an array of the elements that match. Example 6.1
json This node converts to/from a JSON object. If it receives a JavaScript object, it outputs JSON, and if it receives JSON, it outputs a JavaScript object. Examples 3.1-3.5, 5.7, 6.6
xml This node converts to/from XML format. If it receives a JavaScript object, it outputs an XML string, and if it receives an XML string, it outputs a JavaScript object. Examples
rbe Report By Exception node. Generates a message only when its input is different from the previous input (string or number) or if the input has changed by a configurable amount (deadband mode) – only for numbers. Example 3.4

Social nodes

The basic social media nodes support interaction with email and with Twitter. They enable flows to send or receive email, or to send or receives tweets.

Node name Description Examples
email in Can be configured to repeatedly read from an IMAP server returning new email as it arrives. Sets msg.topic to email subject and either msg.payload to email text body or msg.html if the email is HTML. Example
twitter in Returns tweets as messages. Can be used to search the public or a user’s stream for tweets containing the configured search term or all tweets by specific users or direct messages received by the authenticated user. Example 1.1
email out Sends the incoming message as an email via the configured IMAP server. Topic and recipient all configurable. Will convert binary data to an attachment. Example 1.2
twitter out Tweets the msg.payload on the configured account. Can send direct messages and will send binary data as an image. Example 2.2

 

Storage nodes

The default node set for storage, because it is targeted at devices such as the Raspberry Pi are quite limited and focus on file based storage.

Figure 4.2 The default storage, analysis, advanced and Raspberry Pi nodes.

 

You should note that FRED, because it’s a cloud service, doesn’t support the basic file nodes. Instead it replaces these with a variety of storage nodes ranging from Mongo to Dropbox. However, for completeness the default nodes are covered here, so that you can use them, if you install Node-RED yourself (see lecture 7).

Node name Description Examples
tail Tails (i.e. watches for things to be added) to the configured file. (Linux/Mac ONLY)

This won’t work on Windows file systems, as it relies on the tail -F command.

Examples
file in Reads the specified file and sends the content as msg.payload, and the filename as msg.filename.

The filename can be configured in the node. If left blank, it should be set on msg.filename in an incoming message.

Examples
file Writes msg.payload to the file specified, e.g. to create a log. The filename can be configured in the node. If left blank, it should be set on msg.filename in an incoming message.  The default behaviour is to append to the file. This can be changed to overwrite the file each time; for example, if you want to output a “static” web page or report. Examples

Analysis nodes

Analysis nodes perform standard analyses on incoming messages. In the default node set, the only node provided is the sentiment node, which can be used to try and determine the sentiment of an incoming message based on the words used in the message, for example, an email or tweet.

Node name Description Examples
sentiment The sentiment node analyses the msg.payload and scores the sentiment of the message based on word analysis. It adds a msg.sentiment object that contains the resulting AFINN-111 sentiment score as msg.sentiment.score. Score typically ranges from -5 to +5. Example 5.5

Advanced nodes

A set of miscellaneous nodes offering various types of functionality.

Node name Description Examples
watch Watches a directory or file for changes. You can enter a list of comma-separated directories and/or files. You will need to put quotes “…” around any that contain spaces. On Windows, you must use double backslashes \\ in all directory names.

The full filename of the file that actually changed is put into msg.payload, while a stringified version of the watch list is returned in msg.topic.

msg.file contains just the short filename of the file that changed. msg.type has the type changed, usually file or directory, while msg.size holds the file size in bytes.

Examples
feedparse This node monitors an RSS/atom feed for new entries and delivers new entries as messages. It can be configured to query the feed at a specific interval. Examples
exec Calls out to a system command and provides 3 outputs: stdout, stderr, and return code. By default uses exec(), which calls the command, blocks while waiting for completion, and then returns the complete result in one go, along with any errors. Examples

 

Raspberry Pi nodes

Node name Description Examples
rpi_gpio in Raspberry Pi input node. Generates a msg.payload with either a 0 or 1, depending on the state of the input pin. You may also enable the input pullup resistor or the pulldown resistor.

The msg.topic is set to pi/{the pin number}

Requires the RPi.GPIO python library version 0.5.8 (or better) in order to work.

Note: we are using the actual physical pin numbers on connector P1 as they are easier to locate.

Examples lecture 7
rpi_gpio out Raspberry Pi output node. Expects a msg.payload with either a 0 or 1 (or true or false). Will set the selected physical pin high or low, depending on the value passed in. The initial value of the pin at deploy time can also be set to 0 or 1. When using PWM mode, expects an input value of a number 0 – 100.

Requires the RPi.GPIO Python library version 0.5.8 (or better) in order to work.

Examples 1.1, lecture 7
rpi_mouse Raspberry Pi mouse button node. Generates a msg.payload with either a 1 or 0 when the selected mouse button is pressed and released. Also sets msg.button to the code value, 1 = left, 2 = right, 4 = middle, so you can work out which button or combination was pressed. Examples lecture 7

 

The extended FRED node set

The FRED service adds a number of nodes to the standard default set. These new nodes, which have either been written for FRED or collected from the public repository, have been added, as they offer useful functionality that expands the capabilities of the vanilla node set.

Note: these nodes are available in FRED by installing them using the FRED installation panel. For a quick example of how to install nodes check out this tutorial.

As you will see, the majority of these additional nodes focus on services and capabilities that match well with FRED’s nature, i.e. a cloud-based service. In many cases they focus on using Node-RED for web based integration or to access enterprise level services.

Screen Shot 2015-10-09 at 3.03.41 PM.png

 

Figure 4.3 The extended FRED nodes

 

Extended set of social media nodes

As you can see, the FRED service adds a large number of social media nodes ranging from Pushbullet to Slackbot to Instagram.

Node name Description Examples
pushbullet in Connects to the popular Pushbullet service and receives Pushbullet data items from all your connected devices. Supports data, links and files. Examples
pushbullet out Allows you to send Pushbullet messages to all devices that have the Pushbullet app installed. Examples
XMPP in Receives messages from an XMPP instant messaging server. The buddy field indicates the buddy or room you are receiving from. Presence information is delivered on a second output link. Examples
XMPP out Sends messages to an XMPP instant messaging server. Uses topics to send to rooms (channels) and supports presence notification. Examples
slack A general node that provides a simple way to post on a slack channel specified via its webhook URL.  Can be configured with user name, emoji and to support attachments. Examples
slackbot in A node to use a bot you’ve created on Slack and provides a listener in any channel of which the Slack bot is a member. Outputs the msg.payload as the incoming message. and outputs msg.SlackObj with full Slack message details. Examples 5.7
slackbot out A node to use a bot you’ve created on Slack Sends the msg.payload to Slack based on the Bot API token provided. You can optionally override the destination channel if required – either in the edit dialogue or by setting msg.channel. Examples 5.7
delicious A node to save bookmarks to your Delicious account. The payload should contain the URL to save, and msg.title contains the bookmark name. An optional description field can be set. Examples
pinboard A node to save bookmarks to your Pinboard account. The payload should contain the URL to save, and msg.title contains the bookmark name. An optional description field can be set. Examples
flickr Saves photos to the configured Flickr account. msg.payload requires a Buffer with the image and can set the optional title, description and tag properties. Examples
foursquare Queries your Foursquare account for venues that meet a configurable set of requirements based on your location. Results can be passed back as a single message or a configurable set of messages. Examples
swarm in A node to poll every 15 minutes for Foursquare storm check-ins. Returned as a JSON object. Examples
swarm out Swarm query node that can be used to search for all Swarm check-ins by the authenticated user. Examples
instagram in Queries the configured Instagram account every 15 minutes for new photos which are delivered one per message, either as Buffer objects or URLs. Examples
instagram Same as Instagram in node, except from being triggered by an incoming message. Examples

Extended set of storage nodes

As you can see, the FRED service adds a large number of social media nodes ranging from Pushbullet to Slackbot to Instagram.

Node name Description Examples
amazonS3 watch Amazon S3 watch node. Watches for file events. By default, all file events are reported, but the filename pattern can be supplied to limit the events to files which have full filenames that match the glob pattern. The event messages consist of the full filename in msg.payload property, the filename in msg.file, the event type in msg.event. Examples
amazonS3 in Amazon S3 input node. Downloads content from an Amazon S3 bucket. The bucket name can be specified in the node bucket property or in the msg.bucket property. The name of the file to download is taken from the node filename property or the msg.filename property. The downloaded content is sent as msg.payload property. If the download fails, msg.error will contain an error object. Examples
amazonS3 out Amazon S3 out node. Uploads content to an Amazon S3 bucket. The bucket name can be specified in the node bucket property or in the msg.bucket property. The filename on Amazon S3 is taken from the node filename property or the msg.filename property. The content is taken from either the node localFilename property, the msg.localFilename property or the msg.payload property. Examples
box watch Box is an enterprise version of Dropbox. This node watches for file events on Box. By default all file events are reported, but the filename pattern can be supplied to limit the events to files which have full filenames that match the glob pattern. The event messages consist of the full filename in msg.payload property, the filename in msg.file, the event type in msg.event and the full event entry as returned by the event API in msg.data. Examples
box in Box input node. Downloads content from Box. The filename on Box is taken from the node filename property or the msg.filename property. The content is sent as msg.payload property. Examples
box out Box out node. Uploads content to Box. The filename on Box is taken from the node filename property or the msg.filename property. The content is taken from either the node localFilename property, the msg.localFilename property or the msg.payload property. Examples
dropbox watch Watches for file events on Dropbox.

By default all file events are reported, but the filename pattern can be supplied to limit the events to files which have full filenames that match the glob pattern.

The event messages consist of the full filename in msg.payload property, the filename in msg.file, the event type in msg.event and the dropbox.js API PulledChange object inmsg.data.

Examples
dropbox in Dropbox input node. Downloads content from Dropbox. The filename on Dropbox is taken from the node filename property or the msg.filename property. The downloaded content is sent as msg.payload property. If the download fails, msg.error will contain an error object. Examples
dropbox out Dropbox out node. Uploads content to Dropbox.

The filename on Dropbox is taken from the node filename property or the msg.filename property. You can pass in content either as a filename by setting the localFilename field or msg.localFilename property, or you can pass in content directly using msg.payload.

The file will be uploaded to a directory on Dropbox called Apps/{appname}/{appfolder}, where {appname} and {appfolder} are set when you set up the Dropbox application key and token.

Examples
MongoDB in Calls a MongoDB collection method based on the selected operator.

Find queries a collection using the msg.payload as the query statement as per the .find() function. Count returns a count of the number of documents in a collection or that match a query using the msg.payload as the query statement.

Aggregate provides access to the aggregation pipeline using the msg.payload as the pipeline array.

You can either set the collection method in the node config or on msg.collection. Setting it in the node will override msg.collection.

Examples
MongoDB out A simple MongoDB output node. Can save, insert, update and remove objects from a chosen collection.

Save will update an existing object or insert a new object, if one does not already exist.

Insert will insert a new object.

Update will modify an existing object or objects.

Remove will remove objects that match the query passed in on msg.payload. A blank query will delete all of the objects in the collection.

Examples
mysql Allows basic access to a MySQL database.

This node uses the query operation against the configured database. This does allow both INSERTS and DELETES. msg.topic must hold the query for the database, and the result is returned in msg.payload.

The returned payload will typically be an array of the result rows. If nothing is found for the key, then null is returned.

Examples
postgres A PostgreSql I/O node.

Executes the query specified in msg.payload with optional query parameters in msg.queryParameters. The queryParameters in the query must be specified as $propertyname.

When receiving data from the query, the msg.payload on the output will be a JSON array of the returned records.

Examples

FRED’s IoT nodes

FRED adds a number of nodes to access popular IoT platforms, including Sense Tecnic’s Community edition of its popular IoT platform, WoTKit, and Bug labs’ dweet.io.

IoT platform nodes
wotkit in WoTKit Sensor Input Node which retrieves new data from a WoTKit Sensor. The node generates a message each time new data is received on the sensor within the WoTKit IoT platform. The message.payload contains a map of sensor field names to sensor values. The node is configured with WoTKit user credentials. The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id. Examples
wotkit out WoTKit output node to send data to a registered WoTKit sensor. The node will use the message.payload to create a data object.

The message.payload must contain an object of key-value pairs matching the sensor fields. The node is configured with WoTKit user credentials.The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id.

Examples
wotkit data WoTKit Historical Data Node which retrieves historical data from a WoTKit sensor by either number of elements or relative time before the request. The message.payload contains a map of sensor field names to sensor values. The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id. Examples
wotkit control out The WoTKit Control Output Node provides the ability to send events to the control channel of a registered WoTKit sensor/actuator. This node will use the message.payload object to create a control event. The message.payload must contain an object of key-value pair for each message. The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id. Examples
wotkit control input The WoTKit Control Input Node provides access to the control channel of a registered WoTKit sensor/actuator. When a control event is received by a WoTKit sensor this node will create a message.payload object containing the event. The sensor name should be in the form {username}.{sensorname}, or you can use the numeric sensor id. Examples
dweetio in Listens for messages from Dweet.io

The Thing ID should be globally unique, as they are all public. It is recommended you use a GUID. The Thing ID is set into msg.dweet, and the TimeStamp into msg.created.

Examples
dweetio out Sends the msg.payload to dweet.io

Optionally uses msg.thing to set the Thing ID, if not already set in the properties.

You need to make the Thing ID unique – you are recommended to use a GUID.

Examples

Extended set of analysis nodes

The FRED platform only adds a limited number of analysis nodes as the default set is quite extensive.

Node name Description Examples
smooth A simple but flexible node to provide various functions across several previous values, including max, min, mean, high and low pass filters. Only works on numbers and will fail if it can’t convert the input to a number. Examples
wordpos Analyses msg.payload and classifies the part-of-speech of each word, returning msg.pos as an array of nouns, verbs, adjectives, adverbs, etc.. Examples

 

Extended set of advanced nodes

FRED adds the following advanced nodes.

Node name Description Examples
ping Pings a machine and returns the trip time in MilliSeconds.

Returns false, if no response received within 5 seconds, or if the host is unresolveable.

Default ping is every 20 seconds, but can be configured.

 

Examples
sunrise Uses the suncalc module to generate an output at sunrise and sunset based on a specified location. Several choices of definition of sunrise and sunset are available.

The first output emits a msg.payload of 1 or 0 every minute, depending if in between selected times or not. The second output emits only on the transition between night to day (-> 1) or day to night (-> 0).

Also sets msg.topic to sun and msg.moon to the fraction of the moon between 0 and 1.

Examples

Other nodes

The following nodes are available in FRED and don’t fit into the basic categories that vanilla Node-RED provides. These include nodes to access and control Google services, enterprise nodes such as Salesforce access, nodes for accessing transportation API (e.g. Transport for London) and other miscellaneous nodes for access to weather services, etc.

 

Fig 4.4 FRED’s other nodes

 

Google nodes

FRED supports several Google nodes to interact with a range of Google services.

Node name Description Examples
Google plus Interacts with the Google+ API to get information about people, activities, and comments.

People – Allows you to interact with Google+ profiles. You can get a particular profile, search for a profile, or gather a list of people that have +1’d or reshared an activity.

Activities – Allows you to interact with Google+ activities. You can get a particular activity, search for an activity, or gather a list of activities directly related to a person.

Comments – Allows you to interact with Google+ comments. You can get a particular comment, or gather a list of comments attached to an activity.

Examples
googleplaces A highly flexible node that utilizes the Google Places API in order to find and learn more about local establishments. Search can be based on ranking, radius, price, keywords, language, etc. Examples
Google calendar in A node to watch a calendar and return a message before, at or after the event in the calendar. Configurable to set exactly when the message is generated. Examples
Google calendar out Create an entry in a Google Calendar based on:

payload – either a string to describe the event using quick add format or an object representing the request body for an insert request

calendar – the calendar to which the event is added (optional, defaults to the node calendar property or the user’s primary calendar)

sendNotifications – a Boolean to determine if notifications should be sent to attendees (optional, defaults to false)

Examples

Fitness nodes

The FRED fitness nodes cover a number of popular fitness devices.

Node name Description Examples
strava Get your most recent activity on Strava.

This node returns the most recent activity in the authenticated user’s account whenever it receives a message. Returns the activity, location and time when available.

Examples
Fitbit in Polls Fitbit for new data at regular intervals. The generated messages are determined by the nodetype property (goals, sleep, or badges). Examples
Fitbit activities Retrieves user data from Fitbit and returns a msg.payload determined by nodetype properties (as fitbit in).

The msg.date property may be set to an ISO 8601 format date (e.g. 2014-09-25) to retrieve historical data for activities and sleep log. If no date is supplied, then data for today will be retrieved. In the case of sleep, this is the data for the preceding sleep.

Examples
jawbone The Jawbone Up node can be used to retrieve the workouts completed since the provided time (given as Epoch). This time can be passed in as settings on the node or as the msg.starttime section of the message input. The value set on the node will take precedence over the contents of incoming message. Examples

Weather nodes

FRED adds a set of nodes to give access to a variety of weather services.

Node name Description Examples
openweathermap A node to query the openweathermap.com site for weather information of a city/country or lat/long pair. Two nodes exist: one that is UI-configured and one that can accept the configuration info as an input message. Examples
forecastio A node to query the forecastio.com site for weather information of a lat/long pair. Two nodes exist: one that is UI-configured and one that can accept the configuration info as an input message. Examples
wunderground A node which queries The Weather Underground API for current weather data periodically and returns when a change is detected. Two nodes exist: one that is UI-configured and one that can accept the configuration info as an input message. Examples

Salesforce nodes

A set of experimental SalesForce nodes are available with FRED.

Node name Description Examples
salesforce A set of 5 nodes that interact with the popular salesforce.com service. The nodes can generate SOSL queries, DML statements, subscribe to the salesforce streaming API or parse outbound salesforce message objects. Examples

Transport nodes

Node name Description Examples
TfL bus Get live bus departure/arrival info for London (UK) buses and river buses.

This node enables the user to get bus or river bus arrival information for selected lines arriving at selected stops. The node returns the first vehicle/vessel to arrive at a particular stop. The data is provided by Transport for London.

Examples
TfL underground A node to get the underground line status for the London Underground. It returns a variety of status information for the specified line including overall status, disruption info etc Examples

Formatting nodes

Node name Description Examples
moment moment is a date/time formatter node that takes as input either a JS datetime object or a string that moment can resolve to such. If the input is null, doesn’t exist or is a blank string, the current date/time will be used. This can be used to add a current timestamp to a flow of any kind easily.

Output is a formatted string or a date object onmsg.payload by default, change by setting the output field.

Examples

Summary

In this lecture, you were presented with a summary of the default Node-RED nodes available on installation, and the extended set of nodes that FRED adds. As you have seen, there is a wide variety of nodes allowing you to build complex flows with little or no programming.

You’ve also seen that not all nodes are always available, for example the Raspberry Pi nodes don’t work unless you are actually running Node-RED on a Pi. So it always pays to check that you can run the nodes you need in your environment.

However, given the large number of nodes available, and the fact that the community is creating new nodes on a daily basis, you are likely to be able to find a node that meets your needs. If not, you can fall back on the flexible function node, or even create your own, something that will be discussed in the advanced lectures.


About Sense Tecnic: Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures, and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


 

© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

Node-RED: Lecture 3 – Basic nodes and flows

In this lecture you will get to see a few of the more commonly used nodes and build on some of the things you learnt in previous lectures. You’ll start off with a series of examples based around the popular MQTT protocol that shows how to wire together a set of basic but very useful message processing nodes. Then you’ll briefly look at other ways to get messages into and out of your flows using protocols such as TCP, UDP and websockets.

In this lecture you will get to see a few of the more commonly used nodes and build on some of the things you learnt in previous lectures. You’ll start off with a series of examples based around the popular MQTT protocol that shows how to wire together a set of basic but very useful message processing nodes. Then you’ll briefly look at other ways to get messages into and out of your flows using protocols such as TCP, UDP and websockets.

At the end of this lecture you will have a better understanding of some of the basic nodes used in Node-Red flows. You will also have seen how easy it is to build sophisticated processing flows that take real-world events, process them and generate responses that use popular Internet protocols to communicate the results outside of your flow.

 

We use a cloud hosted version of Node-RED for these lectures called FRED. Sign up for a free account at FRED. Examples in the early lectures will work with other installations of Node-RED, later lectures use nodes you will need to install yourself if you don’t use FRED.

 

Examples

Example 3.1 Receiving JSON via an MQTT message

Example 3.2 Using the switch node to handle a JSON object

Example 3.3 Using a change node to change or manipulate a message payload

Example 3.4 Using the rbe (report by exception) node

Example 3.5 Scaling input with the range node

Example 3.6 Using an mqtt output node to test the flow

Example 3.7 Using Websockets with Node-RED

Example 3.8 Sending TCP requests.

Example 3.1 Receiving JSON via an MQTT message

The following series of examples builds on the mqtt node, which provides a convenient way to take input from an MQTT broker. For those not familiar with MQTT, it is an example of a publish/subscribe system (usually shortened to pub/sub system) which lets sensors publish updates that all delivered to client subscribed to that sensor. MQTT uses a topic model allowing publishers (eg sensors) to create topics and publish data to the topics Equally, others can subscribe to a topic and will receive asynchronous notification of data posted to the topic.

Pub/Sub systems are a great way to connect loosely coupled distributed systems and they map well to typical IoT patterns where devices or things generate events that you want to share. The MQTT protocol, apart from being asynchronous, is also lightweight and doesn’t have as high an overhead as HTTP; which for resource-constrained devices is often an important advantage. MQTT was originally developed in the late 1990s and has been used in a variety of IoT settings. MQTT became an OASIS standard in 2014 and is a standard part of many IoT toolboxes. MQTT actually stands for Message Queueing Telemetry Transport.

To use the mqtt node, you need to have access to a broker. There are a number of free MQTT servers running, for example http://test.mosquitto.org/, or the one that will be used in this lecture, www.hivemq.com. Using the broker address and the topic, you can configure the mqtt input node to subscribe on that topic, causing it to generate a new message whenever new data is published on that topic. The message will contain information on the published data, including the data itself in msg.payload and the MQTT broker topic in msg.topic.

To get you started with the mqtt node, you’ll be using the free mqqt broker hivemq – which is available via (http://www.hivemq.com/showcase/public-mqtt-broker/). Of course you can use any MQTT broker, including your own, if you have installed one.

First, drag and drop an mqtt input node and configure it for the broker. Don’t forget to configure the topic to something unique, in the case of this example we are using noderedlecture/sensor but you should use your own unique topic, i.e. <your name here>/sensor

Figure 3.1 Configuring the mqtt node with broker address and topic

There are many ways you can send mqtt messages to hivemq. You can use their websockets client showcase (http://www.hivemq.com/demos/websocket-client/), the mqtt dashboard (http://www.mqtt-dashboard.com/dashboard) or your own library. You’ll be using their websocket client in this example, so navigate to that page and connect to the broker. You will publish a JSON encoded string to the topic you configured to see both the use of the mqtt node and the json node.

Figure 3.2 Sending an MQTT message (publishing) using the HiveMQ client page.

Since you are sending a JSON string, you will need to parse the message that the mqtt node generates when it receives the MQTT message. To do that, you’ll need to drag and drop a json node and connect it to the mqtt node’s output.

Node-RED’s json node is a sort of convenience function, in that it parses the incoming message and tries to convert it to/from JSON. So if you send it a JSON string, it will convert it to a JavaScript object, and vice versa.

If you wire up the usual debug node to the json node and deploy, then use the HiveMQ dashboard to send the JSON string {“analyze”:false, “value”:10} as shown in Fig 3.2. You will see it printing in the debug tab (Fig 3.3).

 

Figure 3.3 Receiving and parsing an MQTT message sent as a JSON string

If you look closely at the output, you can see that the msg.payload contains an object, which itself has two fields, analyze and value, each with their own values. As you saw in lecture 2, you can access these fields via msg.payload.analyze and msg.payload.value. Let’s take a look at a node that can do that.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-1_mqqtmessages.json

Example 3.2 Using the switch node to handle a JSON object

One of the nice features of having a JSON object is that you can easily act on its properties. A useful node for this is the switch node. Its role is to ‘switch’ or route messages depending on the incoming message properties.  For example, you can check the msg.payload.analyze property and, depending on its value (true/false), decide to route a message to one of the switch node’s outputs.

Drag a switch node and double-click on it. Configure it to evaluate the property “msg.payload.analyze”. If true, send the message to the first output; if false, send it to the second output as shown in in Fig 3.4.

Figure 3.4 Configuring the switch node to route based on a message property

Now you can connect two debug nodes as shown in Fig 3.5 – when you set up multiple outputs for a node, they are numbered from the top, so output 1 is the top output and output 2 is at the bottom in  Fig. 3.5.

Figure 3.5 Connecting a switch node to two debug nodes

If you now go back to the HiveMQ input page and send the MQTT message {“analyze”:true, “value”:6}, you will see that the first (top) output is activated and the incoming messages is routed, or ‘switched’, to output 1. If you send the original message {“analyze”:false, “value”:10}, the switch node will activate output 2 and the original debug node will fire. Hovering the pointer over the debug message will show which debug node is printing out the message as shown in Fig. 3.6.

 

Figure 3.6 Confirming the operation of the switch node

As you can see, this provides you with a built-in Node-RED node that allows you to quickly determine the contents of incoming messages and direct the message through to different parts of the flow depending on the input.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-2_switchnode.json

Example 3.3 Using a change node to change or manipulate a message payload

Another useful node is the change node, which will allow you to change a message payload or add new properties. You can use this node to affect the properties in a message, either by changing existing ones, deleting them or adding new properties.

In this example, you’ll continue with your MQTT theme and see how, now that you have successfully ‘switched’ the message flow based on the incoming MQTT message, you can add a new message property msg.payload.note.

First, let’s drag and drop a change node and connect it to the second output of the switch node (Fig 3.7). As you will remember, this is the output that fires when msg.payload.analyze is set to false.

Figure 3.7 Adding in a change node to set a new message property.

 

Now configure it to set the property msg.payload.note to “this is not being analyzed” as shown in Fig 3.8.

 

Figure 3.8 Using a change node to add a new message property

 

When you receive a message that the switch node sends on the 2nd output, it will be modified to contain a “note” element with the string “this is not being analyzed”. If you deploy and test the flow by sending the MQTT message from HiveMQ, you’ll see the output as shown in Fig 3.9.

 

Figure 3.9 The result of switching and then changing the message

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-3_changenode.json

Example 3.4 Using the rbe (report by exception) node

In this example, you’ll continue your message analysis theme and add nodes to the part of the flow that is used when you determine that the flow should be analyzed. You’ll be using the rbe (report by exception) node which only passes on data if it has changed. You can set it to examine a message payload and either block until a message changes (rbe mode) or when a messages changes by a specified amount (deadband mode). In rbe mode, it works on numbers and strings. In deadband mode, it works on numbers only and uses the configured deadband as a + or – ‘band’, so that the incoming value can fluctuate within a range before it fires.

You’ll start by adding in another change node which you’ll connect to output 1 of the switch node. You’ll then connect an rbe node to the switch node as shown in Fig 3.10.

Let’s connect a change node, and an rbe node like this. To remind us that this output deals with the flag “analyze”, add a comment node and write “Analyze = true”. Comments are useful when writing complex flows.

Figure 3.10 Adding an rbe node to check if our input data has changed by more than 20%

Edit the change node to set the msg.payload to msg.payload.value. This will set the output of this node to the value found in the msg.payload.value element of the input received (Fig 3.11).

Figure 3.11 Using a change node to set payload

Since you want to determine if this value has changed by 20% or more, you’ll need to double-click on the rbe node and configure it to block unless the value changes by more than 20%.

Fig 3.12 Setting the rbe node to check msg.payload.value

To test the flow, deploy this flow and then return to the HiveMQ page and send a series of messages. Firstly, you’ll need to set the analyze value to true so that the switch node sends through a message on output 1. If you use the original message value of 6, this will fail to pass the rbe node. If you then send a second message of value 10, the rbe node will evaluate the difference between 6 and 10, see it is greater than 20%, and send on a message to the final debug node which will print on the debug pane as shown in Fig 3.13.

Fig 3.13 Confirming that 10 is more than 20% larger than 6 using an rbe node.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-4_rbenode.json

Example 3.5 Scaling input with the range node

When dealing with real world input from sensors and other devices, an ability to scale input data is often required. Node-RED provides the scale node to support this and allows you to scale (linearly) an input value. Let’s assume you want to scale your value (originally in the range 0-10) to a range (0-255) when you aren’t doing any analysis. This means we are dealing with the lower part of the flow fired when the switch node evaluates the Analyze property as false.

To do this, select the change node you configured above (set msg.payload) and copy it with ctrl+c, then ctrl+v. Attach a range node as shown in Fig 3.14.

 

Figure 3.14 Scaling input values using the range node.

Double-click on it, and configure it to map the input from 0-10 to 0-255 as shown in Fig 3.15.

The scale node has three options set by the action field. The default will scale according to the mapping given but will happily scale values outside the given ranges, using the same mapping. Scale and limit to target range means that the result will never be outside the range specified within the result range. A third option, Scale and wrap within the target range means that the result will essentially be a “modulo-style” wrap-around within the result range.

Then return to the HiveMQ test page and post {“analyze”:false, “value”:10} as a new MQTT message to the same topic.

Figure 3.15 Setting the scaling ranges for input and output of the scale node

If you return to your Node-RED window, you will see that the debug node associated with the lower part of the flow has fired, showing that the msg.payload.value property that you set as 10 when you published it to MQTT, has been scaled up 255 as shown in Fig 3.16.

Figure 3.16 Final scaled output when analysis is turned off

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-5_rangenode.json

Example 3.6 Using an mqtt output node to test the flow

As an alternative to using the HiveMQ test page to publish on the MQTT topic, we can configure an mqtt output node. This is the mirror of the mqtt input node and allows you to configure an MQTT service and the topic you are publishing on. You can then send the node messages with the exact same JSON string we’ve been sending via the HiveMQ test page.

To try that, drag and drop three inject nodes and an mqtt output node as shown in Fig 3.17, and configure and wire up as shown

Figure 3.17 using an mqtt node to publish on the noderedlecture/sensor topic

You can now test the flow you created to analyze the MQTT message directly from the workspace by clicking on the three inject nodes in sequence. The first will exercise the part of the flow handling the case when analysis is turned off; the 2nd two messages will cause the path with the rbe node to be exercised.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-6_mqqtout.json

Example 3.7 Using Websockets with Node-RED

Websockets are another useful communication capability that is built into Node-RED via the the websocket node. Websockets provide a duplex TCP connection and were designed to allow web browsers and servers to maintain a ‘backchannel’ that could be used to augment traditional HTTP interactions, allowing servers to update web pages without the client making a new pull request.

The websocket node comes in two flavours, input and output, allowing you to listen for incoming data (input) or to send (output) on a websocket. The output version is designed to check to see if the output payload originated at a websocket in a node, in which case it responds to the original sender. Otherwise it will broadcast the payload to all connected websockets.

In addition, both input and output websocket nodes can be configured as either server or client – in server mode they ‘listen on’ a URL, and in client mode they connect to a specified IP address.

To see how the websocket nodes work, you’ll use a public websockets echo server which runs on the public site: (https://www.websocket.org/echo.html).

Drag an inject, websocket in, websocket out and a debug node onto the workspace and connect them as shown in figure 3.18.

Figure 3.18 using websockets to communicate with a Node-RED flow

Configure the inject node to send a string payload of “Hello There” (Fig 3.19)

Figure 3.19 Configuring an inject node to send on a websocket

Configure the websocket nodes to connect to wss://echo.websocket.org as shown in Fig 3.20.

Figure 3.20 Configuring the websocket to send to a public echo server. Do the same for the websocket out node.

Deploy. When you click on the inject node you will see the message printed out as shown in Fig 3.21

 

Figure 3.21 Output from a websocket listening for incoming data

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-7_websockets.json

Example 3.8 Sending TCP requests.

This example shows you how to send TCP requests using the tcp node. In this case you will make an HTTP request following the specifications in (http://tools.ietf.org/html/rfc2616#section-5.1.2).

This example shows the use of the tcp node. It could equally be configured with the udp or http nodes in a similar manner.

To get started, let’s connect an inject, function, tcp request, and debug nodes as shown in Fig 3.22.

Figure 3.22 building a TCP request and sending on a tcp output node

Edit the first function node to add a function that sets the string “GET / HTTP/1.1\r\n\r\nHost: www.google.com” as payload as shown in Fig 3.23.

This string is a standard HTTP request, indicating it is a GET request, the protocol is HTTP 1.1 and the host is www.google.com. The \r\n\r\n is two return/newline pairs which is required in the HTTP protocol.

Figure 3.23 Building a TCP request in a function node

Configure the tcp request node to connect to the www.google.com server, on port 80. Configure it to close the connection after 1 second (1000 ms) as shown in Fig 3.24.

Figure 3.24 Configuring the end-point for a TCP request

The tcp request node response is a buffer and needs to be parsed. Configure the second function node to parse the tcp request node response as shown in Fig 3.25

Figure 3.25. A function node to parse a response buffer into a string

If you deploy the flow and click on inject, you will make a request to Google and will get a TCP response. The debug node will print the response as a string as shown in Fig 3.26.

Figure 3.26 Printing out the response to a well formed HTTP request sent over a TCP connection.

Some of you may be wondering why you need to use a function node to build the HTTP request that we sent over TCP. Why not just use the inject node to input the string? The reason is that the inject node ‘escapes’ the string it uses, causing the return/newline you inserted to be removed. This in turn confuses the receiving server (Google) into not returning a response as it waits for the missing return/newlines. So instead, you build the string in a function node. This is one of those ‘gotchas’ that trips up even experienced Node-RED programmers, so always read the info pane for nodes to make sure you understand any limitations or constraints.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson3/3-8_tcp.json

Summary

In this lecture you have seen a series of small examples that use a number of the processing and communication nodes that are available in Node-RED. As you have seen, it is both easy and fast to wire together basic flows that take real-world input, carry out some processing, such as simple data analysis, and return results.

In these examples, you have done little or no coding, but still have been able to build quite complex programs – this is the power of Node-RED.

The next lecture is a quick summary of the basic set of nodes that are available in both vanilla Node-RED as well as the extended set that the FRED service provides. You can read through the lecture to understand the default capabilities, or you can treat the lecture as a reference and use it to look up examples of each node as used in this lecture series.

 

 


About Sense Tecnic: Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures, and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

 

Node-RED: Lecture 2 – Building your first flows

This lecture will introduce you to the Node-RED visual tool and get you started building your first flow. You’ll learn how to create simple flows, use debug nodes to track messages through the flows and how to use the function node to write simple JavaScript code that tailors the node to your specific needs.

This lecture will introduce you to the Node-RED visual tool and get you started building your first flow. You’ll learn how to create simple flows, use debug nodes to track messages through the flows and how to use the function node to write simple JavaScript code that tailors the node to your specific needs.

To get you started, you’ll be using a free cloud service that provides you with a preconfigured Node-RED setup and a wide range of built-in nodes. Later lectures will explain how to set-up Node-RED for yourself on devices such as a Raspberry PI.

At the end of this lecture you’ll be able to create your own simple flows, you’ll know your way around the Node-RED UI and you’ll be familiar with some of the basic built-in nodes that Node-RED offers.

FRED – Front end for Node-RED

FRED is a cloud service that we’ve created that allows you to sign up and use Node-RED without the hassle of downloading and installing on a device such as a PI. It makes it easy to get started with Node-RED by managing an instance of Node-RED for you, keeping the instance up to date with the latest code releases and applying bug fixes or patches. Also included is a set of extra Nodes that are contributed by the community, which add to the standard basic set you get when you install Node-RED yourself. However, FRED is just a wrapper for Node-RED, so anything you write in FRED will work on a Node-RED instance running on a Pi or a Beagleboard.

To begin our examples, make sure you have created your own Node-RED instance in the cloud by signing up for a free account at http://fred.sensetecnic.com.

After registering, make sure to activate your account via your email. You will not be able to log in until you validate your account.

The Node-RED UI

Once you have logged in to your FRED account, you’ll see the standard Node-RED UI which consists of three main panes, as shown in Fig 2.1

Screen Shot 2015-10-22 at 4.53.21 PM.png

Figure 2.1 The Node-RED UI  – showing the node palette (left), flow canvas (centre) and output pane (right)

The main pane is the flow creation workspace in the middle. This is where you drag and drop nodes and connect them with wires. Along the top of the workspace pane is a set of tabs. Each tab opens a previously created workspace and shows any flows created using that workspace.

On the left is the node pane that contains all the built-in nodes that your instance of Node-RED supports. In the example above, you are seeing the FRED node set which contains a large selection that has been added to the basic set that comes with Node-RED. In later lectures you will learn about these nodes and use them to develop new flows. As you can see, nodes are grouped into categories. Opening up a category shows the individual nodes.

On the right-hand side is the output pane that can be toggled between the info and debug tabs. When info is selected, documentation for the selected node is shown there.  When debug is selected, it will display the output of debug nodes, errors and warnings.

Above these three main panes is the usual toolbar, and on the right-hand side are three widgets, a deploy button, a user info icon and a pulldown menu for admin and control. You’ll look in more detail at the pulldown later in these lectures. The user info icon allows you to return to the FRED front page with links to tutorials, your account information, status and other information as well as to log out of the FRED service.

The Deploy button is used when a flow has been constructed and causes the flow to be deployed onto the Node-RED system and executed. You’ll be introduced to the details of what’s actually happening under the covers in lecture 5 and beyond. For now just treat the Deploy button as the way to get your flow running.

A quick tour of Node-RED nodes and messages

As you saw in lecture 1, Node-RED allows you to wire together nodes to create flows which carry out your programming task. Messages pass between nodes, moving from input nodes through processing nodes to output nodes. Let’s take a brief look at nodes, flows and messages.

There are three main  types of nodes:

  1. Input Nodes (e.g. inject)
  2. Output Nodes (e.g. debug)
  3. Processing Nodes (e.g. function)

basic-fred-intro-02

Figure 2.2 The main types of nodes: input, output and processing

Input nodes allow you to input data into a Node-RED application or “flow”. They have at least one output endpoint represented by the small grey square only on their right side. You use input nodes to connect data from other services, for example the  Twitter, Google, serial, websockets or tcp nodes, or to manually input data into a flow using the inject node.

Output nodes allow you to send data outside of a Node-RED flow. They have a single input endpoint on their left side. You use output nodes to send data to other services, for example via Twitter, tcp, serial or email nodes, or to use the debug node to output to the debug pane.

Processing nodes allow you to process data. They have an input endpoint and one or more output endpoints. They allow you to transform the data type (e.g. json, csv, xml) nodes, use the data to trigger a message (e.g. trigger, delay) nodes and to write custom code that uses the data received (e.g. function node).

Note that some nodes, like the inject and debug messages, have a button that allows you to actuate a node (in the case of the inject node) or to enable and disable a node (in the case of the debug node).

Flows consist of multiple nodes wired together, with output tabs linked to input tabs of the next node in the flow. Messages flow along the nodes carrying data from node to node.

Node-RED nodes consume input messages and produce output messages. Messages are JavaScript objects that contain at least a “payload” parameter, like this:

Listing 2.1 A basic Node-RED message structure

  1. msg = {
  2.   payload:”message payload”
  3. };

Nodes consume and produce messages, usually using msg.payload as the main placeholder for the data they consume and produce. However, messages can be extended to contain other parameters. For example, to set the topic of a message and add a new parameter, location, you could create a new msg object as shown in listing 2.2.

Listing 2.2 A more complex Node-RED message structure

  1. msg = {
  2.   payload:”message payload”,
  3.   topic:”error”,
  4.   location:”somewhere in space and time”
  5. };

Let’s use this knowledge to create your first flow.

Example 2.1 Building your first flow: Hello World

Let’s go ahead and start building your first flow so that you can see how simple it is to use the Node-RED UI to build and deploy a flow.

Let’s start with the easiest flow possible, a node to inject some information into the flow, wired to a debug node to see the output from the flow as a debug message. Once you have that running, you’ll build it up to the full Hello World flow.

Since this is the first time you’ll be shown how to actually build a flow, let’s start slowly and explain each step with a screen-shot. Once you’ve created your first flow and seen how easy it is, you’ll mostly be shown a single picture of the final flow rather than all the screenshots.

Let’s start with the simplest Node, the comment Node. You’ll find this in the function section of the node palette. Drag and drop a comment node onto the flow workspace as shown in Fig 2.3.

 

Screen Shot 2015-07-31 at 2.03.20 PM.png

 

Figure 2.3: Using a comment node is a great way to add visible comments into flows

 

Once you’ve done that, take a look at the info pane on the right (remember to switch to info if the debug tab is selected). You’ll see a little info on the node, including the node name, a unique ID, and a properties field with a description of the node. For the comment node there’s not much to say; however, more sophisticated nodes have a lot more info.

Double-click on the comment node, and you’ll see a configuration window (Fig 2.4) You can give the comment node a name and add detailed text if you like.

 

Screen Shot 2015-07-31 at 12.16.33 PM.png

Figure 2.4 Give the comment a name and add any info you want in the text box

 

Ok, now let’s add the first node that actually does something, an inject node. The inject node is used to generate input into a flow and is one of the first nodes in the node palette under input. If you drag and drop an inject node onto the flow workspace, and then look at the info tab, you’ll see the documentation for the inject node. Note that the name of the node on the workspace changes from inject to timestamp, because the default behaviour for the node is to inject a timestamp – the current time in milliseconds since January 1, 1970.

 

Screen Shot 2015-07-31 at 12.03.59 PM.png

Fig 2.5 The inject node allows you to insert events as messages, defaulting to a timestamp

 

You’ll also notice that the inject node (now named timestamp) has a blue dot top right and a grey square centre right. The blue dot indicates that the node hasn’t been deployed since it was last changed; the grey square is the output point for the node. This is where you attach ‘wires’ that route output message from the inject node to the next node in the flow.

To get a sense for the inject node and the whole flow deployment process, let’s add a debug node to see what happens, wire them together and then deploy the flow and test it.

Start by dragging a debug node from the node palette to the workspace. Again you can look at the info for the node.

Then you’ll wire the two nodes together. To do that, click on the grey output point for the inject node and, holding the mouse button down, drag towards the debug node. An orange wire appears, which you can then attach to the grey input point on the debug node.

Screen Shot 2015-07-31 at 12.40.55 PM.png

Figure 2.6 Wiring an inject node to a debug node

 

This is the simplest flow possible and will send the current timestamp to the debug node for display in the debug pane. Let’s try it!

Click the deploy button in the Node-RED window (top right). You’ll see a pop-up saying the flow has been successfully deployed. You will also notice that the blue dots on the nodes disappear, indicating there are no undeployed changes.

Now, before you try the flow, make sure the the debug tab is selected on the right pane. Then click on the left tab on the inject node and look at what appears in the debug pane.

 

Screen Shot 2015-07-31 at 12.48.44 PM.png

Figure 2.7 Debug output from your first flow – the time when you clicked the inject node.

 

As you can see, the inject node, when clicked, generates a timestamp (number of milliseconds since January 1, 1970), which is converted to a message and sent along the output wire, which is delivered to the debug node as an input message. The debug node’s default behaviour is to show any message it receives, which it does in the debug pane on the right.

Congratulations, you created and deployed your first flow!

Let’s now augment it a little to see what else we can do with this simple flow. Firstly, we’ll edit the inject node to deliver a text message rather than a timestamp. To do that, select the inject node in the flow and double-click it. You’ll see a configuration window like the one in Fig 2.8.

 

Screen Shot 2015-07-31 at 12.54.29 PM.png

Figure 2.8 Editing an inject node to send text instead of a timestamp

 

In the payload field, select string instead of timestamp and then type any string you like into the blank field below the payload field. As is traditional, let’s start with “Hello World – from my first NR flow!”

Once you’ve made the change, click ok to save the changes and take a look at the flow. You’ll see the blue dot has appeared on the inject node (which has also returned to being called inject rather than timestamp) to indicate that you have undeployed changes. Click the deploy button again to resolve that and then go ahead and click the tab on the inject node. If you look at the debug output you’ll see that instead of a timestamp, your text has been delivered as a message to the debug node, which displays it as usual.

Screen Shot 2015-07-31 at 1.01.40 PM.png

Figure 2.9 Sending a text message rather than a timestamp

 

As you can see, it’s very easy to wire up some simple nodes and get data to pass through your flow as messages. At this stage, you’ll probably be wondering a little about the messages that are flowing between nodes. The debug node can be used to examine them in more detail.

All messages in Node-RED have three default properties: the payload, which we are seeing above, a message topic, which is a user-defined string describing what the message is about (its topic, if you will) and an internal identifier. You can actually see this information if you edit the debug node configuration. Let’s do that and look at the output (Fig 2.10)

 

Screen Shot 2015-08-06 at 11.15.08 AM.png

 

Figure 2.10: setting the debug node to show the internals of a message

 

Select the debug node, double click and change the node output field to “Complete msg object”. If you then save this, deploy again and click the inject node, you will see in the debug pane a JSON structure that contains 3 fields: a “topic” which is currently blank, a “payload” which contains the string you set in the inject node and an internal ID field “_msgid”. Generally the internal message ID field isn’t used. However, both the topic and payload are used extensively when developing flows. You’ll see more of that later in the lecture series.

These fields are extensible, so you can define new message properties, for example, msg.location, which could be used to add the latitude and longitude values for the source of the message. Let’s take a look at that in the next example flow.

You can find the node-red description of this flow at:

https://github.com/SenseTecnic/nrguideflows/blob/master/lesson2/2-1-firstflow.json

Example 2.2 A second flow: weather alerts

In this example, which is similar to the one introduced in lecture 1, you’ll monitor the weather in your hometown and send yourself a tweet when the weather looks like it’s going to be good. You’ll use a weather node – openweathermap – that retrieves the weather from openweathermap.org for the location you set it for. A simple function node will be used to check for ‘clear weather’ and a Twitter node will be used to send yourself a tweet when the weather looks good.

If you are using FRED, the openweathermap node must be installed using the Add or Remove Nodes navigation tab on FRED control panel left side of the Node-RED palette.  Click on this, and then look for the “openweathermap” node.  If you are running Node-RED on a Pi or desktop computer, nodes are managed through the Node-RED palette manager available in the Node-RED menu in the top right. Select Manage palette, then Install. From there you can search for “openweathermap” and click the Install button.

First you will need to get an API key at OpenWeatherMap. OpenWeatherMap offers a great service that provides detailed weather information for worldwide locations. Visit http://openweathermap.org/appid and follow the instructions as shown in the screenshots below.

 

 

You will need to sign up for an OpenWeatherAccount as shown below:

After signing up you will be redirected to your Home page where you will be able to access or re-generate your API Key. It should look something like this:

Now, let’s use that API key to build a weather alerts flow. Drag and drop a weather node from the left pane onto the workspace, as shown in Figure 2.11.

 

Figure 2.11. Drag and drop a weather node onto a blank workspace

 

If you select the info pane on the right, you can see a description of the openweathermap node with detailed information on how to configure and use it. Some interesting things to note:

  • It has a full JSON structure as its msg.payload, with quite a lot of weather detail, all presented as name:value pairs, e.g. wind speed and tempc
  • The node defines 3 new message properties, msg.location, msg.time and msg.data. As mentioned above, you are free to add properties to messages, and the openweathermap node has added these new properties to carry extra information associated with the message.

Let’s configure the node and look at the actual data structure it generates after it has queried your local weather. Start by double-clicking on the node and fill out the form with your location. Type your city and country in the fields. Add the API key you obtained from http://openweathermap.org/appid and click on “Ok” as shown in Fig. 2.12.

 

 

Figure 2.12. Set your city and country in the configuration form, use the API key you obtained in the step above.

 

Then drag and drop a debug node and wire it to the openweathermap node. Click on “Deploy” to see that the payload object from the openweathermap node in the debug pane.

 

Listing 2.3 The message payload for the openweathermap node is a JSON structure describing weather conditions, temperature, wind, cloud cover and sunrise times.

  1. {
  2.   “weather”: “Clear”,
  3.   “detail”: “sky is clear”,
  4.   “tempk”: 295.104,
  5.   “tempc”: 21.903999999999996,
  6.   “humidity”: 53,
  7.   “maxtemp”: 295.104,
  8.   “mintemp”: 295.104,
  9.   “windspeed”: 2.22,
  10.   “winddirection”: 273.007,
  11.   “location”: “Vancouver”,
  12.   “sunrise”: 1432038196,
  13.   “sunset”: 1432094081,
  14.   “clouds”: 8,
  15.   “description”: “Vancouver weather(49.25,-123.12) is Clear (sky is clear).”
  16. }

As you can see, the node delivers quite a lot of information about your local weather, all as the usual name:value pairs. For this example, you want to use the “weather” field which you’re going to test to see if it’s reported as “Clear”. If it’s “Clear” you’ll then send a tweet.

To program that logic, you’ll use a function node. You saw one of these in lecture 1, but without the details. Let’s see how they are used by dragging one onto the flow workspace and double clicking it to open up the node editor.

Figure 2.13. Add a function node and wire it to the openweathermap node.

Now double-click on the function node and type/copy this (Note if you use copy/paste – make sure you paste as plain text, wordpress sometimes inserts odd characters)

 

Listing 2.4 Function node “If Clear Weather”

  1. if (msg.payload.weather === “Clear”) {
  2.     msg.payload = “Clear skies ahead today!”
  3.     return msg;
  4. }
  5. return null;

Figure 2.14. Edit the function node adding the JavaScript shown in listing 2.4

Looking at Listing 2.4, you can see that this will parse the incoming message payload for the weather parameter and compare it to the string “Clear” (line 1). If it is equal, it will rewrite the message payload with your own string “Clear skies ahead today!” (line 2). Otherwise it will return a null message (line 5). This last bit is important because in Node-RED nodes ignore null messages.

You can do all sorts of things now, for example wire this message to an email node, or a pushbullet node. For this tutorial we will use the Twitter output node. Drag  a Twitter node onto the workspace, double-click and and fill out your Twitter account credentials as shown in Fig 2.15.

Figure 2.15. Set your Twitter account credentials into the node configuration

Once you’ve set up these credentials, Node-RED stores them and you can reuse them when you create another Twitter node in future flows.

Once you’ve wired up the flow, you can hit the deploy button and then watch your Twitter account to see the new tweet every time the weather is reported as clear.

Click on “Deploy”:

Figure 2.16. The full weather flow sending tweets if the weather report mentions clear skies.

Other useful UI features

One of the most useful features of the Node-RED UI is the ability to copy and paste nodes and even full flows using the standard cut’n’paste clipboard. Node-RED flows are simply coded as JSON strings and can be exported from a workspace, and imported into a workspace using the Node-RED pulldown menu in the top right of the Node-RED window.

Rather than building the flows in this example, or in fact anywhere in this lecture series, by hand yourself, you can simply copy them from our website and paste them into a workspace. After that all you need to do is configure them correctly, e.g. with credentials, and deploy them.

Let’s quickly show you how to do that.

Click on the link for this example which is at the end of the example below and look for the JSON code. Copy it to the clipboard using the usual CTL C or copy item in the edit menu.

Using the pulldown menu in the top right of the Node-RED window, select Import->Clipboard as shown in Fig 2.17

Screen Shot 2015-08-15 at 12.55.58 PM.png

Figure 2.17 Selecting the Import from Clipboard menu item in Node-RED

 

You will see a popup with an input field (Fig 2.18). Paste your clipboard into the input window and click OK.

 

Screen Shot 2015-08-15 at 12.56.27 PM.png

Figure 2.18 Pasting a flow into the import window of Node-RED

 

The new flow will appear in the current workspace and can be moved to a location and dropped. Once it’s in place, you can configure it as you would a flow you created yourself and deploy it.

 

You can find the node-red description of this flow at:

https://github.com/SenseTecnic/nrguideflows/blob/master/lesson2/2-2_weatheralerts.json

Summary

In this lecture you have created your first flows and seen how to wire together a set of basic nodes to achieve quite complex tasks. We’ve skipped over a few of the underlying details in order to get you going quickly, and we’ll return to those in lectures 3 and 4. However, so far you’ve seen how to use the Node-RED visual flow builder, the basic classes of input, output and processing nodes. You were given a brief overview of messages and how to use the function node to write your own JavaScript code to do simple processing on messages within a flow. In the next lecture, you’ll take a more in-depth look at the programming model of Node-RED and get a better understanding of the main programming elements and nodes and how to craft more complex flows using a variety of nodes.

 


About Sense Tecnic: Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures, and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

 

Node-RED: Lecture 1 – A brief introduction to Node-RED

This lecture will introduce you to some examples of Node-RED, both to give you a sense of its capabilities and to introduce its basic concepts. We’ll also discuss the history of Node-RED and its origins to give you an idea of what Node-RED is good for and of what types of tasks are best left to a more traditional programming language and environments.

Node-RED is a powerful tool for building Internet of Things (IoT) applications with a focus on simplifying the ‘wiring together’ of code blocks to carry out tasks. It uses a visual programming approach that allows developers to connect predefined code blocks, known as ‘nodes’, together to perform a task. The connected nodes, usually a combination of input nodes, processing nodes and output nodes, when wired together, make up a ‘flows’.

Originally developed as an open source project at IBM in late 2013, to meet their need to quickly connect hardware and devices to web services and other software – as a sort of glue for the IoT – it has quickly evolved to be a general purpose IoT programming tool. Importantly, Node-RED has rapidly developed a significant and growing user base and an active developer community who are contributing new nodes that allow programmers to reuse Node-RED code for a wide variety of tasks.

Although Node-RED was originally designed to work with the Internet of Things, i.e. devices that interact and control the real world, as it has evolved, it has become useful for a range of applications.

This lecture will introduce you to some examples of Node-RED, both to give you a sense of its capabilities and to introduce its basic concepts. We’ll also discuss the history of Node-RED and its origins to give you an idea of what Node-RED is good for and of what types of tasks are best left to a more traditional programming language and environments.

By the end of this lecture you should have a high-level understanding of Node-RED, its programming model and some simple pre-built nodes as well as an insight into the origins and best usage of Node-RED.

Getting to know Node-RED

Let’s get started by looking at a few simple examples of the kinds of thing you can do with Node-RED. Our goal in this lecture is not to get you programming with Node-RED, but to give you a sense of its capabilities through some high level examples. If you really want to dive straight in, you can skip ahead to lecture 2 where you’ll be taken through the steps to create and run your first flow.

This first example flow will receive a message from a Twitter feed containing a hashtag and turn on and off a LED when a new tweet has arrived. For this example let’s assume that NODE-RED is running on a Raspberry PI – a common situation for many Node-RED users.

Example 1.1 Using Twitter to control a Raspberry PI

The Node-RED flow to check Twitter and to see if a hashtag #led is detected, turns on a LED connected to a Pi is shown in Fig. 1.1. As you can see, it’s quite simple; it consists of 3 nodes wired together – a twitter node on the left, a trigger node in the middle and a Raspberry Pi (gpionode) node on the right. (gpio stands for General Purpose Input/Output and is the name given to the generic Raspberry PI node that handles IO). The nodes, represented by rounded boxes, usually have an icon to represent their type, e.g. the Twitter bird icon in the first node, and a text name that can be set by the programmer.

Each of these three nodes is built into the Node-RED visual programming tool and can be dragged from a node palette onto the main workspace. They are then ‘wired’ together by joining their output tabs to the input tab of the next node in the flow (the little grey blobs on the right or left of the node boxes.) The next lecture will look at the UI in detail, but for now let’s focus on the high-level capabilities.

 

Figure 1.1 A simple 3 node flow that uses a Twitter tag to trigger a LED on a Raspberry PI.

The Twitter node is a built-in node in Node-RED and hides all of the complexity of using the Twitter API. It can be configured with a user’s account credentials and a variety of search strings, in our case it’s simply looking for hashtag ‘#led’.

When the Twitter node sees the tag either in a tweet to the user, or the public tweet feed, it creates a new message with the details of the tweet, which is forwards to the next node in the flow. In our example, since the Twitter node’s output is wired to the trigger node’s input, the trigger node receives a message with the details of the tweet.

The trigger node is another built-in node in Node-RED and its default behaviour is to wait for any message on its input. When it receives a message, it ‘triggers’, and sends a message on its output with the value “1” as the message body. It then waits 1 second and sends a second message with the value “0” in the message body.

Since the trigger node is connected to the gpionode, which controls the Input/Output or IO pins on the Raspberry PI, it get’s these two messages, 1 second apart, on its input.

Again, the gpionode is a built-in node in Node-RED that hides the details of setting the IO pins on the PI. When it gets a message with a value of “1” in the body, it takes the IO pin high, and when it receives a message with body of “0”, it takes the pin low. In this example, the gpionode is configured to control pin number 12, and so the node shows that in the node icon.

If you have your PI wired up with an LED connected to Pin 12, the gpionode going high for 1 second and then low will cause the LED to blink on for 1 second.

If you think about what’s happening here, it’s actually an incredibly powerful example of Node-RED and the Internet of Things (IoT). It’s connecting the ‘Twitterverse’ to a PI and an LED, but requires no programming – all the tasks are carried out by pre-built nodes in Node-RED which just need configuring. This really demonstrates the power of Node-RED when running on a simple device such as a Raspberry PI.

So just to recap. Node-RED is a flow-based programming tool with a visual editor that allows you to wire together nodes to create flows. In this case, 3 nodes create our first flow. Real-world events, e.g. a tweet with the #led hashtag, are converted to messages, which ‘flow’ along the wires between the nodes. Nodes process messages on their input and, as a result, send messages on their outputs to the next node in the flow. In this simple example, we look for a tweet with hashtag #led and use it to trigger a message to a Raspberry PI node that cause IO pin 12 to go high for 1 second, which in turns causes a LED to blink on for 1 second.

This example – and the rest in this lecture – are mostly to show you the kinds of things we can do with Node-RED. If you want to play around with the example, you should look at lecture 2 to see how to setup an account with FRED (the cloud based Node-RED service we use) and how to get started. You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson1/1-1_twitter.json

Example 1.2 Using Node-RED to alert you when you’re falling behind on your exercise schedule

Our second example shows a different usage of Node-RED. Rather than controlling a device such as a Raspberry PI, this flow combines weather info and your fitbit usage to send you an email alert when the weather is good and you’re behind on your exercise schedule. The flow is a little more complicated, using 6 nodes (see Fig 1.2), but again, by using built-in nodes you can accomplish complex tasks with little or no programming.

 

 

 

Figure 1.2 A simple flow to alert you when you are not exercising and the weather is good

 

As before, nodes are dragged from the node palette in the Node-RED UI and wired together on the flow canvas. Starting at the leftmost node, let’s begin with an openweather node which you can configure to check the weather for any major city at set intervals. When it checks, it packages up the results in a message which is forwarded to the next node, in this case, to a switch node. The switch node performs a simple test, checking the input message for a value and sending a message with a value of “1” if the test is true. In our case, it’s been configured to test the temperature value (tempc) to see if it’s 15 degrees celsius or higher.

When the switch node tests to true, or ‘switches on’, it sends a message onto the fitbit node which is the next node in the flow. The fitbit node is another powerful built-in node that you can configure to get statistics on your fitbit device from the fitbit.com portal. The fitbit node packages the statistics for your device into a message and forwards it to the next node in the flow.

The next node is another switch node. This one has been configured to check if the number of steps you’ve walked today is less than 1000. It does this by testing to see if the fitbit data field, summary.steps, is less than 1000. Remember, it just got these statistics in a message from the fitbit node, which in turn got them by making a call to the fitbit API on fitbit.com.

If you have walked less than 1000 steps today, then the switch node generates a message that is passed on through the flow to a function node. The function node is a generic all-purpose node that allows you to write your own programming logic in Javascript. This node will be discussed in more detail in the next lecture. For now you can assume that this node just creates a new message with an email subject line and a text body that can be sent by email.

Once this message is created within the function node, it is sent through the flow to the final node which is an email node. This node will take the body of any incoming message and send it to the email account configured when the node is initialized. In this case, you’d set it up to send a message to yourself.

So again, a very simple flow that uses the power of Node-RED’s built-in nodes to check your local weather. And if it’s reasonably warm and you haven’t done much walking today, it sends you an email reminder.

Although the example seems quite simple, just checking the weather and your fitbit activity, you can use similar flows to check the status of home devices, of computers in your server farm, of devices in a factory, etc., and you can then use output nodes to send mail, tweets, make API calls to back-office software, control real world devices – in fact the choices are endless. Node-RED is an extremely powerful tool for wiring together inputs and outputs and offers many nodes, either built in or developed by the Node-RED community, to carry out an amazing range of tasks.

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson1/1-2_weatheralert.json

Example 1.3 Building a simple web service using Node-RED’s built-in HTTP nodes

Our final example in this high-level introduction is yet another class of flows. It shows how to create a simple web service that responds to a browser sending a HTTP request by querying some real-world data, and then delivers a web page with a graphical representation of that data back to the browser – essentially Node-RED is being used to provide a web server and simple service all with a few nodes.

Sticking with the fitbit node introduced in the previous example, and this example creates a simple web service that allows you to query your fitbit stats for the day and, using a nice donut chart, display how many calories you’ve ‘burned off’ through exercise.

The flow is shown in Fig 1.3 and consists of 4 nodes. The first and final nodes are http input and output nodes that work together to listen for HTTP requests and send HTTP responses. This flow listens for a HTTP request from any source, but let’s assume a regular browser here. When it arrives, it queries your fitbit stats from fitbit.com, then uses a template node to build a HTTP page, and passes that to the http output node which sends back the webpage to the browser.

Figure 1.3 A simple web server example to graph fitbit data

 

Again, this is just to give you a sense of the power and flexibility of Node-RED without going into all the details of how this flow works. You’ll get to know more about that in later lectures. At a high level, the http input node has been configured to listen for HTTP requests on a URL, which is a combination of the hosting service that’s used in these lectures plus your login name {username} and the /fitbit string. This will be explained in more detail in lecture two.

When a HTTP request comes in, the http input node creates a message to trigger the fitbit node, which is the next node in the flow. The fitbit node gets the current statistics for the user, and then passes those statistics, as a message, to a template node. The html template node is another pre-built node in Node-RED, which, like the function node, allows you to build arbitrary code. However, rather than using JavaScript like the function node, the template node works with text such as HTML.

The HTML code in the template node is shown in Listing 1.1, so you can see what’s happening. However, there’s no need to examine it in detail if you don’t want to. Basically, the template node takes the calorie consumption data from fitbit and, using a 3rd party graphing library, creates a simple HTML file that will tell the receiving browser to use the 3rd party graphing library to display a donut chart of calorie consumption.

 

Listing 1.1 A simple HTML template to display a donut chart of calorie usage

 

  1. <!doctype html>
  2. <head>
  3.     <title>A Node RED Example</title>
  4.     <link rel=”stylesheet” href=”//cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.css”>
  5.     <script src=”//cdnjs.cloudflare.com/ajax/libs/raphael/2.1.0/raphael-min.js”></script>
  6.     <script src=”//ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js”></script>
  7.     <script src=”//cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.min.js”></script>
  8. </head>
  9. <html>
  10.     <div id=”chart-example” style=”height: 250px;”></div>
  11.     <script>
  12.         Morris.Donut({
  13.            element: ‘chart-example’,
  14.            data: [
  15.              {label: “Activity Calories”, value: {{payload.summary.activityCalories}} },
  16.              {label: “Calories BMR”, value: {{payload.summary.caloriesBMR}} },
  17.              {label: “Calories Out”, value: {{payload.summary.caloriesOut}} }
  18.           ]
  19.        });
  20.     </script>
  21. </html>

 

Once the template node has generated the HTML file, it passes it as a message to the final node, which is a http response node. This node packages up the HTML as a HTTP response which it send back to the browser.

The user will then see a simple donut chart of their calorie usage for the day (see Fig 1.4)  – all built and served by a simple Node-RED flow!

 

 

Figure 1.4 A donut chart, served up by Node-RED showing calorie counts from fitbit

 

You can find the node-red description of this flow at:

https://raw.githubusercontent.com/SenseTecnic/nrguideflows/master/lesson1/1-3_webservice.json

The History of Node-RED

As the examples in the previous section have shown, Node-RED is a powerful tool for building IoT applications and services. Its genesis was triggered exactly by this need to rapidly prototype IoT applications and Node-RED was created as an open source project by the IBM Emerging Technologies group, in particular by two researchers, Nick O’Leary and Dave Conway-Jones. They created Node-RED initially as a tool for themselves as they were working on IoT projects and were “looking for a way to simplify the process of hooking together systems and sensors when building proof-of-concept technologies for customers.”

An initial version of Node-RED was released as an open source project in late 2013 and built up a small but active user and developer group during 2014. At the time of writing, Node-RED is still an emerging technology, but has already seen significant adoption by makers, experimentalists and a number of large and small companies experimenting with using it for their needs.

Today there is a vibrant user and developer community, with a core group working on the Node-RED code itself and most developers contributing nodes or flows to the flow library. You can check out the mailing list and github repositories using the links provided in Appendix A or just google them.

Since Node-RED is still a rapidly evolving technology, be aware that things can change quickly. These lectures were written against the 0.11.0 release of Node-RED and the examples will be updated as Node-RED evolves. However, it’s always wise, as with any new technology, to check compatibility if things aren’t working the way you expect.

Node-RED and the IoT

When the IBM folks created Node-RED, they were mostly focused on the Internet of Things, i.e connecting devices to processing and processing to devices. As a tool for rapid application development for the IoT, Node-RED is both powerful and flexible. Its power comes from a combination of two factors:

  • Node-RED is an example of a flow-based programming model – messages representing events flow between nodes, triggering processing that results in output. The flow-based programming model maps well to typical IoT applications which are characterised by real-world events that trigger some sort of processing which in turn results in real-world actions. Node-RED packages these events as messages which offer a simple and uniform model for events as they flow between nodes that make up the flows.
  • The set of built-in nodes is the second part of the Node-RED story. By building a set of powerful input and output nodes, each of which hides much of the complexity of interacting with the real world, Node-RED offers developers powerful building blocks to allow them to quickly put together flows that accomplish a lot, without having to worry about the programming details.

 

These two factors make Node-RED a powerful tool for IoT application development. When combined with the flexibility to create and use nodes such as the function node, which allows the developer to quickly write arbitrary JavaScript, plus the Node-RED community, who are constantly creating and sharing new nodes, Node-RED is likely to be one of the main tools in the IoT developers toolbox.

However, Node-RED isn’t always the best tool for IoT application development. While powerful and flexible, it isn’t always the right solution. Some situations where Node-RED may not be the first choice include:

  • Complex multi-function IoT applications. Node-RED excels at rapid application development and at acting as the glue for connecting events to actions, or sensors to actuators, if you like. However, when an application gets above a certain size, it becomes complex to visually program and manage through Node-RED. There are some features to help with this, e.g. sub-flows (see lecture 5), but eventually the UI becomes a bottleneck.
  • Flow-based programming is a programming model that is not always the most suitable for application development. In much the same way that certain programming languages excel at some tasks but not others, flow-based programming has its weaknesses. A good example are loops: Node-RED is cumbersome when handling loops.
  • Specific use cases. Flow-based programming is a general purpose model and not targeted or optimized for specific needs, for example Data Analytics or User Interface development. At the moment Node-RED has no specific support for these type of applications and no easy way to add such support. Obviously, because the underlying technology of Node-RED is JavaScript, you can leverage its capabilities for these needs. If want to use Node-RED to prototype something, one approach is to implement parts or all of your application in another language more suited to your task and use Node-RED for the overall control.

 

Having said that, as has been shown in this lecture and will be demonstrated in the following lectures,  Node-RED is a powerful tool for a large number of IoT applications. As it evolves it will become both more sophisticated and more usable in a wider range of circumstances.

Perhaps just as important, as you explore Node-RED in subsequent lectures, you will realise that although Node-RED’s roots are in the IoT, it is a tool that can be used to build a wide variety of applications, not just IoT applications. In fact, in these lectures you will see example of Node-RED being used for web apps, social media apps, back-office integration, IT task management, i.e. across the range of modern computing tasks.

Our hope is that by the end of this lecture course, you will see Node-RED, as we do, as a flexible and powerful tool that can be used in many situations, both for prototyping but also for production level development.

Summary

In this lecture we’ve taken you through a high-level look at Node-RED. You’ve seen how Node-RED wires together building blocks, using a visual tool to rapidly create simple flows that actually carry out sophisticated real-world tasks. You’ve also learnt a little about the origins of Node-RED as a rapid application development tool for the IoT and how, as it has evolved, it’s now being used for a variety of tasks, not just IoT programming. Finally, we’ve discussed the limitations of Node-RED and indicated some areas where other languages and tools may serve you better. In the next lecture, we’ll start on some real programming and show you how to build your own flows using the basic nodes that Node-RED provides.

 


About Sense Tecnic: Sense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures, and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


© Lea, Blackstock, Calderon

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.