Tutorial: Advanced dashboards for Node-RED (and cryptocurrency)

In this tutorial we’ll combine two interesting areas, how to build interactive dashboards in Node-RED with use of the new Binance cryptocurrency node that we’ve developed.

The dashboard we are aiming for is shown below – and you can access a realtime, live DEMO of it if you want to see it in action!


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


Overview of the flow

As you can see, the dashboard we’ve created allows a user to choose a specific cryptocurrency (the coloured tabs along the top) which, when clicked, fetches the realtime data from the binance cryptocurrency exchange node. The latest data, eg bid price, volume etc is shown in the left panel, and a graph of the latest data is also shown. A slider in the bottom left allows you to interactively select the amount of data you want to see and an input field allows you to limit the number of datapoints selected.

The flow for this dashboard is shown below, it’s a little more complex than other flows we’ve shown, so we wont explain all aspects in detail, however we’ll give you enough to understand how it works and of course, you can copy the flow (see bottom) and experiment with it yourself.

There are 3 main components to the flow:

  • Text elements of the demo UI: At top is the part of the flow that handles the user selecting the cryptocurrency they want to chart. These are they 9 UI button nodes top left. When selected these generate the correct ticker pair, eg BTCUSDT, which is displayed using a UI text node, then the ticker pair is passed through a rate limit node and into a function node that transfers the message payload into the message topic (msg.topic) which is then sent through to the Binance getBookTicker node for bid/ask prices or to the Binance getDatStats node for the rest of the data. All data is then sent through to UI text nodes for display. The ticker is also used in the load logo function node to pull an image of the cryptocurrency for display on the UI.
  • Graphing element of demo UI: the next piece of the flow creates the chart. On initial start the initialize input node triggers this part of the flow, otherwise it is triggered when the user selects a cryptocurrency. A message is generated in set values for api function node which the load chart options function node combines with an interval and number of data points (set by the slider and #datapoints input node) to ask the Binance candleStick node for the appropriate data. The build chart data function node uses the data from Binance to then create the appropriate data for the standard node-RED chart ui node.
  • Historical slider element: a simple slider is used to set how much data should be charted (from 5mins through to 1 month) which is used in the load chart options node to ask Binance for the correct amount of data.

A fourth element, a Generic error handler: There is a generic error handler flow at the bottom of the page which catches all errors and generates a UI message indicating there was a problem.

Some critical details

Although the flow may look complicated, it is actually quite simple and mostly uses standard nodes. The main function nodes are:

  • set values for API function node in the upper part of the flow
if (typeof msg.payload === 'string') {
 flow.set("selectedTickerPair", msg.payload);
}
msg.topic = flow.get("selectedTickerPair");
return msg;

Which simply copies over the ticker pair in the msg.payload to the topic which the Binance nodes need to generate the correct query.

  • load chart options function node
msg.topic = flow.get("selectedTickerPair");
msg.payload = {};
msg.payload.interval = flow.get("interval");
msg.payload.limit = flow.get("limit");

return msg;

This sets its own topic to the message topic from the previous node (now set to the cryptocurrency ticker pair symbol) and adds two fields to the message. The limit (max data points) which is set by the #datapoints user input field and simply passes along whatever value the user sets. The interval is set by a ui slider node. Since the slider just generates integer values, we need to convert it to a string, eg 15m, 12h, 3d etc that the Binance node will understand. This is generated in the set values for API function node in the lower part of the flow and is basically a large case switch that converts integers to the appropriate string. A subset is shown below.

var interval = "1d";

switch(msg.payload) {
 case 0:
      interval = "1m";
      break;
 case 1:
      interval = "3m";
      break;
 case 2:
      interval = "5m";
      break; 
 case 3:
      interval = "15m";
      break;
 case 4:
      interval = "1h";
      break;
  • build chart data function node
var series = [msg.topic]
var data = msg.payload.map(function (d) {
 return {
 x: d[0],
 y: d[4] // close price
 }
});

msg.payload = [{
 series: series,
 data: [data]
}]

return msg;

The Binance getCandlestick node returns an array of data (specified by tickerpair, interval and limit) which this small piece of code simply uses a map operation to extract the data it uses, i.e. the d[0] value (time) and the d[4] value (closing price) which are then passed as an array to the UI chart node.

Full flow

You can copy the full flow from the code box below and paste into your own Node-RED instance so you can experiement with the dashboard.

Have fun!

[{"id":"3e6934b2.79480c","type":"ui_text","z":"d7d920d1.1e628","group":"684f06d8.eb11a8","order":2,"width":"0","height":"0","name":"","label":"Bid Price","format":"{{msg.payload.bidPrice}}","layout":"row-spread","x":800,"y":60,"wires":[]},{"id":"283ee802.c99158","type":"ui_chart","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":3,"width":"18","height":"10","label":"","chartType":"line","legend":"true","xformat":"auto","interpolate":"linear","nodata":"Click on a crypto or provide a ticker pair","dot":true,"ymin":"","ymax":"","removeOlder":"30","removeOlderPoints":"","removeOlderUnit":"86400","cutout":0,"useOneColor":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"useOldStyle":false,"x":890,"y":600,"wires":[[],[]]},{"id":"592d6f1e.79145","type":"binance-get-candlesticks","z":"d7d920d1.1e628","name":"","ticker":"","interval":"1d","limit":"30","startTime":"","endTime":"","x":860,"y":480,"wires":[["e918db8d.87e3d8"]]},{"id":"e918db8d.87e3d8","type":"function","z":"d7d920d1.1e628","name":"built chart data","func":"\nvar series = [msg.topic]\nvar data = msg.payload.map(function (d) {\n return {\n x: d[0],\n y: d[4] // close price\n }\n});\n\nmsg.payload = [{\n series: series,\n data: [data]\n}]\n\n\nreturn msg;","outputs":1,"noerr":0,"x":860,"y":540,"wires":[["283ee802.c99158"]]},{"id":"3a5bf45.9251f0c","type":"function","z":"d7d920d1.1e628","name":"set values for API","func":"if (typeof msg.payload === 'string') {\n flow.set(\"selectedTickerPair\", msg.payload);\n}\nmsg.topic = flow.get(\"selectedTickerPair\");\nreturn msg;","outputs":1,"noerr":0,"x":390,"y":200,"wires":[["90a8d0bc.4aeac","304459df.84fe86","c0332b3e.107478","58d47e8a.52b39"]]},{"id":"b1a9caa7.d0e068","type":"ui_button","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":2,"width":"2","height":"1","passthru":false,"label":"Refresh","color":"","bgcolor":"","icon":"refresh","payload":"selectedTickerPair","payloadType":"flow","topic":"","x":100,"y":60,"wires":[["71902fa5.40cf","686b8209.90607c"]]},{"id":"2d31df00.dddce","type":"ui_button","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":6,"width":"2","height":"1","passthru":false,"label":"Ethereum","color":"","bgcolor":"#555555","icon":"","payload":"ETHBTC","payloadType":"str","topic":"","x":100,"y":140,"wires":[["71902fa5.40cf"]]},{"id":"4a25114.13304f","type":"ui_button","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":7,"width":"2","height":"1","passthru":false,"label":"Neo","color":"","bgcolor":"#58be02","icon":"","payload":"NEOBTC","payloadType":"str","topic":"","x":90,"y":180,"wires":[["71902fa5.40cf"]]},{"id":"1a781586.60994a","type":"ui_button","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":4,"width":"2","height":"1","passthru":false,"label":"BTC","color":"","bgcolor":"#ff8e13","icon":"","payload":"BTCUSDT","payloadType":"str","topic":"","x":90,"y":100,"wires":[["71902fa5.40cf"]]},{"id":"7cc83e6.13d54c","type":"ui_button","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":5,"width":"2","height":"1","passthru":false,"label":"Litecoin","color":"","bgcolor":"#bebebe","icon":"","payload":"LTCBTC","payloadType":"str","topic":"","x":100,"y":220,"wires":[["71902fa5.40cf"]]},{"id":"90a8d0bc.4aeac","type":"binance-get-book-ticker","z":"d7d920d1.1e628","name":"","ticker":"","x":620,"y":80,"wires":[["3e6934b2.79480c","30816eb1.acf7b2"]]},{"id":"30816eb1.acf7b2","type":"ui_text","z":"d7d920d1.1e628","group":"684f06d8.eb11a8","order":3,"width":"0","height":"0","name":"","label":"Ask Price","format":"{{msg.payload.askPrice}}","layout":"row-spread","x":800,"y":100,"wires":[]},{"id":"71902fa5.40cf","type":"ui_text_input","z":"d7d920d1.1e628","name":"","label":"Ticker pair","group":"3fa75da0.6d85c2","order":1,"width":0,"height":0,"passthru":true,"mode":"text","delay":300,"topic":"","x":370,"y":120,"wires":[["673df297.a4404c"]]},{"id":"304459df.84fe86","type":"binance-get-day-stats","z":"d7d920d1.1e628","name":"","ticker":"","x":610,"y":180,"wires":[["38c0811d.2ab6fe","319d75a2.00504a","8e238a8f.ec6f48","bf8d0cbf.6f50f"]]},{"id":"2bb366e1.79afaa","type":"catch","z":"d7d920d1.1e628","name":"","scope":null,"x":100,"y":640,"wires":[["6206c86d.bf05b8","15052e06.11e1a2"]]},{"id":"4a8fb8a1.e63068","type":"ui_toast","z":"d7d920d1.1e628","position":"top right","displayTime":"3","highlight":"","outputs":0,"ok":"OK","cancel":"","topic":"","name":"","x":550,"y":640,"wires":[]},{"id":"6206c86d.bf05b8","type":"debug","z":"d7d920d1.1e628","name":"","active":true,"console":false,"complete":"error","x":240,"y":680,"wires":[]},{"id":"15052e06.11e1a2","type":"function","z":"d7d920d1.1e628","name":"","func":"msg.topic = \"API Error\";\nif (msg.error) {\n msg.payload = msg.error.message\n} else {\n msg.payload = \"\"\n}\nreturn msg;","outputs":1,"noerr":0,"x":230,"y":640,"wires":[["22694b27.9c4fd4"]]},{"id":"38c0811d.2ab6fe","type":"ui_text","z":"d7d920d1.1e628","group":"684f06d8.eb11a8","order":4,"width":"0","height":"0","name":"","label":"24 hr Low","format":"{{msg.payload.lowPrice}}","layout":"row-spread","x":800,"y":140,"wires":[]},{"id":"319d75a2.00504a","type":"ui_text","z":"d7d920d1.1e628","group":"684f06d8.eb11a8","order":5,"width":"0","height":"0","name":"","label":"24 hr high","format":"{{msg.payload.highPrice}}","layout":"row-spread","x":800,"y":180,"wires":[]},{"id":"8e238a8f.ec6f48","type":"ui_text","z":"d7d920d1.1e628","group":"684f06d8.eb11a8","order":6,"width":"0","height":"0","name":"","label":"24 hr Price % change","format":"{{msg.payload.priceChangePercent}}%","layout":"row-spread","x":840,"y":220,"wires":[]},{"id":"bf8d0cbf.6f50f","type":"ui_text","z":"d7d920d1.1e628","group":"684f06d8.eb11a8","order":7,"width":"0","height":"0","name":"","label":"24 hr volume","format":"{{msg.payload.volume}}","layout":"row-spread","x":810,"y":260,"wires":[]},{"id":"cf4f08bb.9ceda8","type":"ui_button","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":8,"width":"2","height":"1","passthru":false,"label":"Ripple","color":"","bgcolor":"#0086be","icon":"","payload":"XRPBTC","payloadType":"str","topic":"","x":90,"y":260,"wires":[["71902fa5.40cf"]]},{"id":"6a86f23d.54b9ec","type":"ui_slider","z":"d7d920d1.1e628","name":"","label":"","group":"684f06d8.eb11a8","order":9,"width":0,"height":0,"passthru":true,"topic":"","min":"2","max":"14","step":1,"x":90,"y":580,"wires":[["963cb0a3.0c805"]]},{"id":"11b7370d.f93ff9","type":"function","z":"d7d920d1.1e628","name":"set values for api","func":"\nvar interval = \"1d\";\n\nswitch(msg.payload) {\n case 0:\n interval = \"1m\";\n break;\n case 1:\n interval = \"3m\";\n break;\n case 2:\n interval = \"5m\";\n break; \n case 3:\n interval = \"15m\";\n break;\n case 4:\n interval = \"1h\";\n break;\n case 5:\n interval = \"2h\";\n break;\n case 6:\n interval = \"4h\";\n break;\n case 7:\n interval = \"6h\";\n break;\n case 8:\n interval = \"8h\";\n break;\n case 9:\n interval = \"12h\";\n break;\n case 10:\n interval = \"1d\";\n break;\n case 11:\n interval = \"3d\";\n break;\n case 12:\n interval = \"1w\";\n break;\n case 13:\n interval = \"1M\";\n break;\n}\n\nflow.set(\"interval\", interval);\nmsg.payload = interval;\n\nreturn msg;","outputs":1,"noerr":0,"x":410,"y":580,"wires":[["5b8217e7.65f4f8","c0332b3e.107478"]]},{"id":"963cb0a3.0c805","type":"delay","z":"d7d920d1.1e628","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"3","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":230,"y":580,"wires":[["11b7370d.f93ff9"]]},{"id":"90628d7.2b2047","type":"ui_button","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":9,"width":"2","height":"1","passthru":false,"label":"Monero","color":"","bgcolor":"#ff6600","icon":"","payload":"XMRBTC","payloadType":"str","topic":"","x":100,"y":300,"wires":[["71902fa5.40cf"]]},{"id":"5b8217e7.65f4f8","type":"ui_text","z":"d7d920d1.1e628","group":"684f06d8.eb11a8","order":8,"width":"0","height":"0","name":"","label":"Chart time interval","format":"{{msg.payload}}","layout":"row-spread","x":630,"y":580,"wires":[]},{"id":"8d98a98c.b79718","type":"inject","z":"d7d920d1.1e628","name":"initialize","topic":"10","payload":"10","payloadType":"num","repeat":"","crontab":"","once":true,"x":100,"y":540,"wires":[["6a86f23d.54b9ec"]]},{"id":"9bfab0f0.119d1","type":"ui_numeric","z":"d7d920d1.1e628","name":"","label":"# datapoints","group":"684f06d8.eb11a8","order":10,"width":0,"height":0,"passthru":true,"topic":"","format":"{{value}}","min":"1","max":"500","step":1,"x":110,"y":480,"wires":[["2eaadaeb.ab8fb6"]]},{"id":"79abf723.549248","type":"inject","z":"d7d920d1.1e628","name":"initialize","topic":"30","payload":"30","payloadType":"num","repeat":"","crontab":"","once":true,"x":100,"y":440,"wires":[["9bfab0f0.119d1"]]},{"id":"1895296a.741bc7","type":"function","z":"d7d920d1.1e628","name":"set values for api","func":"flow.set(\"limit\", parseInt(msg.payload));\n\nreturn msg;","outputs":1,"noerr":0,"x":470,"y":480,"wires":[["c0332b3e.107478"]]},{"id":"c0332b3e.107478","type":"function","z":"d7d920d1.1e628","name":"load chart options","func":"msg.topic = flow.get(\"selectedTickerPair\");\nmsg.payload = {};\nmsg.payload.interval = flow.get(\"interval\");\nmsg.payload.limit = flow.get(\"limit\");\n\nreturn msg;","outputs":1,"noerr":0,"x":670,"y":480,"wires":[["592d6f1e.79145"]]},{"id":"2eaadaeb.ab8fb6","type":"delay","z":"d7d920d1.1e628","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"3","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":270,"y":480,"wires":[["1895296a.741bc7"]]},{"id":"686b8209.90607c","type":"ui_toast","z":"d7d920d1.1e628","position":"top right","displayTime":"3","highlight":"","outputs":0,"ok":"OK","cancel":"","topic":"Refreshing chart","name":"","x":390,"y":60,"wires":[]},{"id":"96138b39.2d61d8","type":"ui_template","z":"d7d920d1.1e628","group":"684f06d8.eb11a8","name":"Logo display","order":1,"width":"4","height":"3","format":"<div ng-bind-html=\"msg.payload\"></div>","storeOutMessages":true,"fwdInMessages":true,"templateScope":"local","x":810,"y":300,"wires":[[]]},{"id":"58d47e8a.52b39","type":"function","z":"d7d920d1.1e628","name":"load logo","func":"var symbol = msg.topic.slice(0,-3);\nsymbol = symbol.toLowerCase();\n\nif (symbol === \"btcu\") { // USDT case\n symbol = \"btc\";\n}\n\nvar imgUrl = \"https://raw.githubusercontent.com/cjdowner/cryptocurrency-icons/master/128/color/\"+symbol+\".png\";\n\nmsg.payload = \"<img src='\"+imgUrl+\"'alt='Logo image not found' />\";\n\nreturn msg","outputs":1,"noerr":0,"x":660,"y":300,"wires":[["96138b39.2d61d8"]]},{"id":"673df297.a4404c","type":"delay","z":"d7d920d1.1e628","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"2","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":370,"y":160,"wires":[["3a5bf45.9251f0c"]]},{"id":"2bf673c4.ad97ac","type":"ui_button","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":9,"width":"2","height":"1","passthru":false,"label":"Stellar","color":"","bgcolor":"#99a2a6","icon":"","payload":"XLMBTC","payloadType":"str","topic":"","x":90,"y":340,"wires":[["71902fa5.40cf"]]},{"id":"e2a8bad4.68cb78","type":"ui_button","z":"d7d920d1.1e628","name":"","group":"3fa75da0.6d85c2","order":9,"width":"2","height":"1","passthru":false,"label":"Zcash","color":"","bgcolor":"#e49b2e","icon":"","payload":"ZECBTC","payloadType":"str","topic":"","x":90,"y":380,"wires":[["71902fa5.40cf"]]},{"id":"22694b27.9c4fd4","type":"delay","z":"d7d920d1.1e628","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"x":370,"y":640,"wires":[["4a8fb8a1.e63068"]]},{"id":"684f06d8.eb11a8","type":"ui_group","z":"","name":"Numbers","tab":"122ee689.480fd9","order":1,"disp":false,"width":"5"},{"id":"3fa75da0.6d85c2","type":"ui_group","z":"","name":"Chart","tab":"122ee689.480fd9","order":2,"disp":false,"width":"18"},{"id":"122ee689.480fd9","type":"ui_tab","z":"","name":"Binance Node Demo","icon":"fa-bitcoin"}]

Dashboards and graphs for cryptocurrency data using Node-RED: tutorial

Graphing cryptocurrency data using Node-RED

This is the second tutorial in a series on using the Binance cryptocurrency node we have developed. Take a look at the first tutorial if you haven’t already done so as it introduces you to the basics of the Binance node. This tutorial develops a simple dashboard showing bid and ask prices for symbol BTCUSDT and will look a little like this:

The Binance node

For those of you who aren’t familiar, Binance is one of the largest crypto trading platform (perhaps largest by volume), and importantly has a comprehensive and sophisticated API. The resident cryptocurrency expert, Ted, over at Sense Tecnic Systems (STS) recently decided to jump in and develop a new Node for accessing and using the Binance cryptocurrency platform.

Like all these tutorials and lectures, we use the FRED cloud Node-RED service to develop our flows as  Of course you can run Node-RED on your own machine. If you plan to use your own Node-RED installation, then you can download the Binance node from NPM download . If you are using FRED it’s already pre-loaded and can be activated by using the add/remove nodes button under tools on the management panel.

Basic dashboard example

In this tutorial we are going to show you how to use the Node-RED ui nodes with the Binance nodes to get basic exchange data and show it on a dashboard.

We’ll assume you have some knowledge of using the Node-RED ui nodes, if not, you may want to take a look at part 2 of the dashboard and UI techniques lecture.

Displaying bid/ask data

To get started, let’s set up a simple flow that shows the bid/ask price for BTCUSDT ticker pair that we used in the previous tutorial. We’ll use the getBookTicker node which returns bid and ask price, and we’ll place that info on a dashboard using the simple text widget ui nodes. The flow is shown below.

The first two nodes should be familiar from the previous tutorial. The text widget nodes require a little explanation.

Two things to note here are:

  • We’ve created a new UI group Binance Demo which is displayed on the newly created Cryptocurrency demo tab – again checkout lecture 7 part 2 if this is new to you. If you use the default home that’s ok and it will appear, with any other dashboard elements on your default home dashboard.
  • Secondly, we use the moustache formatted string {{msg.payload.bidPrice}} in the Value format field to tell the text UI node that we want it to display the bidPrice element of the message payload.

The askPrice UI node is set up in a similar manner, but obviously specifies the askPrice element of the msg.payload field.

Once deployed, you can access your dashboard using the dashboard tab on the Node-RED info pane (next to debug) or if you are using FRED, go directly to your own UI using the generic: URL https://[USERNAME].fred.sensetecnic.com/api/ui/#/0

Don’t forget to change USERNAME to your FRED user name – and you will need to select the correct tab if you have more than one dashboard.

When you click on the inject node in your flow, and then take a look at your dashboard, you should see a simple UI like the one shown below with the bid and ask price for BTCUSDT.

Adding a chart

Let’s spice up our dashboard a little by adding a chart. For this, we keep the same basic flow, but add a graph UI node. We’ll also need to take the output from the getBookTicker node and ‘massage’ it a little so the graph node can display both the ask price and the bid price on the same graph. Here’s the updated flow:

As you can see we’ve added two change nodes and named them isolate bidPrice and isolate askPrice. Change nodes allow you to take input messages and change the output based on a set of rules. To be able to use the chart properly, we need to deliver a msg.payload with a chartable value, and a msg.topic set to the name of the chartable value. If you take a look at the screenshot below, you can see how we set up a change node to do that.

Our first rule sets the output msg.payload, to the value received in the msg.payload.bidPrice element. Our second rule sets the msg.topic to the text field “bidPrice”. We do the same for the isolate askPrice change node and wire both up to the chart node.

Lastly, for the chart node, we leave it setup to default, except we change the group to place it into the same UI group as our text ui nodes, in our case Binance Demo [Cryptocurrency demo]

Now that we have two messages arriving at the chart node, it will use the msg.topic field to distinguish between the two and will apply a series colour to the data series that arrive, in our case dark blue to series 1 (bidPrice)and light blue to series 2 (askPrice).

The last thing we do is change the inject node to repeat every 10 seconds so we don’t have to keep clicking on it. Deploy and take a look at the dashboard which should look like the screenshot below.

The dashboard will update automatically every 10 seconds. Moving the mouse over the chart lines will show you data for the ask and bid prices at that point in time.

Lastly, you should clean up the UI by adding labels etc.

Have fun!

 


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.

Cryptocurrencies and Node-RED

Node-RED and cryptocurrencies

Node-RED is a perfect tool for rapid development of cryptocurrency applications. There are a wide range of uses, starting with simple access to exchange data (eg prices, trends) right up to developing your own blockchain!

Here are a few tutorials we’ve written to get you started plus some links to other informative sites and pages of people using Node-RED to explore cryptocurrencies.

Accessing basic price data via the Binance node

More sophisticated charting using the Binance node

Very sophisticated charting using the Binance node (coming soon)

Balance and trades using the Binance node (coming later)

Other interesting Info

  • marciogranzotto has written a wrapper node for Node-RED for the Poloniex exchange. The wrapper is quite basic but is a start point if you want to access that exchange.
  • If you are interested in the Cryptocompare API then a nice blog post by remko de knikker discusses how he created and added a set of nodes for cryptocompare to Node-RED. The blog post is quite involved as it discusses creating and publishing nodes, but it also refers to the nodes and some test flows he has published.
  • For those interested in the new lightning protocol, a set of nodes for accessing it has been developed by the
  • The folks at VISEO have developed a node for accessing parts of the etherium blockchain API.

 

Cryptocurrency prices and data from Binance with Node-RED: tutorial

Getting and displaying basic cryptocurrency data using Node-RED

In this brief tutorial, we’ll introduce the STS Binance node set and some of its capabilities. A follow-on tutorial will show how to develop a dashboard using the nodes and a final tutorial will discuss some possibilities using the trading API that the nodes offer.

The Binance node

For those of you who aren’t familiar, Binance is one of the largest crypto trading platform (perhaps largest by volume), and importantly has a comprehensive and sophisticated API. The resident cryptocurrency expert, Ted, over at Sense Tecnic Systems (STS) recently decided to jump in and develop a new Node for accessing and using the Binance cryptocurrency platform.

Like all these tutorials and lectures, we use the FRED cloud Node-RED service to develop our flows as  Of course you can run Node-RED on your own machine. If you plan to use your own Node-RED installation, then you can download the Binance node from NPM download . If you are using FRED it’s already pre-loaded and can be activated by using the add/remove nodes button under tools on the management panel.  See left image below. This will add the Binance node set, see right image below.

 

 

A couple of caveats:

  1. you’ll need a paid FRED tall (or higher) account to really use the node – otherwise your ability to track in real-time won’t be great
  2. Use it at your own risk – while you can use it for trading, you do so entirely at your own risk. 

Simple flow to get market data

Before we discuss the details of the Binance node, let’s take a look at a simple flow that accesses market data. For this demo we’ll use the Binance getPrice node. In FRED, on a new blank canvas, drag and drop three nodes, an inject node, a Binance getPrice node and a debug node as shown below.

The Binance getPrice node takes a simple ticker pair as input and returns the current price for that ticker. For this example, we’ll use the Bitcoin to Tether USD ticker pair, which is BTCUSDT. We’ll use the inject node to send that to the getPrice node, so double click on the inject node, set the payload to an empty string, and set it’s topic  to BTCUSDT as shown below.

The getPrice node looks for a ticker pair on the incoming message topic and uses that to make an API call to the Binance platform, which returns the current price. Our simple flow then passes that to a debug node, which will show the results in the debug pane on the right side of Node-RED window.

Go ahead and wire up your flow as described, deploy it and then click on the inject node and you’ll see the current price in the debug pane as shown below.

Pretty simple eh! – play around with different ticker pairs to look at your favourite currencies.

Binance node main features

The Binance node set covers two main areas of functionality – accessing market data and account related functionality. The market info nodes all access public APIs on the Binance platform and don’t require an account. The account related nodes access balance and trading features and will require you to setup an account with Binance and to provide an API key. We’ll cover that in a later tutorial, for now we’ll stick with the nodes that use the public API.

Accessing market info

  • getPrice: Gets the latest price of a symbol. Takes a ticker pair as input and returns the current price
  • getAllPrices: Gets the latest price of all symbols. No input and returns an {object} map of all available ticker pairs and their current prices
  • getBookTicker: Gets the bid/ask prices for a symbol. Takes a ticker pair as input and returns an {object} info on latest book price
  • getDayStats: Get the 24hr ticker price change statistics for a symbol. Takes a tickerPair as input and returns the latest 24h stats
  • getCandlesticks: Get Kline/candlestick data for a symbol. Takes a tickerPair, time interval and start/ed times and returns an {array} of candlesticks/kline data specified by parameters

Balance and trading (we’ll explore these in a later tutorial)

  • getOrders: Get open orders for a symbol. Takes and API credential and a tickerPair. Returns an {array} of current orders for that tickerPair
  • getBalance:  Get a list of current balances.Takes and API credential and a tickerPair and returns an {object} map of ticker symbols and quantity on the account
  • cancelOrders: Cancel all open orders of a ticker pair. Takes and API credential and a tickerPair and returns an {object} with the binance API response
  • getTradeHistory: Get trade history of a ticker pair.Takes and API credential and a tickerPair and returns an {array} list of previous orders
  • buy: Create a limit or market buy order. USE AT YOUR OWN RISK.
  • sell: Create a limit or market sell order. USE AT YOUR OWN RISK

Simple flow to get all market data

Let’s continue our simple example shown above but extend it to get all market info. For that we’ll need the getAllPrices node which, as you’d expect, returns all the prices that Binance handles. It’s a long list and is returned as an object. However, we can display it in the debug pane, so let’s modify our original flow and replace the getPrice node with a getAllPrices node as shown below. Note that the inject node doesn’t need to set the topic – although its ignored if you do.

As you can see, you’ll get back a json object – which if you click on to expand lists all the ticker pairs that Binance trades.

Getting bid/ask prices

As a final example, let’s return to the BTCUSDT ticker pair and get the full book info for it using the getBookTicker node. Using the first flow that injects BTCUSDT as a message topic, replace the getPrices node with a getBookTicker node as shown below.

As you can see, the object returned in the message payload for BTCUSDT includes bid price and quantity as well as ask price and quantity.

You can make a simple substitution of the getDayStats node for the getBookTicker node in the flow above to get a full set of day statistics for BTCUSDT. This is quite comprehensive and covers opening prices, previous day close, low and high prices for the day as well as trading volumes etc.

The last node in the set of market access nodes is the getCandlesticks node which gets Kline/candlestick data for a symbol. This requires some knowledge of setting up UI elements and dashboards in Node-RED so we’ll cover that in our next tutorial.

Have fun!


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.

Tutorial: Node-RED dashboards – creating your own UI widget II (using external charts)

Creating your own UI widget for a Node-RED dashboard using an external charting library

Node-RED’s dashboard nodes provide a comprehensive set of UI components for building basic dashboards suitable for the Internet of Things (IoT) – offering graphs, gauges, basic text as well as sliders and inputs. However, there will always be situations when you need something custom. The template node is the solution and in this tutorial we’ll show you a quick example of using it to create your own UI widget.

If you want a general introduction to the Node-RED dashboard or to UI techniques for Node-RED, then check out these resources which are part of our lecture series:

The template node

The ui template node is a generic node that takes valid html and Angular/Angular-Material directives and can be used to create a dynamic user interface element.

In this example, we’ll create a simple dashboard element that displays some fake random data mimicking data from the UK’s power generation network.

The dashboard widget using morris.js

We want our UI element to look as shown on the right of the screen shot below (you can read the tutorials on how to create the Table UI widget and the 2 line chart seperately)

To build this widget, we’re going to use a 3rd party JavaScript charting library morris.js – which is simple, quite powerful, but very easy to use. It has several charts including excellent line and area charts. For this example we’ll use a donut chart since it’s not available in the Node-RED dashboard widget set.

Apart from setting up the fake data, and drawing the donut chart, two key aspects of this tutorial which are generic issues and will be needed whatever external libraries you use are:

  • loading the external libraries – and making sure they are loaded before we get started
  • watching for new messages as they arrive at the template node and using the msg.payload to update the chart.

The Node-RED flow

We’ll create a very simple flow for this example consisting of a timer to trigger action every 10 seconds, a function node to generate the fake data and the template node to display the data.

The timestamp node can be configured to fire every 10 seconds. The function node (Generate fake Electricity data) is a simple as possible. It generates a random number and then uses that as a seed to calculate fake values for the different generation types, ie Gas,Wind, Coal, Oil and Nuclear.

The fake data is dropped into the output msg.payload and sent on to the template node.

The template node is a little more complex than previous examples because we are pulling in an external charting library (morris.js) and it’s dependencies and we are writing some JavaScript code to use the incoming data (from the function node) to draw the donut chart.

The first part of the template node is fairly standard and uses script tags to define the src for the JavaScript libraries that we will be using. These are hosted on cloudflare.com or googleapis.com (they can be hosted anywhere – although if you are using FRED, you can’t store them locally).

After the libraries are referenced, we define a <div> (line 10) that is the element that the chart library will use and a simple header for the chart (line 14).

The actual javscript starts at the <script> tag at line 18 and is shown below.




There are 3 main pieces of this code -checking that the external libraries are loaded (line 20-22), the creation of the Morris.Donut object and its initialization (lines 26-35), watching for new Node-RED messages and updating the chart (lines 37-49). All of this is wrapped in a function that accesses the Angular scope for this DOM object.

Check external libraries are loaded

Lines 20-22 use a setInterval function to set up a simple repeating function that firstly checks that the external libraries are loaded. It does this by checking to see if the Morris object exists (line 21). If not the function returns and exits at line 50 where we wait (100 ms) before trying again. This simple code pattern is critical and is required whenever you use external libraries in the template node. This is because Angular is async and will happily tell the browser to start loading external libraries and then get on with running your code without waiting for them to finish loading. If the libraries aren’t loaded, the first time you try and use them you’ll get an undefined error.

Create the morris.js donut chart element and initialize it

It’s beyond the scope of this tutorial to explain the details of the morris.js library. You can check out an example we’ve written previously or the many examples available online, eg here or  donut example

However, as can be seen, line 26 creates a new instance of the Morris.Donut object. Creation requires a minimum of an element (the HTML element that will be used for displaying the donut chart) and an array of data to display structured as <label:labelname><value:actualvalue>. We create a dummy array with all values set to 1.

Watch for incoming msg.payload

To update the donut chart, we want to look at the incoming Node-RED message, and use its payload to update the donut chart. To do this, we use a watch function at line 37 to look for changes to the scope variable msg, which contains the incoming Node-RED message.

Again, the details of how Angular works is beyond this simple tutorial. A good overview is available on the Angular docs page. Briefly, Angular implements a Model-View-Controller (MVC) in which the model (the data) is separate from the view (the DOM). Controllers (code), Expressions and Directives manipulate the data which dynamically updates the view (the Dom) so you see the changes. The model is managed in a structure referred to as the scope.

So coming back to our example, the code at line 37 tells the template node to look at the scope (all our data) and watch for changes to the variable msg. When msg changes, in this case because a new Node-RED message has arrived, the function then executes (lines 38-48) and simply extracts the new data from the msg.payload and uses it to call the setData() function on the Morris.Donut object (myDonut) which dynamically updates the donut chart.

All of the above is wrapped in a function (lines 18-51) which passes the scope so that it can be accessed using the watch method.

Conclusion

Once you setup the flow, you’ll be able see the new UI element on your dashboard and as the flow triggers a new message the donut chart will update.

We’ve chosen to use a simple external charting library for this example, Morris.js – which is quite simple. However the techniques used here are the same for all external libraries so feel free to replace Morris.js with any of the many other charting libraries out there.

 <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.css">
 <script src="https://cdnjs.cloudflare.com/ajax/libs/raphael/2.1.0/raphael-min.js"></script>
 <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>
 <script src="https://cdnjs.cloudflare.com/ajax/libs/morris.js/0.5.1/morris.min.js"></script>

<div id="chart-example" style="height: 250px;"></div>

<h3 align="center">UK power generation by type (in GigaWatt) </h3>

<script>

(function(scope) {
 var timer = setInterval(function() { //check that the Morris libs are loaded, if not wait
 if (!window.Morris) return;
 clearInterval(timer);
 
 var myData;
 
 var myDonut = Morris.Donut({
 element: 'chart-example',
 data: [
 {label: "Coal GW", value: 1 },
 {label: "Gas GW", value: 1 },
 {label: "Nuclear GW", value: 1 },
 {label: "Oil GW", value: 1 },
 {label: "Wind GW", value: 1 }
 ]
 });
 
 scope.$watch('msg', function (msg) { //watch for an incoming NR msg
 if (msg) {
 
 myData = msg;
 myDonut.setData([
 {label: "Coal GW", value: myData.payload.coalPct },
 {label: "Gas GW", value: myData.payload.gasPct },
 {label: "Nuclear GW", value: myData.payload.nucPct },
 {label: "Oil GW", value: myData.payload.oilPct },
 {label: "Wind GW", value: myData.payload.windPct }
 ])
 };
 }); //close out the watch 
 }, 100); // close out the setInterval 
})(scope);


</script>

About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.

Tutorial: Node-RED dashboards – creating your own UI widget

Creating your own UI widget for a Node-RED dashboard

Node-RED’s dashboard nodes provide a comprehensive set of UI components for building basic dashboards suitable for the Internet of Things (IoT) – offering graphs, gauges, basic text as well as sliders and inputs. However, there will always be situations when you need something custom. The template node is the solution and in this tutorial we’ll show you a quick example of using it to create your own UI widget.

If you want a general introduction to the Node-RED dashboard or to UI techniques for Node-RED, then check out these resources which are part of our lecture series:

You can also check out a related tutorial that describes how to use an external charting library (morris.js) to create your own UI widgets with alternative graphs/charts.

The template node

The ui template node is a generic node that takes valid html and Angular/Angular-Material directives and can be used to create a dynamic user interface element.

In this example, we’ll create a simple dashboard element that displays some fake random data mimicking data from a factory machine sensor.

We want our UI element to look as shown on the right of the screen shot below (you can see how to create the 2 line chart in this tutorial)

screen-shot-2016-12-07-at-7-31-32-am

As you can see – our UI widget is named “and more” and has four data columns. These data column show the fake factory machine data and are dynamically updated as new data arrives.

We’ll set up a flow as shown below – the function node will generate the fake data, and the UI template node, which we name Machine status, will format and present the data in the table shown above.

screen-shot-2016-12-07-at-7-21-49-am

If we take a look inside the function node, you can see we use the usual Math.random function to generate some fake data. However, instead of just passing this data in as payload, we insert the data into a JSON structure called factory. Factory has two elements -machine00 and machine01 which in turn have four elements min, max, failProb and name.

screen-shot-2016-12-07-at-7-22-44-am

min = Math.round(Math.random()*1000);
max = Math.round((Math.random()*10000)+1000);

msg.payload = {
  factory: {
    machine00: {
      min: min,
      max: max,
      failProb: 20,
      name: "GE 5200"
    },
    machine01: {
      min: min*5,
      max: max*2,
      failProb: 10,
      name: "BB SQ100"
    }
  }
}

return msg;

Once we have set up our function node to create the fake data. We now need to create a UI template node and set it up to display on our dashboard. As you can see below, we assign it to the Home dashboard and use a new group name “and more”. (Details on setting up dashboard groups can be found here)

screen-shot-2016-12-07-at-7-23-09-am

Let’s take a look in more detail at the HTML used to display the data we generate in the function node.

The first <div> sets out the labels for the columns defining them using the <span flex> tag so they will fit the space nicely.

<div layout="row" layout-align="start center">
  <span flex>RPM Min</span>
  <span flex>RPM Max</span>
  <span flex>Failure Prob</span>
  <span flex>Machine type</span>
</div>
<div layout="row" layout-align="start center" ng-repeat="machine in msg.payload.factory">
  <span flex style="color: green">{{machine.min}}</span>
  <span flex style="color: red">{{machine.max}}</span>
  <span flex style="color: black">{{machine.failProb}}%</span>
  <span flex style="color: black">{{machine.name}}</span>
</div>

The second <div> then uses the ng-repeat command to iterate through the JSON structure in payload and display the data in the columns. It works by looking for each element named machine in the factory element. Remember, we named each set of data associated with a machine, machine00, machine01 – so we iterate through those.

For each machine element, we use the sub-elements; min, max, failProb and name to fill the data fields. Note how we also use a style directive to set the colour for the text. The result is shown below.

screen-shot-2016-12-07-at-7-22-08-am

This is a simple use of the template node, but it should be enough to get you started. However, the template node is much more sophisticated, it can be used to dynamically set styles (e.g. colours), show images, create clickable elements that generate messages etc – so once you master the basics, you can go on to create some sophisticated UI widgets for your own dashboards.

To get you started with more sophisticated uses of the template node, we’ve written a related tutorial that uses an external charting library and a template node.


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.

Tutorial: Node-RED dashboards – multiple lines on a chart

Showing multiple lines on a single chart in Node-RED

This simple tutorial explains how to display multiple lines on a Node-RED chart. We’ll be building on a previous example in our lecture series (Example 7.2 in lecture 7)

As discussed in that lecture, let’s set up a simple flow to generate a random number and display it on a chart.

We use a cloud hosted version of Node-RED for these tutorials called FRED. Sign up for a free account at FRED. You can use this tutorial with a vanilla N0de-RED, eg on a Pi, but make sure you are accessing the correct URL for the dashboard (and not the FRED specific one).

To start, let’s wire up a simple flow that sends a random number between 0 and 99 to a simple chart. For that you’ll need an inject node to repeatedly fire every few seconds, a function node to generate the random number and one of the  node-red-dashboard nodes – in this case the chart node.

screen-shot-2016-10-12-at-4-38-47-pm

Before we look a how the chart node works, let’s configure the inject node to send a timestamp every 5 seconds by setting the payload to timestamp and the repeat field to an interval of 5 seconds.

Screen Shot 2016-05-16 at 12.46.11 PM

This will act as our repeating trigger. Now we need to set up the function node to generate a random number – we’ll use a simple JS math function to do this:

msg.payload = Math.round(Math.random()*100);
return msg;

Screen Shot 2016-05-16 at 12.48.42 PM

This will generate a random number between 0 ~ 99 which is passed to the chart node.

So now let’s take a look at the chart node. When you double click it, you’ll see it’s configuration options:screen-shot-2016-10-12-at-4-46-53-pm

If you click on the button of the Group field, you will be prompted  to configure the tabs of the UI.

screen-shot-2016-10-12-at-4-42-04-pm

The Tab option allows you to specify which tab of the UI page you will see the UI element on – in this case our chart. The default tab is Home – which we are using here. If you select the edit button to the right of the Tab field you can create a new tab and then select that. However, we’ll use the default home for now.

The Name field is the standard Node-RED node name – by default this is chart but you can set it to anything you like.

The Group field allows you to group UI elements – we’ll show you how that works when we add another UI element so let’s use group “Default[Home]” for now – of course, you can use any string you like.

The X-axis field allows you to tell the chart how much data it should store and display – the longer the ‘last‘ filed is set to, the more data is stored and displayed by the chart. Let’s use a short 5 mins which will start to throw away the data that is 5 minutes old.

Lastly the Interpolate field defines how the chart will interpolate values in between actual data values it receives, you can select between linear, step, b-spline and cardinal – which are standard interpolation algorithms. We’ll use the default linear.

Wire these nodes up, hit the deploy button – check that your debug node is showing that random values are showing. Then head over to your default dashboard page to see the results. By default, under FRED, you’ll see your UI at:

https://{your user name}.fred.sensetecnic.com/api/ui/

When you visit that page you’ll see your initial chart as shown below:

screen-shot-2016-10-12-at-4-48-47-pm

As you can see, you get a nice line chart, scaled for your data values (X axis) and the time they arrived (y axis). You can also see “Default” at the top left of the chart, indicating this is UI group “Default” – which is the group name we set in the configuration fields for the chart node.

 

Adding a second line to our chart

Ok, now let’s see how to add another line to the same chart.

If you take a look at the info box for the chart node, you’ll see it mentions that the the msg.topic field can be used to show multiple lines. Essentially the chart node will receive messages, take a look at the msg.topic field, and then append the data to the appropriate line as defined by the topic.

So, to extend our example to handle multiple lines on a single chart, we have to do two things:

  1. update our function node to generate two random numbers and assign these to different messages with different topics
  2. Wire up a second output from the function node to handle the second message

To do that, open up the function node and update the code inside as shown below. As you can see we explicitly create two messages, msg and msg1 (lines 1-2).

At line 4-5 we set msg.topic to ‘Line1’ and a random number is assigned to the payload. We repeat that at line7-8 setting msg1.topic to ‘Line2’ and assigning a different random number.

screen-shot-2016-12-05-at-5-28-56-pm

Both messages are returned using the return message (line 9) – because we are generating two return message, we need to tell the function node that it has two outputs by setting the outputs field to 2 (see above).

More details of generating multiple messages from nodes can be found in the main lectures Example 5.2 in lecture 5.

Once you’ve done this, your flow should look like the screen shot below.

screen-shot-2016-12-05-at-5-23-17-pm

Wire up both outputs from the function node to the chart node and then take a look at the dashboard and the chart. As you can see (shown below) the chart now shows two lines (Line1 and Line2) with different colours. Note we set the dashboard theme to dark to make the two lines clearer.screen-shot-2016-12-05-at-5-24-06-pm

Obviously this example is contrived, in a more realistic example you might have two sources of data, and so two function nodes delivering messages (rather than the single function node with two outputs).

In addition, you’d be pulling the data from a real source, e.g. a web scrape, an MQTT feed or perhaps a DB. You can see examples of these:


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide these lectures and FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.

Tutorial: SQLite and Node-RED

This tutorial will show you how to make use of an SQLite database using the litedb node on the FRED hosted Node-RED platform to create database tables and store data. Under the hood, the litedb node uses SQLite.  As a very lightweight relational database, SQLite does not need complex setup procedures, making it an ideal database management system to use for embedded systems and rapid prototyping.

The tutorial will cover the basics of:

  • Setting up the dblite (SQLite) node and creating a DB table
  • Generating random data and storing in the SQLite DB
  • Reading from the SQLite DB
  • Graphing the data from the DB using the standard Node-RED UI nodes

 

For the tutorial, we’ll be using the FRED cloud based Node-RED service (You can install the litedb node on your own machine if you prefer). Because FRED is a shared service, the litedb node limits your use of SQLite in a couple of ways:

  • There is no way to specify the database file.  Your FRED instance only uses a single SQLite database.
  • The SQLite database file on FRED is limited to 4mb

Keep in mind that, although there is a limit of 4mb, you are still able to create as many tables as you want up to the size limit. With the sample table that we created for this tutorial, the 4mb database file is able to hold approximately 140,000 records.  For more information on SQLite see More documentation can be found on https://sqlite.org/.

Sample flows for this tutorial is also available in here: SQLite with litedb node on FRED

What is Node RED?

Node-RED is a visual tool for wiring the Internet of Things developed by IBM Emerging Technology and the open source community. Using Node-RED, developers wire up input, output and processing nodes to create flows to process data, control things, or send alerts. It works by allowing you to wire up web services or custom “nodes” to each other, or to things, to do things like:

  • Send an email on a rainy weather forecast.
  • Push sensor data to services like Twitter.
  • Perform complex analysis on data with ease.

showcase-flow

If you’re new to Node RED you might find our Introduction to Node RED tutorial very helpful.

What is FRED

The Front End for Node-RED (FRED) manages instances of Node-RED for multiple users in the cloud. We manage and optimize your instance of Node RED so you worry about accomplishing your project, not setting up and maintaining your Node-RED instance.


About Sense TecnicSense Tecnic Systems Inc have been building IoT applications and services since 2010. We provide FRED, cloud hosted Node-RED as a service to the community. We also offer a commercial version to our customers, as well as professional services. Learn more.


Create a FRED Account

To get started, create your own Node-RED instance in the cloud. Register for a free account at http://fred.sensetecnic.com.

After registering, make sure to activate your account via your email. You will not be able to login until you validate your account.

Installing the litedb node

First, you will need to ensure that you have installed the litedb node since it is not installed by default in FRED. After you register for FRED, or after you login, you will be presented with the FRED dashboard.

In the management panel, click “Add or Remove Nodes“, and find the litedb node.

Screen Shot 2016-08-31 at 4.31.41 PM

Make sure the litedb node is installed for your instance. If the box beside ‘litedb’ is empty in your Nodes list, just simply check the box and restart your instance.

Make sure you start your instance by hitting the start button.  If it is already started, stop, and start it to ensure the new litedb node is available.  Create a new tab to start with an empty canvas.

Once the litedb node is installed, you should see the litedb node in your left panel. Drag a litedb node to the empty canvas, and we are ready to move on. You can also name the litedb node however you like by double clicking on the node to open the configuration dialog if you like.

Screen Shot 2016-08-31 at 3.00.32 PM

For demo purposes, we are going to create a table and inject random numbers at a constant time interval. We will also look at how to view and manage this data later on.

Creating your first table

As documented on the info tab of litedb, the node takes in SQL queries in the input msg.topic, and then returns responses in the output msg.payload. To create a table to store some data, all we need to do is send the appropriate SQL command to the litedb node. A simple way to do that is to connect an inject node to the litedb node with the following settings:

createdb

where in the topic field, we have the query

CREATE TABLE RANDOMNUM( TIMESTAMP INT PRIMARY KEY NOT NULL, VALUE INT NOT NULL, BOOL INT NOT NULL)

As you can see, this demo table is called “RANDOMNUM“, and it has three columns, TIMESTAMP( Primary key column ), VALUE, and BOOL. (We will use BOOL to store an attribute which indicates if the number is larger than 50 or not) These three columns are all set to NOT NULL and with type INT.

Once you are done, connect a debug node to the litedb node, and deploy the current settings. Hit the inject button, and you should see that RANDOMNUM is created.

Screen Shot 2016-08-31 at 3.35.38 PM

 

Inserting Data

We can use the same method to send an SQLite query to the database to inject a new value into the table.

Let’s set up another inject node that injects a new timestamp every 15 seconds simulating a sensor event. Followed by a function node, we will write a query with the input of the timestamp, and send the insertion query to the litedb node.

First, drag a new inject node, and add these settings.

Screen Shot 2016-08-31 at 3.57.21 PM

then, let’s connect this timestamp node to a function node.Screen Shot 2016-08-31 at 3.44.30 PM

 

We name the function node “write query” and we define the code inside the function node would be as follows:

var randomNum = Math.round(Math.random()*100);
var largeBool = (randomNum > 50)?1:0;
var newMsg = {
 "topic": "INSERT INTO RANDOMNUM VALUES ( " + msg.payload + ", " + randomNum + ", " + largeBool + ")"
}

return newMsg;

In the above code, a new record is inserted into RANDOMNUM with a timestamp as its primary key, a randomly generated value (ranged from 0 to 100), and a boolean value as if the number is larger than 50. Notice that there is no actual boolean data type in SQLite, so we represented boolean as either 0 or 1.

Then, we can just connect the function node to the litedb node as follow. Once you hit deploy, the inject node will start inserting new records to the table.

Screen Shot 2016-08-31 at 3.48.47 PM

 

Selecting from database

To see the records from our table, we can set up another new inject node, with this in the settings:

Screen Shot 2016-08-31 at 3.58.15 PM

where the Topic field is

SELECT * FROM RANDOMNUM ORDER BY TIMESTAMP DESC LIMIT 100;

As you can see, we are showing the first 100 records in table RANDOMNUM ordered by the TIMESTAMP column in descending order.

After you deploy the nodes, and inject the SELECT query command, you should see something like this as the output of the debug node:

SELECT * FROM RANDOMNUM ORDER BY TIMESTAMP DESC LIMIT 100; : msg.payload : array [1][ { "TIMESTAMP": 1472684511852, "VALUE": 5, "BOOL": 0 } ]

 

Controlling the size of your database

If you follow this guide, you will notice how we set up an inject node that injects records in 15s intervals. In real world scenarios, the database grows in size and may eventually affect the performance of servers. In addition, and as mentioned above, the litedb node has a limit of 4Mb in size. Here we will show you an example of how to delete older records that you do not want to keep.

We will set up another new inject node, and call it ‘delete’. For this example, it will run the  delete query every minute, to remove the data that is older than 1 hour.

Screen Shot 2016-09-01 at 9.49.37 AM

where the query in Topic field is

DELETE FROM RANDOMNUM WHERE TIMESTAMP <= strftime('%s','now', '-1 hours')*1000

Notice that, the timestamp generated from the inject node is UNIX epoch time in milliseconds, but the default timestamp return from the SQLite function strftime() is in seconds. That’s why we need to multiply 1000 at the end.

As you can see here, we use a modifier, ‘-1 hours’, to specify the time interval of data that we want to keep. In our case, this inject query means that we want to clean data that is one hour old every minute. We choose 1 hour in this case just for demo purpose. You can use different modifiers to suit your application, more info on the function strftime() and the modifiers can be found in: https://www.sqlite.org/lang_datefunc.html

While you are trying this deletion node, you can also set up an inject node to count the size of the table with this query

SELECT COUNT(*) FROM RANDOMNUM

Don’t forget to connect the inject nodes to the litedb node in order to send the query command into SQLite!

Now sit back and see how the node clears out the old data for you in the debug tab.

Visualizing data

Users are able to visualize the data by using the dashboard nodes. You can refer to the tutorial in A Node-RED dashboard using node-red-contrib-dashboard for more information on how to do this.  In this tutorial, we will show you another trick to visualize the data in a table form.

From the previous exercises, we know that when we send

SELECT * FROM RANDOMNUM ORDER BY TIMESTAMP DESC LIMIT 100;

into the litedb node, it returns an array of records in the response, we can simply use the ui template node to parse the information we want.

For example, we would like to have a table showing the first 20 records, just simply connect the template UI node to the litedb node:

in the template node, enter the following code:

<table style="width:100%">
  <tr>
    <th>Index</th> 
    <th>Timestamp</th>
    <th>Value</th> 
    <th>Bool</th>
  </tr>
  <tr ng-repeat="x in msg.payload | limitTo:20">
    <td>{{$index}}</td>
    <td>{{msg.payload[$index].TIMESTAMP}}</td>
    <td>{{msg.payload[$index].VALUE}}</td> 
    <td>{{msg.payload[$index].BOOL}}</td>
  </tr>
</table>

 

and it looks like this in the config tab:

screen-shot-2016-09-13-at-3-56-20-pm

This code simply generates a table in HTML, and places the index, TIMESTAMP, VALUE and BOOL fields from  msg.payload into the appropriate cells of the table. It then repeats for the next 19 records (using ng-repeat & limitTo). You can set up the table however you like. Once you deploy the nodes, the UI template node will update the table whenever SELECT * is sent to the litedb node, and an array of records are returned from the litedb node to the UI template node. The resulting table would look like this when you hit the URL https://{REPLACE-WITH-YOUR-USERNAME}.fred.sensetecnic.com/api/ui:

 

This covers basic usage of the litedb node. Although the example in this guide just stores random generated numbers into the database, you can grab data from any data source and insert it into the database, and query it any way you like. For more information on specific SQLite queries, see https://www.sqlite.org/lang.html