An experiment to see how far is it possible to build an Erlang backend for the Node-RED flow editor that seemlessly executes existing Node-RED flows.

The idea is not to provide Erlang nodes for NodeRED rather the idea is to replace the NodeJS backend of Node-RED with an Erlang equivalent.

The goal is to offer the advantages of visual flow based programming using a programming language that is designed for message passing and concurrency from the ground up - that's why Erlang was selected.

All Node-RED flows upon which this is tested are located in priv/ and can be viewed in Node-RED using the serverless instance of Node-RED.

Use the import functionality and paste the included .json into the dialog.

Eventually this repository will contain a serverless version of Node-RED that will interact with a Erlang backend - but that's future works. This is now the case.

Node-RED is a amazing[*] tool for creating flows that describe concurrent processing, it is just a shame the NodeJS is single threaded. So why not use something that is multi-process from the ground up? That way concurrency is guaranteed.

Also Erlang isn't the most understandable of programming language - unless one was born has fallen into in a cauldron of Prolog and Lisp!

So won't it be great to have the simplicity of low-code visual flow based programming and the performance (and concurrency) of Erlang?

[*] = amazing in the sense of extendability (the codebase is Javascript both front and back), understandability (Node-RED terminology is straightforward) and usability (many would disagree but once a rectangle becomes a unit of computation and a line becomes a pathway for data - it's simple!).

A kind of HOWTO describes my development process, best described as flow driven development.

The description is aimed at folks who have never used Node-RED but who would like to contribute. Feel free to send me questions relating to the project and its development.

The architecture, along with a background to Flow Based Programming and Node-RED has been written.

The codebase as it stands, has many interdependence because of testing flows: assert nodes need to know about web-socket communication, or because of the nature of nodes: the complete node needs to know of completed messages.

While the exception node needs to know about exceptions when they happen.

So an architectural diagram of this interconnectedness would be probably be just as confusing the code itself.

This is a non-complete list of nodes that partially or completely work:

Contexts are not supported, so there is no setting things on flow, node or global.

$ rebar3 compile

$ rebar3 eunit

$ rebar3 shell --apps erlang_red

Open the Node-RED visual flow editor in a browser:

$ open -a Firefox http://localhost:9090/node-red

I use docker to develop this so for me, the following works:

prompt$ git clone git@github.com:gorenje/erlang-red.git
prompt$ docker run -it -v $(pwd)/erlang-red:/code -p 9090:8080 -w /code --rm erlang bash
docker> rebar3 shell --apps erlang_red

Then from the docker host machine, open a browser:

prompt$ open -a Firefox http://localhost:9090/node-red

That should display the Node-RED visual editor.

A release can be bundled together:

$ rebar3 release

All static frontend code (for the Node-RED flow editor) and the test flow files in priv/testflows are bundled into the release.

Cowboy server will listen on port 8080.

A sample Dockerfile Dockerfile.fly is provided to allow for easy launching of an instance as a fly application. The provided shell script (fly_er.sh) sets some common expected parameters for the launch. Advanced users may wish to examine the fly launch line therein and adjust for their requirements.

Using the container stack at heroku, deployment becomes a git push heroku after the usual heroku setup:

• heroku login --> heroku git:remote -a <app name> --> heroku stack:set container --> git push heroku

However the Dockerfile.heroku does not start the flow editor, the image is designed to run a single flow, in this case (at time of writing) a simple website with a single page. Basically this flow is the red-erik.org site.

The image does this by setting the following ENV variables:

• COMPUTEFLOW=499288ab4007ac6a - flow to be used
• DISABLE_FLOWEDITOR=YES - any value will do, if set then floweditor is disabled.

Also be aware that Erlang-RED supports a PORT env variable to specifying the port upon which Cowboy will listen on for connections. The default is 8080.

Heroku uses this to specify the port to connect for a docker image so that its load balancer can get it right.

What the gif shows is executing a simple flow using Erlang as a backend. The flow demonstrates the difference in the switch node of 'check all' or 'stop at first match'.

All nodes are are pids - that is shown on the left in the terminal window.

Obviously this example is extremely trivial but it does lay the groundwork for expansion. I, for one, would rather code Erlang visually, low-code than in my Emacs ;)

To create unit tests for this, Node-RED frontend has been extended with a "Create Test Case" button on the export dialog:

Then flow is then stored in the testflows dir and will be picked up the next time make eunit-test is called. In this way it is possible to create unit tests visually.

To better support testing of flows, two new nodes have been created:

"Assert Failed" node cases unit tests to fail if a message reaches it, regardless of any message values. It's basically the same as a assert(false) call. The intention is to ensure that specific parts of a flow aren't reached.

The second node (in green) is an equivalent to a change node except it contains test on attributes of the message object. Possible tests include 'equal', 'match', 'unset' and their inverses. Here the intention is that a message passes through is tested for specific values else the unit test fails.

These nodes are necessary since there is no other way to test whether flow is working or not.

Also remember these flow tests are designed to ensure the Erlang backend is correctly implementing node functionality. The purpose of these nodes is not to ensure a flow is correct, rather that the functionality of implemented nodes works and continues to work correctly.

My plan is to create test flows that represent specific NodeRED functionality that needs to be implemented by Erlang-RED. This provides regression testing and todos for the implementation.

I created a shortcut storing and creating these flows but I was still using the terminal to execute tests make eunit-test became painful. Instead I pulled this testing into Node-RED, as the gif demonstrates:

What the gif shows is my list of unit tests, which at the press of a button, can all be tested. Notifications for each test shows the result. In addition, the tree list shows which tests failed/succeed (red 'x' or green check). Also tests can be executed individually so that fixes may be applied.

The best bit though is that all errors are pushed to the debug panel and from there I get directly to the node causing the error. Unit testing is now completely integrated into Erlang-RED.

My intention is to create many small flows that represent functionality that needs to be implemented by Erlang-RED. These unit tests shows the compatibility to Node-RED and less the correctness of the Erlang code.

Contributions very much welcome in the form of Erlang code or as Node-RED test-flows, ideally with the Erlang implementation.

Each test flow should test exactly one feature and use the assert nodes to check correctness of the test. Tests can also be pending to indicate that the corresponding Erlang functionality is still missing.

Questions and Answers at either the Erlang Forum or the Node-RED Forum.

To branch or not to branch, that isn't really a question. I'm currently working directly on main but ensure that all tests succeed before pushing, so main branch will always work.

Versioning is completely random and has little or no meaning at the moment. I prefer to use Milestones since these are arrived at and are not planned, I don't know when the next milestone will be reached.

Locally I work with branches but have no desire to make those branches public since I'm working on my own.

If this project becomes more collaborative or a "production ready piece of software", then more certainity will be applied to the development process.

I'm more than happy to deal with conflicts if someone developed something on a branch and it doesn't merge - I understand that multiple direct pushes to main everyday isn't the done thing but I don't like to have code lying around for weeks on end not being merged because it's not on a release schedule.

Coding is a creative process, creativity cannot be planned. Imagine Van Gogh working to a release plan.

Nick and Dave for bring Node-RED to live - amazing quality and flexibility and the entire Node-RED community.

Much thanks to

• @mwmiller for providing a fly server for running a live version of Erlang-RED,
• @joaohf many tips on coding Erlang and structuring an Erlang project, and
• @Maria-12648430 for debugging my initial attempt to create a gen_server for nodes.

No Artificial Intelligence was harmed in the creation of this codebase. This codebase is old skool search engine (ddg), stackoverflow, blog posts and RTFM technology.

Also be aware that this project partly uses the Don't do Evil un-enforceable license. The point of the license is not to be enforceable but to make the reader think about what is evil. After all, Pope Leo (the new one) did say "evil will not prevail" - what does that even mean?