Elastic Network of Things with MicroPython

Wei Lin
20170120

 photo source

Abstract

 An IoT system has many nodes, each node with very limited resources and usually needs to be uninstalled to have software modified. Considering an IoT system of 100 nodes, it will be a tedious task to revising them. 

  IPython Parallel is a famous distributed-task-queue framework, I built a small system to demonstrate that we can use ESP8266 modules running MicroPython as Workers(Engines) in a IPython Parallel liked Hub-Workers architecture. In this system we can read/write remote nodes' GPIO pins and control them accordingly in a central-control fashion. We can also send any arbitrary script for remote nodes to execute. Also, script files can be uploaded to nodes and have them start operating autonomously.

 The mechanism described above can be used to deploy and manage an IoT system in a whole new way, in which the functionalities and behavior of nodes can be modified remotely and dynamically,  although they had the identical initial program on installation.

Github Repository

IPython Parallel:

In  IPython Parallel's architecture, a hub is responsible for bridging the communication between worker nodes.  Moreover, with IPython Parallel we can send any arbitrary code for workers to execute. That's impressing. Since we have MicroPython now, why don't we follow this paradigm in building IoT system? It will be far more elastic and powerful !

Just image that we don't need to flash each nodes differently, they all have the same initial code. A system plan will be deployed over the network to each node and modify them accordingly, all at once. Each node and the whole is highly elastic !

photo source

The Devices:
I use  NodeMCU and D1 min
NodeMCU

 D1 mini

Experiment Preparation and Setup:�

config.py

In this configuration file you only need to set the BROKER_HOST address，ip or domain name will do.
Every node will try to connect the broker after boot, so if the broker has a public ip, then every node will be able to connect it.
Modify the HUB_PORT to 80 if necessary, however not recommended (there are scanners out there).

# filename: codes/shared/config.py

## Must config ******************
BROKER_HOST = '192.168.0.105'
BIND_IP = '0.0.0.0' # the ip which broker listens to.
HUB_PORT = 9662
## Must config ****************

Start the Broker�

Run the script below to start Broker. If nodes have already connected the Broker via WiFi network, then you will see a list of nodes' on broker's console. You will need the name list afterward.

 1_start_broker.py 

Tests:�

Read / Write remote GPIO pins�

For demonstration, please run the script ` 2_start_client.py `, there are some example messages in the file.
Before running the script, your have to write the names of nodes into the script. There is a list variable for that.

This message will ask remote nodes to blink LED 

messages['blink_led'] = {'type': 'command',
    'command': 'blink led',
    'kwargs': {'times': 10, 'forever': False, 'on_seconds': 0.05, 'off_seconds': 0.05}}

This message will ask remote nodes to return the status of GPIO pins. 

messages['read_GPIOs'] = {'type': 'command',
     'command': 'read GPIOs',
       'need_result': True}

This message will write values to remote nodes' GPIO pins. 

messages['write_GPIOs'] = {'type': 'command',
      'command': 'write GPIOs',
      'kwargs': {'pins_and_values': [(2, 0), (2, 1), (2, 0),]}}

This message will ask remote nodes to evaluate  '2+3' and return result.

messages['test eval'] = {'type': 'eval',
    'to_evaluate': '2+3',
    'need_result': True}

This message will ask remote nodes to execute a statement.

messages['test exec'] = {'type': 'exec',
    'to_exec': 'print("Testing exec !")'}

This message will sent to remote nodes with this function.

client.request(remote_node, message)

Deploy Script file to remote nodes and have them operating autonomously

If needed, a script file can be uploaded to remote nodes and an algorithm  is put inside a  while True loop, so the node will function autonomously and only report to central periodically.
For example, the on board LED will flash continuously  after the following script file was uploaded onto remote nodes.

# filename: codes/_demo/script_to_deploy.py
print('_______ testing remote deploy ______')
print('_______ deployed from remote _______')

import machine
import time

def blink(pin, on_seconds = 0.5, off_seconds = 0.5, on = 0, off = 1):
 pin.value(on)
 time.sleep(on_seconds)
 pin.value(off)
 time.sleep(off_seconds)
def main():
 on_board_led = machine.Pin(2, machine.Pin.OUT) 
 while True:
  blink(on_board_led) # Main logics here.  

# main() will be invoked after this script is uploaded.
main()

We upload the file with just few lines of code below.

with open('script_to_deploy.py') as f:
 script = f.read() 

messages['test upload script'] = {'type': 'script',
      'script': script}

client.request(remote_node, message)

Summary� 

In this mini system, the network of things has highly elasticity. Functionalities and behavior of nodes can be continuously adjusted if necessary.
Also, unlike some cluster system, in this system a node can communicate with other nodes, ask them to execute a function and return value, without client's intervention, thus M2M communication is feasible.

Reference

ESP8266 WiKi
NodeMCU WiKi
A python proxy in less than 100 lines of code
Boards Running MicroPython Asynchronous Socket Programming
ESP8266 first project: home automation with relays, switches, PWM, and an ADC