Intro to MQTT

Pietro Manzoni
Universitat Politecnica de Valencia (UPV)
Valencia - SPAIN
pmanzoni@disca.upv.es

http://bit.ly/ictp2019-mqtt
Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 1
 Universitat Politècnica de València

¡ The Universitat Politècnica de València (UPV) is a

Spanish public educational institution founded in
1968.
¡ Its academic community comprises 36.823 students,
almost 2.661 lecturers and researchers, and 1.422
administration and services professionals.
¡ The Vera Campus covers around 840.000 m2 and is
almost 2 km long. It is a pedestrian campus with over
123.000 m2 of green areas.
¡ UPV is composed of 10 schools, 3 faculties and 2
higher polytechnic schools.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 2

 From “byte streams” to “messages”

¡ The “old” vision of data

communication was based
on reliable byte
streams, i.e., TCP
¡ Nowadays messages
interchange is becoming
more common
£ E.g., Twitter, Whatsapp,
Instagram, Snapchat,
Facebook,...
¡ Actually is not that new…
£ emails: SMTP+MIME,
£ FTP,

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 3

 Ways to interchange “messages”

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 4

 Request/response approach

¡ REST: Representational State Transfer

¡ Widely used; based on HTTP
¡ Lighter version: CoAP (Constrained Application Protocol)

Server

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 5

 Pub/sub approach

¡ Publish/Subscriber
o aka: producer/consumer

Publisher 1 Publisher 2

Broker

Subscriber 1 Subscriber 2 Subscriber 3

¡ Various protocols:
o MQTT, AMQP, XMPP (was Jabber)
¡ Growing technique
o E.g., https://cloud.google.com/iot/docs/how-tos/mqtt-bridge

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 6

 Pub/sub approach

¡ Pub/Sub separate a client, who is sending a message about a

specific topic, called publisher, from another client (or more
clients), who is receiving the message, called subscriber.
¡ There is a third component, called broker, which is known by
both the publisher and subscriber, which filters all incoming
messages and distributes them accordingly.

HiveMQ©

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 7

 An example

Source: https://zoetrope.io/tech-blog/brief-practical-introduction-mqtt-protocol-and-its-application-iot

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 8

 Intro to MQTT

¡ Fundamental concepts

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 9

 Message Queuing Telemetry Transport

¡ A lightweight publish-subscribe protocol that can run on

embedded devices and mobile platforms è http://mqtt.org/
o Low power usage.
o Binary compressed headers
o Maximum message size of 256MB
• not really designed for sending large amounts of data
• better at a high volume of low size messages.
¡ Documentation sources:
o The MQTT community wiki:
• https://github.com/mqtt/mqtt.github.io/wiki
o A very good tutorial:
• http://www.hivemq.com/mqtt-essentials/

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 10

 Some details about versions

¡ MQTT 3.1.1 is the current version of the protocol.

o Standard document here:
• http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html
o October 29th 2014: MQTT was officially approved as OASIS Standard.
• https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=mqtt

¡ MQTT v5.0 is the successor of MQTT 3.1.1

o Current status: Committee Specification 02 (15 May 2018)
• http://docs.oasis-open.org/mqtt/mqtt/v5.0/cs02/mqtt-v5.0-cs02.html
o Not backward compatible; too many new things are introduced so
existing implementations have to be revisited, for example:
• Enhancements for scalability and large scale systems in respect to setups
with 1000s and millions of devices.
• Improved error reporting (Reason Code & Reason String)
• Performance improvements and improved support for small clients
o https://www.youtube.com/watch?time_continue=3&v=YIpesv_bJgU

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 11

 MQTT works on top of…

¡ mainly of TCP
o There is also the closely related MQTT for Sensor Networks (MQTT-SN)
where TCP is replaced by UDP à TCP stack is too complex for WSN

¡ websockets can be used, too!

o Websockets allows you to receive MQTT data directly into a web browser.

¡ Both, TCP & websockets can work on top of “Transport Layer

Security (TLS)” (and its predecessor, Secure Sockets Layer (SSL))

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 12

 Publish/subscribe interactions sequence

Publisher BROKER Subscriber

connect

connect ACK

connect

connect ACK

subscribe (topic)

subscribe ACK

publish (topic, data)

publish (topic, data)

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 13

 Topics

¡ MQTT Topics are structured in a hierarchy similar to folders and files in a file
system using the forward slash ( / ) as a delimiter.
¡ Allow to create a user friendly and self descriptive naming structures

¡ Topic names are:

o Case sensitive
o use UTF-8 strings.
o Must consist of at least one character to be valid.
¡ Except for the $SYS topic there is no default or standard topic structure.

Special $SYS/ topics

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 14

 Topics wildcards

¡ Topic subscriptions can have wildcards. These enable nodes to

subscribe to groups of topics that don’t exist yet, allowing greater
flexibility in the network’s messaging structure.
o ‘+’ matches anything at a given tree level
o ‘#’ matches a whole sub-tree
¡ Examples:
o Subscribing to topic house/# covers:
ü
house/room1/main-light
ü
house/room1/alarm
ü
house/garage/main-light
ü
house/main-door
o Subscribing to topic house/+/main-light covers:
ü house/room1/main-light
ü house/room2/main-light
ü house/garage/main-light
o but doesn’t cover
ü house/room1/side-light
ü house/room2/side-light

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 15

 Quality of Service (QoS)

¡ Messages are published with a Quality of Service (QoS) level, which specifies
delivery requirements.
¡ A QoS 0 (“at most once”) message is fire-and-forget.
o For example, a notification from a doorbell may only matter when
immediately delivered.
¡ With QoS 1 (“at least once”), the broker stores messages on disk and retries
until clients have acknowledged their delivery.
o (Possibly with duplicates.) It’s usually worth ensuring error messages are
delivered, even with a delay.
¡ QoS 2 (“exactly once”) messages have a second acknowledgement round-
trip, to ensure that non-idempotent messages can be delivered exactly
once.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 16

 Retained Messages!!!

¡ A retained message is a normal MQTT message with the

retained flag set to true. The broker will store the last retained
message and the corresponding QoS for that topic
o Each client that subscribes to a topic pattern, which matches the topic of
the retained message, will receive the message immediately after
subscribing.
o For each topic only one retained message will be stored by the broker.

¡ Retained messages can help newly subscribed clients to get a

status update immediately after subscribing to a topic and
don’t have to wait until a publishing clients send the next
update.
o In other words a retained message on a topic is the last known good
value, because it doesn’t have to be the last value, but it certainly is the
last message with the retained flag set to true.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 17

 Intro to MQTT

¡ Brokers and clients

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 18

 Creating a broker

BROKER

19
Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019
 Available MQTT brokers

¡ The most widely used are:

o http://mosquitto.org/
• man page: https://mosquitto.org/man/mosquitto-8.html
o http://www.hivemq.com/
• The standard trial version only supports 25 connections.

¡ And also:
o https://www.rabbitmq.com/mqtt.html
o http://activemq.apache.org/mqtt.html

¡ A quite complete list can be found here:

o https://github.com/mqtt/mqtt.github.io/wiki/servers

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 20

 Installing Mosquitto on a Raspberry Pi

¡ It takes only a few seconds to install a Mosquitto broker on a

Raspberry. You need to execute the following steps:
sudo apt-get update
sudo apt-get install mosquitto mosquitto-clients

¡ Installation guidelines with websockets

https://gist.github.com/smoofit/dafa493aec8d41ea057370dbfde3f3fc

¡ Managing the broker:

o To start and stop its execution use:
sudo /etc/init.d/mosquitto start/stop
o Verbose mode:
sudo mosquitto –v

o To check if the broker is running you can use the command:

sudo netstat -tanlp | grep 1883
• note: "-tanlp" stands for: tcp, all, numeric, listening, program

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 21

 Cloud based MQTT brokers: CloudMQTT

https://www.cloudmqtt.com/ è based on Mosquitto

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 22

 Cloud based brokers: flespi

https://flespi.com/mqtt-broker

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 23

 Cloud based brokers: flespi

https://flespi.io/#/panel/mqttboard

https://flespi.com/mqtt-api

I1RKMMIUJppLdlQoSgAQ8MvJPyNV9R2HIJgijO1S1gt5rajaeIOaiaKWwlHt2z1z

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 24

 Open brokers (“Sandboxes”)

¡ TCP based:
o https://iot.eclipse.org/getting-started/#sandboxes
• Hostname: iot.eclipse.org
o http://test.mosquitto.org/
• Hostname: test.mosquitto.org
o https://www.hivemq.com/mqtt-demo/
• Hostname: broker.hivemq.com
• http://www.mqtt-dashboard.com/
o Ports:
• standard: 1883
• encrypted: 8883 (TLS v1.2, v1.1 or v1.0 with x509 certificates)

¡ Websockets based:
o broker.mqttdashboard.com port: 8000
o test.mosquitto.org port: 8080
o broker.hivemq.com port: 8000

¡ https://github.com/mqtt/mqtt.github.io/wiki/public_brokers

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 25

 Creating clients

BROKER

26
Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019
 Clients for testing

¡ The Mosquitto broker comes with a couple of useful commands

to quickly publish and subscribe to some topic.

¡ Their basic syntax is the following.

o mosquitto_sub -h HOSTNAME -t TOPIC
o mosquitto_pub -h HOSTNAME -t TOPIC -m MSG

¡ More information can be found:

o https://mosquitto.org/man/mosquitto_sub-1.html
o https://mosquitto.org/man/mosquitto_pub-1.html

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 27

 MQTT clients: iOS

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 28

 MQTT clients: Android

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 29

 MQTT websocket clients

http://test.mosquitto.org/ws.html http://mitsuruog.github.io/what-mqtt/

http://www.hivemq.com/demos/websocket-client/
Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 30
 Intro to MQTT

¡ Time for some exercise: Lab 0

https://bit.ly/ictp2019-lab0

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 31

 Intro to MQTT

¡ More time for some demo/exercise: Lab 0.1

Broker address:

192.168.XX.XX
port: 9001

https://www.raspberrypi.org/products/sense-hat/

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 32

 Intro to MQTT

¡ Clients in Python

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 33

 MQTT clients: Python vs Micropython

¡ The MQTT available versions for Python and MicroPython are

slightly different.
¡ MicroPython is intended for constrained environments, in
particular, microcontrollers, which have orders of magnitude
less performance and memory than "desktop" systems on
which Python3
¡ Basically remember that, when using the LoPy you have to use
the MicroPython version of MQTT
¡ In the following we will see information about both cases.

vs.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 34

 Eclipse Paho Python

¡ Eclipse Paho Python (originally the mosquitto Python client)

o http://www.eclipse.org/paho/
¡ Documentation: https://pypi.org/project/paho-mqtt/
o or: http://www.eclipse.org/paho/clients/python/docs/
¡ Source: https://github.com/eclipse/paho.mqtt.python

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 35

 Paho MQTT Python client: general usage flow

The general usage flow is as follows:

¡ Create a client instance
¡ Connect to a broker using one of the connect*() functions
¡ Call one of the loop*() functions to maintain network traffic
flow with the broker
¡ Use subscribe() to subscribe to a topic and receive messages
¡ Use publish() to publish messages to the broker
¡ Use disconnect() to disconnect from the broker

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 36

 Example 1: a simple subscriber
# File: sisub.py

import paho.mqtt.client as mqtt

THE_BROKER = "iot.eclipse.org"
THE_TOPIC = "$SYS/#"
CLIENT_ID = ""

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
print("Connected to ", client._host, "port: ", client._port)
print("Flags: ", flags, "returned code: ", rc)
client.subscribe(THE_TOPIC, qos=0)

# The callback for when a message is received from the server.

def on_message(client, userdata, msg):
print("sisub: msg received with topic: {} and payload: {}".format(msg.topic, str(msg.payload)))

client = mqtt.Client(client_id=CLIENT_ID,
clean_session=True,
userdata=None,
protocol=mqtt.MQTTv311,
transport="tcp")

client.on_connect = on_connect
client.on_message = on_message

client.username_pw_set(None, password=None)
client.connect(THE_BROKER, port=1883, keepalive=60)

# Blocking call that processes network traffic, dispatches callbacks and handles reconnecting.
client.loop_forever()
More on this later
37
Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019
 Example 1: output
Connected to iot.eclipse.org port: 1883
Flags: {'session present': 0} returned code: 0
sisub: msg received with topic: $SYS/broker/version and payload: b'mosquitto version 1.4.15'
sisub: msg received with topic: $SYS/broker/timestamp and payload: b'2018-04-11 '
sisub: msg received with topic: $SYS/broker/clients/total and payload: b'162523'
sisub: msg received with topic: $SYS/broker/clients/active and payload: b'4103'
sisub: msg received with topic: $SYS/broker/clients/inactive and payload: b'158420'
sisub: msg received with topic: $SYS/broker/clients/maximum and payload: b'162524'
sisub: msg received with topic: $SYS/broker/clients/disconnected and payload: b'158420'
sisub: msg received with topic: $SYS/broker/clients/connected and payload: b'4103'
sisub: msg received with topic: $SYS/broker/clients/expired and payload: b'0'
sisub: msg received with topic: $SYS/broker/messages/received and payload: b'1171291305'
sisub: msg received with topic: $SYS/broker/messages/sent and payload: b'6271921352'
sisub: msg received with topic: $SYS/broker/messages/stored and payload: b'1380714’ …

38
Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019
 Paho MQTT Python client: connect

¡ connect(host, port=1883, keepalive=60, bind_address=””)

¡ The broker acknowledgement will generate a callback

(on_connect).
¡ Return Codes:
o 0: Connection successful
o 1: Connection refused – incorrect protocol version
o 2: Connection refused – invalid client identifier
o 3: Connection refused – server unavailable
o 4: Connection refused – bad username or password
o 5: Connection refused – not authorised
o 6-255: Currently unused.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 39

 Paho MQTT Python client: pub/sub

subscribe(topic, qos=0)
o e.g., subscribe("my/topic", 2)
o E.g., subscribe([("my/topic", 0), ("another/topic", 2)])

o on_message(client, userdata, message) Called when a message has been

received on a topic that the client subscribes to.

publish(topic, payload=None, qos=0, retain=False)

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 40

 Paho MQTT Python client: Network loop

What are network loops for?

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 41

 Paho MQTT Python client: Network loop

loop_forever()
¡ This is a blocking form of the network loop and will not return until
the client calls disconnect(). It automatically handles reconnecting.

loop_start() / loop_stop()
¡ These functions implement a threaded interface to the network loop.
o Calling loop_start() once, before or after connect(), runs a thread in the
background to call loop() automatically. This frees up the main thread for other
work that may be blocking.
o Call loop_stop() to stop the background thread.

loop(timeout=1.0)
¡ Call regularly to process network events.
o This call waits in select() until the network socket is available for reading or
writing, if appropriate, then handles the incoming/outgoing data.
o This function blocks for up to timeout seconds.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 42

 Example 2: subscriber with loop_start/loop_stop
import sys
import time

import paho.mqtt.client as mqtt

THE_BROKER = "test.mosquitto.org"
THE_TOPIC = "$SYS/broker/load/bytes/#"

def on_connect(mqttc, obj, flags, rc):

print("Connected to ", mqttc._host, "port: ", mqttc._port)
mqttc.subscribe(THE_TOPIC, 0)

def on_message(mqttc, obj, msg):

global msg_counter
print(msg.topic+" "+str(msg.qos)+" "+str(msg.payload))
msg_counter+=1

def on_subscribe(mqttc, obj, mid, granted_qos):

print("Subscribed: ", mid, "granted QoS: ", granted_qos)

mqttc = mqtt.Client()
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe

mqttc.connect(THE_BROKER, keepalive=60)

msg_counter = 0
mqttc.loop_start()
while msg_counter < 10:
time.sleep(0.1)
mqttc.loop_stop()
print msg_counter
43
Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019
 Example 3: very basic periodic producer

import random
import time

import paho.mqtt.client as mqtt

client = mqtt.Client(client_id=CLIENT_ID,
clean_session=True,
THE_BROKER = "test.mosquitto.org"
userdata=None,
THE_TOPIC = "PMtest/rndvalue"
CLIENT_ID = "" protocol=mqtt.MQTTv311,
transport="tcp")
# The callback for when the client receives a CONNACK response
from the server. client.on_connect = on_connect
def on_connect(client, userdata, flags, rc):
client.on_publish = on_publish
print("Connected to ", client._host, "port: ", client._port)
print("Flags: ", flags, "returned code: ", rc)
client.username_pw_set(None, password=None)
# The callback for when a message is published. client.connect(THE_BROKER, port=1883, keepalive=60)
def on_publish(client, userdata, mid):
print("sipub: msg published (mid={})".format(mid)) client.loop_start()

while True:

msg_to_be_sent = random.randint(0, 100)

client.publish(THE_TOPIC,
payload=msg_to_be_sent,
qos=0,
retain=False)

time.sleep(5)

Generates a new data every 5 secs client.loop_stop()

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 44

 Example 3: output

Python 3.6.1 (default, Dec 2015, 13:05:11)

[GCC 4.8.2] on linux
sipub: msg published (mid=1) Producer output
Connected to test.mosquitto.org port: 1883
Flags: {'session present': 0} returned code: 0
sipub: msg published (mid=2)
sipub: msg published (mid=3)
sipub: msg published (mid=4)
sipub: msg published (mid=5)
sipub: msg published (mid=6)
sipub: msg published (mid=7)

Python 3.6.1 (default, Dec 2015, 13:05:11)

[GCC 4.8.2] on linux
Connected to test.mosquitto.org port: 1883
Flags: {'session present': 0} returned code: 0
sisub: msg received with topic: PMtest/rndvalue and payload: b'11'
sisub: msg received with topic: PMtest/rndvalue and payload: b'14'
sisub: msg received with topic: PMtest/rndvalue and payload: b'31'
Output obtained with a sisub: msg received with topic: PMtest/rndvalue and payload: b'27'
modified version of Example1.
Which parts of that code
sisub: msg received with topic: PMtest/rndvalue and payload: b'60'
had to be modified?
sisub: msg received with topic: PMtest/rndvalue and payload: b'70'
sisub: msg received with topic: PMtest/rndvalue and payload: b'60'
sisub: msg received with topic: PMtest/rndvalue and payload: b'66'
sisub: msg received with topic: PMtest/rndvalue and payload: b'45'
sisub: msg received with topic: PMtest/rndvalue and payload: b'56'
sisub: msg received with topic: PMtest/rndvalue and payload: b'37’
…

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 45

 Example 4: Pub/Sub with JSON
…
Consumer
Producer # The callback for when a PUBLISH message is received
from the server.
def on_message(client, userdata, msg):
print(msg.topic+" "+str(msg.payload))
…
themsg = json.loads(str(msg.payload))
mqttc.loop_start()
print("Sensor "+str(themsg['Sensor'])+" got value "+
while True: str(themsg['Value'])+" "+themsg['C_F']+
# Getting the data " at time "+str(themsg['Time']))
the_time = time.strftime("%H:%M:%S")
the_value = random.randint(1,100) …
the_msg={'Sensor': 1, 'C_F': 'C',
'Value': the_value, 'Time': the_time}

the_msg_str = json.dumps(the_msg)

mqttc.publish(THE_TOPIC, the_msg_str)
time.sleep(5)

mqttc.loop_stop() paho-code:pietro$ python example4-cons.py

Connected with result code 0
PMtest/jsonvalue {"Time": "12:19:30", "Sensor": 1, "Value": 33, "C_F": "C"}
Sensor 1 got value 33 C at time 12:19:30
PMtest/jsonvalue {"Time": "12:19:35", "Sensor": 1, "Value": 11, "C_F": "C"}
Sensor 1 got value 11 C at time 12:19:35

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 46

 MQTT with MicroPython

¡ Import the library

from mqtt import MQTTClient

¡ Creating a client:
MQTTclient(client_id, server, port=0, user=None, password=None,
keepalive=0, ssl=False, ssl_params={})
e.g., client = MQTTClient("dev_id", "10.1.1.101", 1883)

¡ The various calls:

• connect(clean_session=True):
• publish(topic, msg, retain=False, qos=0):
• subscribe(topic, qos=0):
• set_callback(self, f):

¡ wait_msg():
o Wait for a single incoming MQTT message and process it. Subscribed messages are
delivered to a callback previously set by .set_callback() method. Other (internal) MQTT
messages processed internally.
¡ check_msg():
o Checks whether a pending message from server is available. If not, returns
immediately with None. Otherwise, does the same processing as wait_msg.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 47

 MicroPython: a simple publisher

# file: mp_sipub.py

from mqtt import MQTTClient

import pycom ### if __name__ == "__main__":
import sys
import time ufun.connect_to_wifi(wifi_ssid, wifi_passwd)

import ufun client = MQTTClient(CLIENT_ID, THE_BROKER, 1883)

wifi_ssid = 'THE_NAME_OF_THE_AP' print ("Connecting to broker: " + THE_BROKER)

wifi_passwd = '' try:
client.connect()
THE_BROKER = "iot.eclipse.org" except OSError:
THE_TOPIC = "test/SRM2018" print ("Cannot connect to broker: " + THE_BROKER)
CLIENT_ID = "" sys.exit()
print ("Connected to broker: " + THE_BROKER)
def settimeout(duration):
pass print('Sending messages...')
while True:
def get_data_from_sensor(sensor_id="RAND"): # creating the data
if sensor_id == "RAND": the_data = get_data_from_sensor()
return ufun.random_in_range() # publishing the data
client.publish(THE_TOPIC, str(the_data))
print("Published message with value: {}".format(the_data))
time.sleep(1)

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 48

 MicroPython: a simple subscriber

# file: mp_sisub.py

from mqtt import MQTTClient

import pycom ### if __name__ == "__main__":
import sys
import time ufun.connect_to_wifi(wifi_ssid, wifi_passwd)

import ufun client = MQTTClient(CLIENT_ID, THE_BROKER, 1883)

client.set_callback(on_message)
wifi_ssid = 'THE_NAME_OF_THE_AP'
wifi_passwd = '' print ("Connecting to broker: " + THE_BROKER)
try:
THE_BROKER = "iot.eclipse.org" client.connect()
THE_TOPIC = "test/SRM2018" except OSError:
CLIENT_ID = "" print ("Cannot connect to broker: " + THE_BROKER)
sys.exit()
def settimeout(duration): print ("Connected to broker: " + THE_BROKER)
pass
client.subscribe(THE_TOPIC)
def on_message(topic, msg):
print("Received msg: ", str(msg), print('Waiting messages...')
"with topic: ", str(topic)) while 1:
client.check_msg()

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 49

 Intro to MQTT

¡ Some final details

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 50

 MQTT Keep alive

¡ The keep alive functionality assures that the connection is still

open and both broker and client are connected to one another.
¡ The client specifies a time interval in seconds and
communicates it to the broker during the establishment of the
connection.
o The interval is the longest possible period of time which broker and client
can endure without sending a message.
o If the broker doesn’t receive a PINGREQ or any other packet from a
particular client, it will close the connection and send out the last will and
testament message (if the client had specified one).
¡ Good to Know
o The MQTT client is responsible of setting the right keep alive value.
o The maximum keep alive is 18h 12min 15 sec.
o If the keep alive interval is set to 0, the keep alive mechanism is
deactivated.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 51

 “Will” message

¡ When clients connect, they can specify an optional “will”

message, to be delivered if they are unexpectedly disconnected
from the network.
o (In the absence of other activity, a 2-byte ping message is sent to clients
at a configurable interval.)
¡ This “last will and testament” can be used to notify other parts
of the system that a node has gone down.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 52

 Persistent session

¡ A persistent session saves all information relevant for the client on the
broker. The session is identified by the clientId provided by the client
on connection establishment
¡ So what will be stored in the session?
o Existence of a session, even if there are no subscriptions
o All subscriptions
o All messages in a Quality of Service (QoS) 1 or 2 flow, which are not confirmed by the
client
o All new QoS 1 or 2 messages, which the client missed while it was offline
o All received QoS 2 messages, which are not yet confirmed to the client
o That means even if the client is offline all the above will be stored by the broker and
are available right after the client reconnects.
¡ Persistent session on the client side
o Similar to the broker, each MQTT client must store a persistent session too. So when
a client requests the server to hold session data, it also has the responsibility to hold
some information by itself:
o All messages in a QoS 1 or 2 flow, which are not confirmed by the broker
o All received QoS 2 messages, which are not yet confirmed to the broker

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 53

 Topics best practices

¡ First of all:
o Don’t use a leading forward slash
o Don’t use spaces in a topic
o Use only ASCII characters, avoid non printable characters
¡ Then, try to..
o Keep the topic short and concise
o Use specific topics, instead of general ones
o Don’t forget extensibility
¡ Finally, be careful and don’t subscribe to #

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 54

 A few words on security

¡ MQTT has the option for Transport Layer Security

(TLS) encryption.
¡ MQTT also provides username/password
authentication with the broker.
o Note that the password is transmitted in clear text. Thus, be sure to use
TLS encryption if you are using authentication.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 55

 A few words on security

Smart homes can be easily hacked via

unsecured MQTT servers
https://www.helpnetsecurity.com/2018/08/20/unsecured-mqtt-servers/

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 56

 MQTT vs REST

¡ Can they really be compared?!?!?

o MQTT was created basically as a lightweight messaging protocol for
lightweight communication between devices and computer systems
o REST stands on the shoulders of the almighty HTTP
¡ So it’s better to understand their weak and strong points and
build a system taking the best of both worlds… if required

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 57

 REST advantages

¡ It is always independent of the type of platform or languages

o The only thing is that it is indispensable that the responses to the requests
should always take place in the language used for the information
exchange, normally XML or JSON.

¡ It is stateless è This allows for scalability, by adding additional

server nodes behind a load balancer
o No state can be stored on servers: “keep the application state on the
client.”
o All messages exchanged between client and server have all the context
needed to know what to do with the message.

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 58

 REST disadvantages

¡ Today’s real world embedded devices for IoT usually lacks the
ability to handle high-level protocols like HTTP and they may
be served better by lightweight binary protocols.
¡ It is PULL based. This poses a problem when services depend
on being up to date with data they don’t own and manage.
o Being up to date requires polling, which quickly add up in a system with
enough interconnected services.
o Pull style can produce heavy unnecessary workloads and bandwidth
consumption due to for example a request/response polling-based
monitoring & control systems
¡ It is based on one-to-one interactions

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 59

 Advantages of MQTT

¡ Push based: no need to continuously look for updates

¡ It has built-in function useful for reliable behavior in an

unreliable or intermittently connected wireless environments.
1. “last will & testament” so all apps know immediately if a client
disconnects ungracefully,
2. “retained message” so any user re-connecting immediately gets the very
latest information, etc.

¡ Useful for one-to-many, many-to-many applications

¡ Small memory footprint protocol, with reduced use of battery

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 60

 Energy usage: some number

amount of power taken to establish the initial connection to the server: cost of ‘maintaining’ that connection (in % Battery / Hour):

3G – 240s Keep Alive – % Battery Used Creating and Maintaining a Connection

you’d save ~4.1% battery per day just

by using MQTT over HTTPS to
maintain an open stable connection.

http://stephendnicholas.com/posts/power-profiling-mqtt-vs-https

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 61

 MQTT disadvantages

¡ If the broker fails…

¡ Does not define a standard client API, so application developers
have to select the best fit.
¡ Does not include many features that are common in Enterprise
Messaging Systems like:
o expiration, timestamp, priority, custom message headers, …
¡ Does not have a point-to-point (aka queues) messaging pattern
o Point to Point or One to One means that there can be more than one
consumer listening on a queue but only one of them will be get the
message
¡ Maximum message size 256MB

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019 62

Workshop on Rapid Prototyping of IoT for Science (smr3268) – January 2019