Building "plug-and-play" Internet-of-Things sensors with Raspberry Pi

I have some leftovers of an earlier generation of sensors from the InSPECT project, and want to get them streaming their data over MQTT (as part of my general IoT-in-a-Box project).

Self-identifying sensor/actuator devices

InSPECT has had two main generations of sensors. This first generation connected over USB to a Raspberry Pi "hub", which sampled the data and sent it to over websockets to a server. (The newer, current set connects by audio cable to an ESP-32, which sends the data as MQTT messages.)

Since I wanted to use a Raspberry Pi (and its camera) for my IoT-in-a-Box project, I want to use this first set of sensors:

The sensors each already have code running on an attached microcontroller (A-Star Pololus for this first generation, and Arduino Pro Minis for the second). The Pololus then plug into the Pi's USB ports, where they communicate the sensor data to software running on the Pi.

The Arduino code allows each sensor to send information about itself to the hub, like what type of sensor it is or what its units are. The microcontrollers standardize communications with the sensors, so that they can "plug and play" into the Raspberry Pi. (In the first version of the Dataflow software, sensor icons would appear and disappear as you plugged in or unplugged devices.)

Code for various commercial sensors can be found here:

rhizolab/rhizo-devices

This code can be used to create self-identifying sensor/actuator devices that work with the rhizo controller/client software. - rhizolab/rhizo-devices

GitHubrhizolab

The Python Code

For my project, I should be able to use this sensor code as is, and only change the Python code that runs on the Raspberry Pi and interacts with these devices. The existing library is here:

rhizolab/rhizo

This is the client library for the rhizo-server system. - rhizolab/rhizo

GitHubrhizolab

Most of that code handles things that I won't need, like: websockets to the dataflow/rhizo server, authentication, running programs using the sensor data, etc. I also admittedly struggled to figure out how and where to integrate what I wanted (MQTT messages, camera, possible GPIO) into the existing code. Since this is specifically a learning project, and I want to understand the thing I'm making, I decided to re-write the client to do only the set of things I wanted. That code is here:

lahardy/IoT-in-a-Box

Contribute to lahardy/IoT-in-a-Box development by creating an account on GitHub.

GitHublahardy

How it works

The sensors and relays are one part of the IoT-in-a-Box project, which includes the camera and a webserver for seeing the camera feed and serving a control page. In the future, it might include sensors attached to GPIO also.

Overall, the code takes the connected hardware (e.g. camera, relays, sensors) and turns them each into something like "IoT Things": devices that broadcast MQTT messages about their current state, and that listen for MQTT messages telling them what to do. It does this by creating an "IoT Interface" for each available hardware component:

When the application starts, the following things happen:

1. The USBPortWatcher starts watching the USB ports for connections or disconnections, and broadcasts those events when they happen. An internal EventsManager manages all internal subscriptions to and publications of these different types of events.
2. The DeviceManager subscribes to those connection/disconnection events, and creates or destroys Devices when they happen. Each Device manages a SerialConnection and updates the metadata and state of one or more Components (which represent e.g. a sensor or relay). When a Device adds or removes a Component, it broadcasts this as a component-added or component-removed event. The Components broadcast their own events when their state changes (if they have a new value, or if other new information comes in about them).
3. The IoTInterfaceManager listens for components being added or removed, and creates or destroys an IoTInterface for each component. An IoTInterface listens for updates from a particular component, and sends them to the MQTTServiceProvider to be broadcast. It also subscribes to external command messages addressed to that component, and sends them inward for the component to handle.

When it runs, it will start sending out data from any attached sensors, at a rate of once per second:

Listening to sensor data messages

To continuously listen to a particular sensor's updates, you can use a separate Python script to connect to the MQTT server, subscribe to the sensor's update messages, and loop forever:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import paho.mqtt.client as mqtt
import json

#set up MQTT client
clientName = "data-listener"
serverAddress = "10.0.0.150"
MQTT_TOPIC = "/dev/ttyACM0/c/update"

mqttClient = mqtt.Client(clientName)

def on_connect(client, userdata, flags, rc):
    print("Connected to MQTT Server!")
    mqttClient.subscribe(MQTT_TOPIC)
    print("Subscribed to topic", MQTT_TOPIC)

def on_message(client, userdata, msg):
    payload_str = msg.payload.decode('utf-8')
    payload = json.loads(payload_str)
    print("Message Received: topic: %s, message: %s" % (msg.topic, payload))

mqttClient.on_connect = on_connect
mqttClient.on_message = on_message

#start MQTT client
mqttClient.connect(serverAddress)
mqttClient.loop_forever()

Then in the terminal, you'll see a stream of data from that sensor:

Controlling relays with MQTT messages

Here is a script for publishing a message to tell a particular relay to turn on. The script connects to the MQTT server, sends a single message, and disconnects:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import paho.mqtt.client as mqtt
import json
from time import sleep

#set up MQTT client
clientName = "mqtt-script"
serverAddress = "10.0.0.150"
relayID = "/dev/ttyACM1/r"

def on_connect(client, userdata, flags, rc):
        print("Connected to MQTT Server!")
        msg = json.dumps({"command": "set", "params": 1})
        topic = "%s/command" % relayID
        client.publish(topic,msg)
        print("OUTGOING: TOPIC: %s, MESSAGE: %s" % (topic, msg))

def on_disconnect(client, userdata, rc):
        print("Disconnecting.")

mqttClient = mqtt.Client(clientName)
mqttClient.on_connect = on_connect
mqttClient.on_disconnect = on_disconnect

#start MQTT client
mqttClient.connect(serverAddress)
mqttClient.loop_start()
sleep(1)
mqttClient.loop_stop()
mqttClient.disconnect()

To tell it to turn the relay off, switch the command from "params":1 to "params":0. (I should probably have just  made this a command line input to the script.)

Done!

Now I have plug-and-play USB sensors streaming data over MQTT. And can control the state of a relay using MQTT messages. Woohoo!

BUT. The problem with the current scripts is that you need to know the ID of the component you want to listen to or send commands to. Instead, any client should be able to send a "roll-call/who-there" message and have all available components respond with their componentIDs and their metadata (sensor type, units, model number, etc). Currently, they respond to roll-call with their names/IDs, but their info is sent separately. I am not sure yet how I want to handle who knows about a component's metadata, and when– for example, whether the IoTInterfaces keep record of metadata, or request it of the components when someone else wants to know, etc. I will have to work this out to make a browser client that creates an interface depending on the available components.