A Mini Project

Outline

About everything that has been learned about programming the RPI shall be incorporated into a mini project. A high level description of what shall be achieved is outlined in this figure:

Requirements

The monitoring system shall use at least one Raspberry Pi to monitor the surroundings of some environment such as:

• a room
• the parking area in front of a building
• a birds nest
• cars on the street
• entrance of building; detecting visitors coming in and out
• …

The monitoring shall be done by a Raspberry Pi (RPI) to which sensors are attached. The type of sensors obviously depends on the specific monitoring task. But let us assume that at least a camera is used, possibly a motion sensor (PIR sensor). Perhaps a temperature / humidity sensor could be added later on; and so on …

A program running on the RPI manages those sensors more or less autonomously. It takes images, detects motion either by comparing differences between images or via a PIR motion sensor. If implemented, a PIR sensor could trigger to take additional images and turn on infrared lighting to take images unobtrusively…

A PC is connected via a WLAN-router with the RPI. The PC shall be able to:

• login into the RPI
• start / stop some programs on the RPI
• enable or disable the RPI to send notifications to the PC (eg: new image available, temperature measurement updated, motion detected, lighting conditions changed, …)
• download images and / or measurement data to the PC

The PC’s main purpose is to manage the Raspberry Pi (RPi). Tasks that require a greater computational load, than could be handled on the RPI, are processed on the PC. In this respect the RPi serves as sensor node to collect data.

Some hints what be reasonably processed on the PC rather than the RPI are given here:

• postprocessing of images which had been downloaded from the RPI
  • with the increased processing power of the PC more complex image processing tasks are feasible (eg: object detection, comparing images, extracting features/objects)
• logging of monitoring activities in a logfile / database
  • in the simplest case events could be stored in a local logfile
  • a more versatile approach would store selected events in a database. This has a couple of benefits:
    • it would allow to do queries and analyse event patterns.
    • it would be possible to access the database from a remote computer over the internet.
• using a message broker to publish events to a queue from where other (possibly remote) applications could retrieve messages by subscribing to specific events.
• A browser-based interface running directly on the PC would allow to display logged events, display images and even control the RPI.

With a WLAN adapter it should be possible to:

• access the monitoring system via browser-based interface on a PC in the local network
• access the monitoring system again running a browser on a PC over the internet

The next sections provide some more details which tasks must be solved to implement such a monitoring system and how such tasks could be implemented.

The ultimate goal shall be to implement a demonstration system and document it.

---

Identifying Tasks and where they should be implemented

A figure shall provide an overview of the required tasks and where they should run.

Tasks on the RPI

The program on the RPI shall be a Python program which

• captures images
• queries various other sensors
• sends notifications to the PC if new data (images, measurement values, events) become available.

It will be the responsibility of programs on the PC to react to these notifications.

From earlier experiments it had been found, that such a program should use two processes:

• The first process is entirely dedicated to capturing images and does some image processing such as:
  • comparing successively capture images to detect differences (motion detection)
  • storing images on the local file system
• The second process collects sensor data, processes these data and sends notifications to the PC.

other requirements

• Once the RPI has booted, the program shall be started from the PC. It should be possible to pass some configuration data from the PC to the RPI. These data shall configure the program on the RPI.
• Stopping / terminating the program shall be feasible.
• Starting and stopping the program from the PC requires to login to the RPI via SSH.
• Passing configuration data from the PC to the RPI will have to use some secure file transfer (SFTP or SCP).
• Exchanging notifications between RPI and the PC shall use websockets.

---

Tasks on the PC

On the PC several tasks shall be distinguished. To avoid some sort of monolithic program tasks shall be implemented by dedicated programs.

Task 1:

purpose: starting, configuring and stopping a program running remotely on RPI

The task shall be implemented by a separate program. To ensure portability accross various operating systems it shall be implemented as a Python program. The task shall either start or stop the program on the RPI. It shall be possible to pass a configuration file when starting the program on the RPI. Controlling the RPI shall use methods such as SSH and SFTP for executing commands on the RPI and transferring configuration data as a file to the RPI.

After having started or stopped a program on the RPI, this program shall disconnect from the PC. There shall be no need to keep the SSH connection between PC / RPI alive after a program on the RPI has been started or stopped. (started programs are then still running as background processes)

Task 2

purpose: use a message broker / database

If not already running a message broker and a database shall be started on the PC. MQTT is the preferred message format for messages. For the database any relational database should be feasible although my preference is PostgreSQL.

Message broker

It shall be possible to subscribe to the message broker from a webbrowser. To accomplish this, the message broker – and clients subscribing to it – shall support websockets for communication.

In summary we must accomplish these things:

• find a message broker supporting the MQTT protocol with websockets as a transport mechanism
• find libraries for publishing / subscribing with websocket support. In the case of a webbrowser this implies finding a Javascript library.

Database

A database could be conveniently used for persistent storage. The same functionality would be possible with a log-file. If however we want to filter the dataset or apply queries, using a database would be good choice.

When selecting a library for working with a database we should select a library which supports asynchronous access to avoid blocking other applications which are may be running concurrently.

Task 3

purpose: reacting upon notifications received from RPI

Clearly the range of actions depends on type of notifications provided by the RPI. Here are some examples.

• images can be analysed for pattern; examples:
  • images can be compared to other images which have been captured before; significant differences may indicate that something has moved
  • pattern can be detected allowing to count objects
• images are stored locally on the PC
• messages can be logged and / or published to the message broker
• other measurement data can be stored, logged or published; some kind of preprocessing may be needed depending on what has been measured. Persistent storage would use either logfiles or updates to tables of a database

Basically task 3 is all about:

• preprocessing data from the RPI
• storing data in a way to make it available to other application
  • publishing data (mostly numeric) to message broker; other local / remote application may subscribe
  • putting data into a database for persistent storage
  • storing images into a dedicated directory; from where they can be retrieved by other local/remote applications

Task 4

purpose: define and implement a web application

A server application on the PC shall allow clients running a webbrowser to monitor the data provided by task 3.

We can think of several use cases:

• the user runs a browser locally on the PC (via localhost) to connect to the web-application
• one or more users connect to the web-application over a local network via (W)LAN
• users connect via the internet from a remote PC

To make this work, the PC running task 4 must serve a web-application based on HTML, CSS, Javascript via an internal web-server. Thus the following programs must be implemented:

• a server program; web-application
• a web-page / javascript program

Summary

Having identified the tasks which shall be implemented on the RPI and the PC the implementation details are described in separate web-pages / documents:

Task / Link	description
RPI Monitor 1	an application which starts capturing of images and provides sensor data currently only a camera has been attached to the RPI but no sensors, fake sensor data are generated. Since capturing of images and generating sensor data are run as separate processes there actually 2 programs which must be started
Task 1	starting, configuring and stopping a program running remotely on RPI
Task 2	use a message broker / persistent storage of data
Task 3	reacting upon notifications received from RPI
Task 4	a web application (server, client)
Project Summary	an overview of the what technologies have been used in the mini-project. Moreover some hints for improvements are provided.