Labels

Build a Raspberry Pi classifier: detect different Raspberry Pi models

In part one of this new tutorial series, you can learn how to use a Raspberry Pi AI Camera to detect different Raspberry Pi models. This series was created by David Plowman, an engineer at Raspberry Pi with a special interest in image processing hardware, as well as camera software and algorithms.

In this tutorial, we’ll use a Raspberry Pi AI Camera with a pre-trained model to detect different Raspberry Pi computers. You’ll be able to point your Raspberry Pi AI Camera at a Raspberry Pi 4 and see ‘Raspberry Pi 4’ displayed on the screen, or ‘Raspberry Pi Zero’ for a Raspberry Pi Zero, and so on.

Raspberry Pi AI Camera attached to a Raspberry Pi 5

This is a powerful example of how Raspberry Pi can be trained to perform real-time inference of object detection. It’s a good way to explore the Raspberry Pi AI Camera API (application programming interface) and the relationship between models and inference.

Set up your AI Camera

Connect your Raspberry Pi AI Camera to a compatible Raspberry Pi computer. We are using a Raspberry Pi 5, but the AI Camera is compatible with all Raspberry Pi models. You can read the AI Camera datasheet for more information.

The Raspberry Pi AI Camera features a Sony IMX500 image sensor that provides low-latency, high-performance AI capabilities to camera applications without needing an AI HAT+ (or AI Kit). It runs custom neural network models on the IMX500 sensor.

Raspberry Pi AI Camera contains a Sony IMX500 sensor with AI inferencing ability

Make sure your Raspberry Pi is up to date, then install the IMX500 firmware:

$ sudo apt install imx500-all

This command: 

  • Installs the /lib/firmware/imx500_loader.fpk and /lib/firmware/imx500_firmware.fpk firmware files required to operate the IMX500 sensor
  • Places a number of neural network model firmware files in /usr/share/imx500-models/
  • Installs the IMX500 post-processing software stages in rpicam-apps
  • Installs the Sony network model packaging tools

Now reboot your Raspberry Pi:

$ sudo reboot

Let’s test if it’s all working. The following command runs rpicam-hello with object detection post-processing:

$ rpicam-hello -t 0s --post-process-file/usr/share/rpi-camera-assets/imx500_mobilenet_ssd.json --viewfinder-width 1920 --viewfinder-height 1080 --framerate 30

The following command runs rpicam-hello with pose estimation post-processing:

$ rpicam-hello -t 0s --post-process-file/usr/share/rpi-camera-assets/imx500_posenet.json --viewfinder-width 1920 --view

Get the code

Now that your AI Camera is set up, you’ll need the code. For this tutorial, it’s best to start by copying the files from GitHub. In terminal, enter:

$ git clone https://github.com/davidplowman/pi_classifier.git

Enter the pi_classifier directory and take a look inside:

$ cd pi_classifier
$ ls

Here you will find three files:

  • demo.py — the Python code file
  • labels.txt — a text file containing the labels for different Raspberry Pi models
  • network.rpk — a ‘runtime package’ file trained to detect Raspberry Pi models using the Sony IMX500 image sensor

Run the code with Python:

$ python demo.py

A preview window will appear on the screen, displaying the view from the AI Camera. Hold a Raspberry Pi up in front of the camera, and the preview window will display the model corresponding to the options in the labels.txt file. 

How it works

Let’s open up the demo.py code in Thonny (or a code editor of your choice) and take a look at it:

$ thonny demo.py

parse_and_draw_classification_result

This function is given to the camera system as a callback function, meaning it gets called automatically every time a new frame arrives from the camera without us (the application) having to do anything extra. Because we’re using the Sony IMX500 sensor, we get not only the usual camera image but also an output tensor listing the results of running the neural network on the image.

We need to parse the neural network results using parse_classification_result, then draw them onto the camera image using draw_classification_result.

parse_classification_result

This analyses the output tensor produced by the IMX500 sensor and decides if the image belongs to one of categories that the network was trained on. There are a couple of things to watch out for.

Detecting a Raspberry Pi 3

Firstly, there isn’t always an output tensor. This can happen if the neural network accelerator didn’t quite finish processing the frame in time. When this happens, we should just reuse the result from the previous frame; another frame, where the tensor is once again available, will be along shortly!

Next, we return the category with the highest confidence value. Some networks may tell us that we should apply a softmax operation to the output values to get a more realistic probability for each class.

draw_classification_result

Here we draw a label on the camera image indicating the device category and confidence value — but only if the confidence value is sufficiently high.

We get access to the actual camera image pixels (not a copy of it) in the usual way, using Picamera2’s MappedArray. We measure the size of the box that the category label and confidence value will occupy, then darken that part of the image so that the text will stand out better when we subsequently write it on top.

Detecting a Raspberry Pi Zero

if __name__ == “__main__”

This section of the code is simply concerned with configuring the IMX500, uploading the neural network firmware file (network.rpk), and getting the camera running. There are just a few things to point out:

  1. We first create the IMX500 camera helper object. This will upload the network for us, and also tells us which camera number the IMX500 is.
  2. The NetworkIntrinsics object is a helper for the network that will recommend an appropriate frame rate we can use, which should match the inference rate of the network on the IMX500.
  3. Before starting the camera, we should tell the IMX500 helper that we would like to see a progress bar for the network upload — otherwise it’s easy to think it might have got stuck! Recall that the network firmware uploads more quickly on subsequent occasions, as more of the firmware blocks are already cached within the device.
  4. Finally, we let the script run forever, and all the camera processing happens in background threads.

Download the full demo.py code for the Raspberry Pi Classifier.

The post Build a Raspberry Pi classifier: detect different Raspberry Pi models appeared first on Raspberry Pi.



from News - Raspberry Pi https://ift.tt/lwrh8CX
Labels:

Comments

Post a Comment

Popular Posts