SoMLabs-NN-demo

SoMLabs NN demo for SpaceSOM-8Mplus

---

This tutorial describes the example usage of the Neural Processing Unit in the iMX 8M Plus processor. It requires the SpaceSOM-8Mplus module with SpaceCB-8Mplus-ADV carrier board, SL-MIPI-CSI-OV5640 camera board and connected display (TODO - DSI/HDMI).

The example application uses a pre-built TensorFlow Lite model for image classification. The model is available on the TensorFlow github:

http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz

The model is executed on hardware Neural Processing Unit using the libvx_delegate.so library provided by NXP.

Hardware setup

The camera board should be connected to the MIPI-CSI1 socket of the carrier board.

Software setup

This demo was built with the Yocto Honister default image for SpaceSOM-8Mplus and requires changes that add required packages and demo application. All the changes are collected in the provided somlabs_nn_demo.patch that should be applied to the meta-somlabs repository:

cd imy-yocto-bsp/source/meta-somlabs
git apply somlabs_nn_demo.patch

After that the image should be build according to the Yocto Honister building tutorial. During the image compilation some errors may occur according to the i.MX Yocto Project User's Guide:

cc1plus: error: include location "/usr/include/CL" is unsafe for crosscompilation [-Werror=poison-system-directories]

To get around this error, add the following line in the tim-vx recipe:

EXTRA_OECMAKE += " \
-DCMAKE_SYSROOT=${PKG_CONFIG_SYSROOT_DIR} \
"

Demo application

The application reads a frame from the connected camera and after a simple processing passes it to the neural network model for classification. The processing includes: - cropping the sides of the frame in order to obtain square image from the center field of view of the camera - scaling the image to get size required by the model and to fit it in the display area

After the image is classified by the model the results are displayed in the application window. Each result with the classification score above the defined threshold is shown. The following video shows some objects recognized by the application.