SoMLabs-NN-demo - Revision history

KrzysztofChojnowski at 16:33, 23 June 2022

2022-06-23T16:33:59Z

← Older revision		Revision as of 16:33, 23 June 2022
Line 1:		Line 1:
	{{PageHeader\|SoMLabs NN demo for SpaceSOM-8Mplus}}		{{PageHeader\|SoMLabs NN demo for SpaceSOM-8Mplus}}



			{{#evu:https://www.youtube.com/watch?v=xx63q5c9XgI
			\|alignment=center
			}}



	This tutorial describes the example usage of the Neural Processing Unit in the iMX 8M Plus processor. It requires the SpaceSOM-8Mplus module with SpaceCB-8Mplus-ADV carrier board, SL-MIPI-CSI-OV5640 camera board and SL-TFT7-TP-720-1280-MIPI display.		This tutorial describes the example usage of the Neural Processing Unit in the iMX 8M Plus processor. It requires the SpaceSOM-8Mplus module with SpaceCB-8Mplus-ADV carrier board, SL-MIPI-CSI-OV5640 camera board and SL-TFT7-TP-720-1280-MIPI display.

KrzysztofChojnowski at 09:20, 23 June 2022

2022-06-23T09:20:42Z

← Older revision		Revision as of 09:20, 23 June 2022
Line 1:		Line 1:
	{{PageHeader\|SoMLabs NN demo for SpaceSOM-8Mplus}}		{{PageHeader\|SoMLabs NN demo for SpaceSOM-8Mplus}}

	This tutorial describes the example usage of the Neural Processing Unit in the iMX 8M Plus processor. It requires the SpaceSOM-8Mplus module with SpaceCB-8Mplus-ADV carrier board, SL-MIPI-CSI-OV5640 camera board and ~~connected~~ display ~~(TODO - DSI/HDMI)~~.		This tutorial describes the example usage of the Neural Processing Unit in the iMX 8M Plus processor. It requires the SpaceSOM-8Mplus module with SpaceCB-8Mplus-ADV carrier board, SL-MIPI-CSI-OV5640 camera board and SL-TFT7-TP-720-1280-MIPI display.

	The example application uses a pre-built TensorFlow Lite model for image classification. The model is available on the TensorFlow github:		The example application uses a pre-built TensorFlow Lite model for image classification. The model is available on the TensorFlow github:

KrzysztofChojnowski at 09:04, 23 June 2022

2022-06-23T09:04:52Z

Show changes

KrzysztofChojnowski at 06:46, 23 June 2022

2022-06-23T06:46:46Z

← Older revision		Revision as of 06:46, 23 June 2022
Line 17:		Line 17:
	== Software setup ==		== Software setup ==

	This demo was built with the [[VisionSOM_imx-meta-somlabs-honister \| Yocto Honister default image for SpaceSOM-8Mplus ]] and requires changes that add required packages and demo application. All the changes are collected in the provided somlabs_nn_demo.~~patch~~ that should be applied to the meta-somlabs repository:		This demo was built with the [[VisionSOM_imx-meta-somlabs-honister \| Yocto Honister default image for SpaceSOM-8Mplus ]] and requires changes that add required packages and demo application. All the changes are collected in the provided compressed patch [[File:somlabs_nn_demo.zip \| somlabs_nn_demo.zip]] that should be applied to the meta-somlabs repository:

	<pre>		<pre>
			unzip somlabs_nn_demo.zip
	cd imy-yocto-bsp/source/meta-somlabs		cd imy-yocto-bsp/source/meta-somlabs
	git apply somlabs_nn_demo.patch		git apply somlabs_nn_demo.patch

KrzysztofChojnowski: Created page with "{{PageHeader|SoMLabs NN demo for SpaceSOM-8Mplus}} This tutorial describes the example usage of the Neural Processing Unit in the iMX 8M Plus processor. It requires the Space..."

2022-06-22T15:24:58Z

Created page with "{{PageHeader|SoMLabs NN demo for SpaceSOM-8Mplus}} This tutorial describes the example usage of the Neural Processing Unit in the iMX 8M Plus processor. It requires the Space..."

New page

{{PageHeader|SoMLabs NN demo for SpaceSOM-8Mplus}}

This tutorial describes the example usage of the Neural Processing Unit in the iMX 8M Plus processor. It requires the SpaceSOM-8Mplus module with SpaceCB-8Mplus-ADV carrier board, SL-MIPI-CSI-OV5640 camera board and connected display (TODO - DSI/HDMI).

The example application uses a pre-built TensorFlow Lite model for image classification. The model is available on the TensorFlow github:

http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz

The model is executed on hardware Neural Processing Unit using the libvx_delegate.so library provided by NXP.

== Hardware setup ==

The camera board should be connected to the MIPI-CSI1 socket of the carrier board.

[[File:SpaceSOM-8Mplus-nn-demo.jpg|center|500px]]

== Software setup ==

This demo was built with the [[VisionSOM_imx-meta-somlabs-honister | Yocto Honister default image for SpaceSOM-8Mplus ]] and requires changes that add required packages and demo application. All the changes are collected in the provided somlabs_nn_demo.patch that should be applied to the meta-somlabs repository:

<pre>
cd imy-yocto-bsp/source/meta-somlabs
git apply somlabs_nn_demo.patch
</pre>

After that the image should be build according to the [[VisionSOM_imx-meta-somlabs-honister | Yocto Honister building tutorial]]. During the image compilation some errors may occur according to the i.MX Yocto Project User's Guide:

<pre>
cc1plus: error: include location "/usr/include/CL" is unsafe for crosscompilation [-Werror=poison-system-directories]
</pre>

To get around this error, add the following line in the tim-vx recipe:

<pre>
EXTRA_OECMAKE += " \
-DCMAKE_SYSROOT=${PKG_CONFIG_SYSROOT_DIR} \
"
</pre>

== Demo application ==

The application reads a frame from the connected camera and after a simple processing passes it to the neural network model for classification. The processing includes:
- cropping the sides of the frame in order to obtain square image from the center field of view of the camera
- scaling the image to get size required by the model and to fit it in the display area

After the image is classified by the model the results are displayed in the application window. Each result with the classification score above the defined threshold is shown. The following video shows some objects recognized by the application.