1 Introduction
As machines shoulder more of the workload, massive amounts of data will be generated by devices and sensors, and drive demand for neural network processing. Centralized processing in datacenters is no longer efficient and is prompting the evolution of deep learning outside of the datacenter. This is achieved via AI inference at the edge.
In the past, machine learning and inference relied on powerful x86 platforms and additional graphics cards. Now, AI accelerator modules on arm-based platforms offer a more efficient way to extend AI inference capabilities at the edge. By relocating AI-based tasks to dedicated AI processors, developers can increase on-device inference and run sophisticated deep learning and computer vision applications while minimizing power consumption.
Advantech and Hailo’s partnership exemplifies the power of collaboration. In this article, we demonstrate how Advantech integrates Hailo’s Hailo-8™ M.2 Module with RSB-3720 to create a scalable, energy-efficient edge AI solution. With Hailo’s AI SDKs, TAPPAS, and development environment already integrated into AIM-Linux Software Services, we further demonstrate object detection applications on RSB-3720 with Hailo-8. This is the best way to streamline the use of these mature, power-efficient, and cost-effective solutions. This means developers can save time and bring products to market faster.
2 Prerequisites
For this demo, we will focus on the most common surveillance scenario that uses object detection models. The following items are required for this scenario.
Necessary Hardware:
1 x Advantech RSB-3720 single board computer
1 x Hailo-8™ M.2 Module compatible with NGFF M.2 form factor A+E keys (Advantech part number: 968DD00322)
1 x UVC camera connected to the USB interface
1 x HDMI monitor plugged into the single board
1 x USB to serial cable for debug console (when there’s no additional keyboard plugged into the single board)
3 System Diagram:
Yocto BSP development in RSB-3720 is used by default.
4 Building Instructions
4.1 Host development environment setup:
Setup RSB-3720 docker environment
#update CA certificates
$ sudo apt update
$ sudo apt-get install apt-transport-https ca-certificates -y
$ sudo update-ca-certificates
$ export GIT_SSL_NO_VERIFY=1
$ git config --global user.name "Your Name"
$ git config --global user.email you@example.com
$ git config --global url."https://".insteadOf git://
$ sudo chown adv:adv rsb-3720-hailo/
$ repo init -u git://github.com/ADVANTECH-Corp/adv-arm-yocto-bsp.git -b refs/tags/imx8LBVA0333 -m imx8LBVA0333.xml
$ repo sync
4.2 BSP Integration:
4.2.1 Prepare hail meta layer into the Yocto BSP folder
$ cd rsb-3720-hailo/source
$ git clone https://github.com/hailo-ai/meta-hailo -b zeus
$ cd ..
$ MACHINE=imx8mprsb3720a1 DISTRO=fsl-imx-xwayland source imx-setup-release.sh -b hailo
4.2.2 Add meta layer to build folder in the /conf/bblayers.conf
BBLAYERS += " ${BSPDIR}/sources/meta-hailo/meta-hailo-accelerator \
${BSPDIR}/sources/meta-hailo/meta-hailo-libhailort \
${BSPDIR}/sources/meta-hailo/meta-hailo-tappas"
4.2.3 Append HAILO application and drivers in the /conf/local.conf
INHERIT += "rm_work"
IMAGE_INSTALL_append = " libhailort hailortcli pyhailort libgsthailo hailo-pci hailo-firmware "
IMAGE_INSTALL_append = " libgsthailotools tappas-apps "
4.2.4 Build the image
$ bitbake imx-image-full
4.2.5 Flash the image to sd card.
$ sudo dd if=./imx-image-full-imx8mprsb3720a1-20220309072100.rootfs.wic of=/dev/sdx
5 Running Yolov5 Object Detection Application on RSB-3720 with Hailo-8
After flashing the image and power on RSB-3720, enter the command to execute the application in the terminal of the device:
$ gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw,format=YUY2,width=1280,height=720 ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! hailonet hef-path=yolov5m_yuv.hef debug=False is-active=true qos=false batch-size=1 ! queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailofilter function-name=yolov5 so-path=/usr/lib/hailo-post-processes/libyolo_post.so qos=false debug=False ! queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! hailofilter so-path=/usr/lib/hailo-post-processes/libdetection_draw.so qos=false debug=False ! queue leaky=downstream max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! videoconvert ! fpsdisplaysink video-sink=autovideosink name=hailo_display sync=false text-overlay=false
6 Conclusion
Adding an AI accelerator module with M.2 and mPCIE interface to an Arm-based computing platform, such as Hailo-8 on Advantech i.MX 8M Plus-based platforms, can easily create a scalable AI processing unit. When combined with the Hailo-8TM AI processor, it further provides maximum power and transfers learning efficiency for machine learning on Arm-based solutions.