Understanding the Xilinx reVISION Stack: A Comprehensive Guide

The Xilinx reVISION stack includes a rich set of platform, algorithm, and application development resources, supporting popular neural networks such as AlexNet, GoogLeNet, SqueezeNet, SSD, and FCN. Additionally, the stack provides library elements, including predefined and optimized CNN layer implementations, essential for building custom Deep Neural Networks (DNN)/Convolutional Neural Networks (CNN). The machine learning elements are complemented by a range of OpenCV functions that support acceleration features, meeting the requirements of computer vision processing. For application layer development, Xilinx supports industrial-grade frameworks, including Caffe for machine learning and OpenVX for computer vision. The reVISION stack also includes development platforms provided by Xilinx and third parties, along with various sensors.

xFOpenCV Library Functions

The OpenCV library functions are crucial for developing many computer vision applications. Xilinx’s xFAST computer vision library is based on key OpenCV functions and can help you easily write and accelerate computer vision functionalities, fully leveraging FPGA structures through SDx or HLx design environments. Moreover, xFAST functions are consistent with OpenCV and optimized for high performance, high utilization, and ease of use. For more details, please visit: china.xilinx.com/revision.

• Thousands of functions in OpenCV 3.1 library support Cortex A9 and Cortex A53.
• OpenCV functions (including OpenVX subset) are provided as an optimized function library for Xilinx SoCs.
• Comprehensive library user guide supports device utilization and high performance.

The following figure illustrates a typical design flow in the field of computer vision using reVISION.

Frameworks, Networks, and Functions

Xilinx’s SoC is highly suitable for machine learning, providing up to a 6x efficiency improvement over leading embedded GPUs. The Xilinx reVISION stack breaks through traditional FPGA design barriers, allowing you to quickly deploy and infer using trained networks on Zynq SoC and MPSoC.

Features:
• Comprehensive software stack supports machine learning application deployment.
• Hardware-optimized libraries support Conv, ReLU, Pooling, Dilated conv, Deconv, FC, Detector & Classifier, and SoftMax layers.
• Caffe interoperability facilitates easy portability from Proto-Text files to meet network definition and trained weighting requirements.
• Optimized reference models are provided for a range of rich network topologies, including AlexNet, GoogLeNet, SqueezeNet, FCN, and SSD.
• Networks can be customized through software running on ARM processors without editing the FPGA.

The following figure illustrates a typical design flow in the field of machine learning using reVISION.

The reVISION stack includes four initial design examples (more will be provided in the future) to help you get started quickly. These design examples will help you understand the unique advantages of Xilinx All Programmable SoCs in high-performance embedded vision applications. Below is a brief introduction to these four design examples.

• LK High-Density Optical Flow @ 4K60 – Real-time high-density optical flow implementation, detecting object motion at every pixel. This example uses a non-iterative, non-pyramid implementation based on the Sony IMX274 sensor and MIPI interface, with input at 4K@60 FPS.

• Stereo Vision – Real-time stereo disparity computation, including remapping, calibration, and local block matching. It can handle dual 1080p30 stereo camera input from USB3.

• Deep Learning: GoogLeNet – GoogleNet benchmark, utilizing INT8 demonstration and standard ImageNet input.

• Combined Density Optical Flow, Stereo Vision, and Deep Learning Example Design (the above three examples have been introduced) – This design provides real-use cases for an automated vision system, integrating high-density optical flow, stereo vision, and CNN networks into a unified design example.

The reVISION stack is based on the SDSoC environment. SDSoC supports developing algorithms and/or applications in C, C++, and/or OpenCL using reVISION resources. The SDSoC environment can also be used to extend reVISION resources with new software libraries that meet acceleration requirements.

The SDSoC™ development environment provides a familiar embedded C/C++/OpenCL application development experience, including an easy-to-use Eclipse IDE and a comprehensive design environment that meets the deployment requirements of heterogeneous Zynq® All Programmable SoCs and MPSoCs. It comes with the industry’s first C/C++/OpenCL full-system optimized compiler, supporting system-level modeling, automated software acceleration of programmable logic, automated system interconnect generation, and acceleration programming libraries. It also allows end users and third-party platform developers to quickly define, integrate, and validate system-level solutions, providing customized programming environments for end customers.

The reVISION stack is aimed at the Zynq SoC and Zynq MPSoC series.

The development boards launched by Xilinx and its alliance program members can help you quickly start development work on the reVISION stack. The table below lists the Xilinx development boards and sensors supported by the stack.

Xilinx reVISION Target Boards (and BSP) and Kits

The Xilinx Zynq platform provides acceleration capabilities for computer vision and machine learning algorithms, enabling rapid response and reconfigurability, meeting the requirements of hardware platform developers to quickly upgrade to the best types and combinations of sensors while also quickly upgrading connections to new devices and/or the cloud.

Xilinx supports a range of sensor interface interconnect standards as well as video input and output.