Design Guide for Intelligent Video Applications
With the improvement of broadband wired and wireless network infrastructure and the growth of global security market demand, the application of video surveillance is experiencing explosive growth. The development trend of video surveillance systems is very obvious. After undergoing digitization and networking, the next important trend is intelligence, namely the application of intelligent monitoring and video analysis technology. Traditional video surveillance relies on manual monitoring to identify security risks or abnormal conditions, or for post-event analysis. This application has inherent drawbacks, making it difficult to achieve real-time security monitoring and detection management. Monitoring systems with intelligent analysis capabilities can actively collect and analyze data by distinguishing characteristics such as the shape and movement of monitored objects, and execute actions like alarming, recording, and analyzing based on preset conditions. Intelligent monitoring systems can operate on servers or run on DSP-based embedded systems, the latter of which has gradually become mainstream.
Intelligent video applications can be broadly categorized into three areas: security, human behavior detection, and intelligent transportation. Among these, security applications are widely regarded as the most promising market, including several application categories: intrusion detection, which can automatically detect movement behavior characteristics in video footage; item removal detection, which can automatically detect item relocation events—issuing an alarm when an item is taken or moved from a specific location within the monitored area; abandoned object detection, which can automatically detect abandoned items—issuing an alarm when an item is placed or abandoned in a monitored area; and intelligent tracking, enabling the camera to autonomously drive its pan-tilt-zoom (PTZ) functions. Human behavior detection applications include post detection (automatically detecting the presence of personnel at a post) and loitering detection (detecting individuals loitering in important areas). Intelligent transportation applications include detecting illegally parked vehicles—issuing an alarm when a vehicle illegally remains in a monitored area; and wrong-way vehicle detection, which promptly identifies vehicles traveling in the wrong direction.
As accuracy and reliability gradually improve and product costs decrease, intelligent video is being applied in more and more scenarios. It can replace some security equipment, reduce the workload of security personnel, improve work efficiency, and lower management costs. In fact, the application of intelligent video holds enormous potential. As technology matures, the application fields of intelligent video technology are rapidly expanding, mainly including the aforementioned security, transportation, as well as retail and service industries, such as people counting, facial recognition, crowd control, attention control, and traffic flow control.
The demand for real-time video monitoring is rapidly growing, especially with the increasing demand for real-time security monitoring applications. The ability to promptly identify security risks or abnormal behaviors of targets is becoming increasingly significant. Driven by this growing demand, intelligent video monitoring system products are becoming a new hotspot in video surveillance applications.
Design Guide for Intelligent Video Applications
The choice of hardware platform solutions often determines the overall cost, performance, usability of development tools and methods, and the feasibility of future upgrades of the system. Therefore, selecting the right platform is crucial. The unique nature of intelligent video applications requires comprehensive consideration in choosing the hardware platform. The networking and intelligent analysis requirements of video surveillance systems, along with the cost, size, and power consumption constraints of large-scale engineering installations, non-standardized intelligent video analysis methods, and almost customized solution optimization approaches, make the aggregation processing platform that combines the advantages of MCU and DSP, with flexible software design and powerful processing capabilities, exhibit more significant advantages. This article will analyze the main design technology points in intelligent video design based on the unique features of ADI’s Blackfin aggregation DSP processor.
1. Hardware Platform Selection
Customization capability is very important. Many factors restrict the pace of intelligent application in video surveillance systems: firstly, the video algorithms for intelligent monitoring are quite complex and difficult to standardize. Each system provider’s video analysis software has its unique algorithms, leading to a lack of unified standards for products in the market; secondly, the application scenarios of video surveillance systems are complex, and user requirements are diverse, so there are many customization requests. Therefore, video analysis solutions often need to be optimized according to the characteristics and needs of customer applications, with algorithms varying widely. Additionally, due to the high complexity of intelligent video applications, the processing capability requirements for solutions are higher. Traditional MCU+ASIC video surveillance solutions struggle to meet various personalized design and high computation requirements. Even selecting a standard DSP+MCU dual-chip solution often fails to satisfy the complex computation needs of intelligent video monitoring applications, necessitating the addition of co-processors. This complex solution is undesirable in terms of BOM costs, power consumption, and development difficulty. The Blackfin processor fully utilizes the advantages of the MCU+DSP aggregation architecture, meeting the system control and high-intensity computation needs of intelligent video applications. In particular, the high-performance dual-core architecture represented by the BF561 has become the preferred platform solution for intelligent video applications.
The scalability of the solution is also a factor to consider. Intelligent video analysis applications not only require optimization based on the application environment and purpose but may also have different requirements from different customers. For instance, some leading digital video surveillance solutions currently support H.264 baseline @Level3.0 and MPEG4 D1+CIF dual streams, and may expand to support H.264 D1+CIF dual streams in the future. With the broader application of intelligent video analysis, such as IP cameras, wireless video surveillance, intelligent transportation systems, various applications may have significant differences in demands for various interface functions, communication standards, and user interfaces. The flexible scalability of hardware platform solutions to meet diverse needs is crucial. Moreover, as mentioned earlier, intelligent video analysis technology has only developed over the past few years, and with continuous technological advancements and the introduction and improvement of related standards, the upgradeability of products is critical—an important concern for developers and a key characteristic for end customers. The Blackfin DSP has unique advantages in parallel processing of algorithms, especially the ADSP-BF561, which employs dual DSP cores to implement complex intelligent video processing algorithms.
Video application optimization features. Although some solutions possess strong processing capabilities and scalability, whether the solution has been primarily optimized for video applications is also worth noting, as this directly impacts the availability of hardware and software design resources for design engineers, as well as system design difficulty and achievable performance. Taking the Blackfin processor as an example, Blackfin has been specifically optimized for high-intensity and high data rate digital and media processing: Blackfin’s dozens of DMA channels and flexibly configurable Cache meet the demands of video surveillance systems for large computation volumes and high data throughput. ADI has developed fully optimized audio and video codecs, which are provided free of charge to major customers. For video applications, Blackfin integrates many hardware drivers, including drivers for WiFi, audio/video codecs; Blackfin’s four video arithmetic operation units and video pixel instruction set significantly accelerate video computation speeds. In basic operators of intelligent video analysis, such as histogram statistics, median operations, Sobel operations, and morphological dilation operations, Blackfin’s MIN and MAX instructions can eliminate conditional jumps, saving processor cycles. Moreover, Blackfin supports 13 types of vector operations for non-video data. Appropriately designing data structures allows the use of Blackfin’s unique instructions in multiple stages, such as foreground-background separation, threshold calculation, and updates, making intelligent video analysis algorithms faster. Most of these effective instructions can be executed in parallel, effectively doubling Blackfin’s processing capabilities.
Development Tools and Available Resources
Low power consumption and stability are crucial. Considering that intelligent video monitoring devices typically run 24/7, stability and power consumption are also important. In terms of low power consumption, the Blackfin processor employs various energy-saving technologies: it uses a gated clock core design that allows selective power shutdown of functional units on a per-instruction basis; supports multiple power-off modes for periods of minimal CPU activity; and features a self-contained dynamic power management circuit that independently controls operating frequency and voltage to meet the performance requirements of the executing algorithms. Most Blackfin processors provide on-chip voltage regulation circuits and can operate at voltages as low as 0.8V. The unique aggregation processing architecture and 90nm process of Blackfin lay the foundation for its leading low-power processing capabilities. Due to its high processing power, systems based on the Blackfin platform can reduce the number of main chips, and the rich functionality and interfaces can meet various peripheral and functional expansion needs, reducing the number of components and ensuring higher stability and reliability. Currently, Blackfin DSP is the best in terms of low power consumption and stability among DSPs in the same price range.
Support for Embedded Operating Systems. Intelligent video analysis is usually network-based applications that require operating system support; therefore, choosing solutions with broad embedded system support capabilities is essential to ensure that future products do not require hardware platform changes when switching operating systems, maintaining the continuity of R&D achievements. Currently, numerous embedded operating systems are available, each with its advantages, and the support capability of hardware platform solutions for these operating systems is one of the key considerations in solution selection. For example, Blackfin processors can support mainstream operating systems, including uCLinux, ThreadX, Nucleus, uCOS-II, and more than ten other embedded operating systems, allowing customers to choose a software architecture base that they are familiar with or that is more cost-effective according to their requirements.
2. Development Tools and Available Resources
Intelligent video monitoring devices constitute a complex system involving intricate hardware and software design, human-machine interfaces, communication connections, etc., resulting in a high level of system design difficulty. Therefore, whether the chosen hardware platform solution can provide a complete suite of development tools, necessary software modules, mature reference designs, system design support, and a comprehensive design ecosystem is critical for completing system design on time and with high quality. In fact, not all platform solution providers can offer this support.
Taking the Blackfin series processors as an example, hardware platforms using Blackfin processors have been widely adopted by numerous device companies globally, ranging from general DVRs, IP cameras, digital video surveillance to intelligent video monitoring. The Blackfin processor has gained favor from many enterprises, and the availability of complete development tools and reference designs is one of the key reasons for its popularity. ADI provides industry-leading tools, beginner kits, and support, including the well-known ADI CROSSCORE software and hardware tools that support other Blackfin processors. These tools include the award-winning VisualDSP++ integrated development and debugging environment (IDDE), emulators, and EZ-KIT Lite evaluation hardware.
To improve development efficiency and reduce development difficulty, it is advisable to utilize existing resources during development, such as open examples. ADI provides a wealth of examples and materials for this purpose. For instance, ADI offers a free “Image Tool Box” software package optimized for commonly used mathematical functions in image processing applications for customers to call upon during application development. ADI also provides complete reference designs, evaluation boards, development tools, algorithm IP, application modules developed by local partners, and a full set of turnkey solutions provided by third-party partners, including both hardware and software. The video monitoring applications of Blackfin processors currently have several third-party partners in China with rich engineering experience, establishing a complete ecosystem.