China Manufacturing 2025 is the first decade action plan implemented by the Chinese government to strengthen the manufacturing industry, aiming to significantly improve the quality and level of China’s manufacturing industry. It highlights the importance of innovative technologies surrounding industrial robots. Currently, industrial robot technology is widely applied in fields such as automotive processing, food sorting, and automated production assembly. Industrial field communication technology is one of the core technologies of intelligent manufacturing, and with the rapid development of intelligent manufacturing, higher requirements have been put forward for the real-time, reliability, and bandwidth of communication between nodes. Industrial Ethernet masters can run on various platforms, requiring only standard network ports provided by hardware. Therefore, cost-effective, simple, flexible, and high-speed real-time embedded platforms can be used as industrial Ethernet masters, providing a new networked, information-based, and intelligent solution.
Traditional hardware platforms use DSP for image processing, which is relatively slow and cannot meet the real-time requirements of image processing, limiting the application of machine vision. FPGA, as a hardware platform for digital image processing, has advantages such as high speed, high integration, and strong reliability. To improve image quality and enhance the real-time processing of images, this paper proposes a visual servo control system for a six-degree-of-freedom robot based on EtherCAT, integrating the camera into the Zynq-based EtherCAT master station to enhance the real-time capability of visual servoing. Tests show that this platform can respond promptly to changes in visual detection targets, providing a practical solution for industrial automation.
This paper focuses on the research of industrial robots based on machine vision, with the main work including the following points:
(1) Designed the overall scheme of the visual servo control system for the six-degree-of-freedom robot based on EtherCAT. Using the ESR6B robot, Zynq, and camera as hardware foundations, a six-degree-of-freedom robot vision platform was built.
(2) Designed a visual detection scheme based on FPGA. Utilizing the HLS and CV libraries provided by Xilinx for real-time processing of pixel streams captured by the camera, a program flow for identifying the coordinates of objects was designed, allowing the EtherCAT master station to read object coordinate values in real-time. The processed images were output to the display via HDMI.
(3) Completed the EtherCAT master station’s servo position control. Using the Pl layer of Zynq for the transmission and reception of EtherCAT frames and calibration of the master clock, running the bare-metal program of Sum in the PS layer, and porting the inverse kinematics algorithm of the six-degree-of-freedom serial robot into the master station code for real-time following control of the robot’s end position based on visual feedback.
(4) Verified the accuracy and real-time performance of object recognition through experiments, and conducted actual motion control experiments using a self-developed six-degree-of-freedom serial robot. Experimental results demonstrate that the robot can follow the motion of objects displayed on the screen in real-time.
The main challenge in this design is how to improve the real-time performance of visual detection and robot motion control. Innovatively, a Xilinx Zynq heterogeneous processor was used, with FPGA as the main unit for visual processing, and the EtherCAT master station was ported to the Zynq platform. This allows real-time visual detection and motion control to be completed on a single chip. This design employs HLS for visual processing and optimizes the code, achieving a delay of only 18.6ms for the entire visual processing module. Additionally, in terms of communication, this design uses FPGA for the transmission and verification of EtherCAT frames, stabilizing the DC synchronous cycle jitter to less than 1 microsecond. Real-time visual detection and motion control have broad applications in industrial robot visual servoing. This experiment integrates the camera into the EtherCAT master station applied to a self-developed six-degree-of-freedom serial robot, incorporating the robot kinematics algorithm into the EtherCAT master station, allowing the robot to move in real-time according to detected objects. Experimental results show that the real-time performance of this design far exceeds that of PC platform visual processing and achieves real-time visual following motion along the robot’s X-axis direction.
1. ESR6B Robot Hardware System Design
The ESR6B robot is a self-developed robot and a typical multi-degree-of-freedom serial robot, as shown in Figure 2-1.。
Figure 2-1 ESR6B Robot Body
The hardware system for vision-guided robots mainly consists of the robot body, control console, Zynq, and vision hardware, as shown in Figure 2-2.
Figure 2-2 Overall System Scheme
2. Basic Design Parameters of ESR6B Robot
The robot has 6 degrees of freedom, and its main technical parameters include degrees of freedom, load capacity, repeatability, positioning accuracy, and workspace, which are the main indicators reflecting the performance of the robot. The basic technical parameters of the ESR6B robot are shown in Table 2-1.
Table 2-1 Basic Technical Parameters of ESR6B Robot
3. Building the Robot Vision System Hardware Platform
The choice of hardware for the robot vision system directly affects image acquisition, image quality, subsequent processing, and the real-time nature of the entire control system. Therefore, hardware should be strictly selected based on requirements and performance. The six-degree-of-freedom robot visual servo control system based on EtherCAT mainly includes modules such as camera, Zynq, storage, display, and communication.
Zynq is a heterogeneous chip designed by Xilinx that includes FPGA + ARM. In the Zynq system-on-chip, there is a processing system (PS) based on ARM and programmable logic units (PL). The EtherCAT master station and Linux operating system run on the PS side, while image algorithms are hardware-accelerated on the FPGA in the PL side. The FPGA is integrated with the arm through the AXI bus interconnect technology, fully leveraging the advantages of both device structures.
The main characteristic parameters of the camera include resolution, frame rate, and color space. Based on these parameters, a camera that meets experimental requirements is selected. In this system design, the OV 5640 CMOS type digital image sensor is chosen, which supports outputting images with a maximum resolution of 5 million pixels (2592×1944), supports VGA timing output of image data, and the output image data format supports YUV (422/420), YCbCr 422, RGB 565, and JPEG formats. In this design, RGB 565 format data is used.
Figure 2-3 OV 5640 Camera
In this design, the image data is captured by the OV 5640 camera, then preprocessed and coordinate detected by the FPGA. The data stream passes through VDMA and enters DDR3 memory via HP0, then returns through HP0, goes through VDMA, and finally outputs video through the HDMI interface.
The main parameters of the light source include contrast, brightness, surface texture, and uniformity. Based on the characteristics of the light source, a ring light source from Shanghai Dongguan Technology, model Rin-90-6R-10W, with white LED light sources, is selected. The physical image is shown in Figure 2-4.
Figure 2-4 Physical Image of Ring Light Source
The six-degree-of-freedom robot visual servo control system based on EtherCAT is shown in Figure 2-5.
Visual Servo Performance Analysis
Visual servoing requires a very short delay between object recognition and control to reflect the real-time nature of control. This paper designed a set of comparative experiments, where Scheme 1 uses the EtherCAT master station with an integrated camera on the Zynq board, and Scheme 2 connects a USB camera to a PC and uses advertisements to transmit coordinate data to the TwinCAT master station. Both schemes use the same image processing algorithm, and the master stations are in CSP mode.
Figure 4-5 Performance Testing Platform
The delay calculation platform consists of STM32 + TFT color screen and EtherCAT stepper slave based on XMC4300. When the screen starts to refresh red, the timer starts counting, and when the pulse from the stepper slave is received, the timing stops. The delay between Scheme 1 and Scheme 2 is compared (in units of microseconds), as shown in Figure 4-6.
Figure 4-6 Comparison of Schemes
It can be seen that the average detection delay of this design is only 58 milliseconds, while Scheme 2 is as long as 2.48 seconds. Under the same image detection algorithm, the FPGA’s delay is lower. Moreover, since this design integrates the camera directly into the master station board, the image transmission delay is lower, and the cost is reduced.
This article content is reprinted from Industrial Robots.
The Robot Lecture Hall Rob Community is now recruiting! If you are currently engaged in or want to engage in the robotics industry, or want to learn in this direction, you are welcome to join us to discuss cutting-edge robotics technology.
Additionally, legged robots, medical robots, industrial robots, and special robots professional discussion groups are recruiting, and friends from various professional fields are welcome to join.
Reply “Group Chat” in the Robot Lecture Hall public account dialog box to get the group joining method!
The Robot Lecture Hall is recruiting part-time content creators and column writers.
Please send your resume and original works to the email: [email protected]
We have no requirements for profession, location, etc., and welcome interested and capable friends to join!
Are you tired? Tap “Looking” to support us!
