Click on the above “Beginner’s Guide to Vision“, select to add “Star” or “Top“
Important information delivered promptlyVisual sensing technology is one of the seven major categories of sensing technology. A visual sensor refers to a sensor that calculates the characteristic quantities (area, centroid, length, position, etc.) of an object by processing the images captured by a camera, and outputs data and judgment results. Visual sensors are the direct source of information for the entire machine vision system, mainly consisting of one or two image sensors, and sometimes supplemented with light projectors and other auxiliary equipment. The main function of the visual sensor is to obtain the most primitive images that the machine vision system needs to process.
Vision is a way for biological entities to acquire information about the external environment, and it is one of the most effective means for organisms in nature to obtain information, being a core component of biological intelligence. Humans rely on vision to acquire 80% of their information. Inspired by this, researchers began to install “eyes” on machines, allowing them to acquire external information through “sight” just like humans, giving rise to a new discipline—computer vision. By studying biological visual systems, people have mimicked and created machine vision systems. Although they differ significantly from human visual systems, this has been a groundbreaking advancement for sensor technology. The essence of visual sensor technology is image processing technology, which captures signals from the surface of objects and renders them into images for researchers to analyze.
Visual sensors capture thousands of pixels of light from an entire image. The clarity and detail of an image are typically measured by resolution, represented by the number of pixels. After capturing an image, the visual sensor compares it with a reference image stored in memory for analysis. For example, if the visual sensor is set to identify a machine part correctly inserted with eight bolts, the sensor knows to reject a part with only seven bolts or one with misaligned bolts. Moreover, regardless of the position of the machine part in the field of view, or whether the part rotates within a 360-degree range, the visual sensor can make judgments. The emergence of visual sensing technology has resolved issues faced by other sensors due to limitations of space or the size of detection equipment, making it widely welcomed in the industrial manufacturing sector.
Visual sensing technology includes 3D visual sensing technology, which has a wide range of applications, such as multimedia phones, webcams, digital cameras, robotic visual navigation, automotive safety systems, biomedical pixel analysis, human-computer interfaces, virtual reality, surveillance, industrial inspection, wireless long-distance sensing, microscopy, astronomical observation, autonomous ocean navigation, scientific instruments, and more. These various applications are all based on 3D visual image sensor technology. Particularly, 3D imaging technology has urgent applications in industrial control and autonomous navigation for vehicles.
Intelligent visual sensing technology is also a type of visual sensing technology. Intelligent visual sensors under this technology are also known as smart cameras and represent one of the fastest-growing new technologies in the field of machine vision in recent years. A smart camera is a compact machine vision system that integrates image acquisition, image processing, and information transmission functions. It is an embedded computer vision system that combines image sensors, digital processors, communication modules, and other peripherals into a single camera. Due to this integrated design, system complexity is reduced, and reliability is improved. At the same time, the system size is greatly reduced, broadening the application fields of visual technology.
The ease of learning, use, and maintenance of intelligent visual sensors, along with their convenient installation, allows for the rapid construction of reliable and effective visual inspection systems, leading to the rapid development of this technology. The image acquisition unit of visual sensors mainly consists of CCD/CMOS cameras, optical systems, lighting systems, and image acquisition cards, converting optical images into digital images and transmitting them to the image processing unit.
1. Automotive Body Visual Inspection System
The body shaping process is one of the key procedures in automobile manufacturing, requiring strict adherence to various indicators, necessitating 100% inspection of the body. Traditional body inspection methods use coordinate measuring machines, which are complex to operate, slow, and time-consuming, only allowing for sampling inspections. Typically, the critical dimensions of the body include the size of the windshield, the edge position of the door installation, and the location of positioning holes. Therefore, visual sensors are distributed near these positions to measure their corresponding edge, hole, and surface spatial position dimensions. Measurement stations are designed on the production line, and after positioning the body, it is placed within a frame composed of vertically and horizontally distributed metal pillars and rods, where visual sensors can be flexibly installed based on needs. Depending on the number of measurement points, a corresponding number of visual sensors can be installed (usually, each visual sensor measures one testing point), including various types of sensors such as binocular stereo visual sensors and profile sensors.
The measurement system works as follows: the production line transports the body to the measurement station for accurate positioning, then the sensors start working in the required sequence, the computer collects images of the detection points and processes them, calculates the spatial three-dimensional coordinates of the measured points, compares the calculated values with the standard values, derives the detection results, and sends the body out of the measurement station.
In industrial production, seamless steel pipes are an important category of industrial products, and their quality parameters are crucial manufacturing data. Among these, the straightness and cross-sectional area of the pipes are the main geometric parameters that control the quality of seamless steel pipe manufacturing. However, several factors make measuring these parameters a challenge:
1. Seamless steel pipes require non-contact measurement, and the manufacturing environment is harsh;
2. Seamless steel pipes have large spatial dimensions, which also requires the detection system to possess a large measurement space. The emergence of visual sensing technology has resolved these issues, as it employs non-contact measurement and has a large measurement range.
The measurement system consists of multiple structured light sensors. The structured light projector on the sensor projects a light plane that intersects with the measured steel pipe, obtaining a partial arc on the circumference of the pipe’s cross-section, with the sensor measuring the spatial position of the partial arc. Each sensor in the system achieves measurement of a partial arc on a cross-section, and through appropriate mathematical methods, the cross-sectional dimensions and the spatial position of the cross-sectional center can be obtained by fitting the arcs. The spatial envelope distributed from the cross-sectional center yields the straightness parameters. Under the control of a computer, the measurement system can complete measurements within seconds, meeting real-time requirements.
The three-dimensional profile digitization measurement technology is a foundational technology for reverse engineering and product digital design, management, and manufacturing. The mechanism of achieving three-dimensional profile digitization measurement combines non-contact visual measurement, rapid measurement, and the latest high-resolution digital imaging technology. Since the objects measured are often large and have complex surfaces, measurements are typically divided into two parts: local three-dimensional information acquisition and overall stitching. First, visual scanning sensors are used to measure various local areas of the profile, and then stitching technology is employed to combine the parts into a complete image.
The visual scanning probe of this sensor is designed using the principle of local binocular stereo visual measurement. Overall stitching essentially places the collected data onto a common coordinate system, allowing for a comprehensive data description. High-resolution digital cameras collect data from above the measurement space at different angles and positions, using the principle of directed beam intersection adjustment to obtain the spatial coordinates of control points and establish a global coordinate system. Finally, data stitching is completed through the association and transformation of various coordinate systems.
Currently, the selection of machine vision sensors is increasingly widespread in applications, making it worthwhile for us to learn how to choose machine vision sensors. The camera is the “eye” of the machine vision system, and the heart of the camera is the image sensor. The choice of sensor depends on accuracy, output, sensitivity, the cost of the machine vision system, and a thorough understanding of application requirements. A basic understanding of the main performance of sensors can help developers quickly narrow down their search to find suitable sensors.
Most users of machine vision systems recognize that the camera is a key element of the system, often treating it as the “chip” of the vision system. The camera itself is a complex system: it includes lenses, signal processors, communication interfaces, and the core component that converts photons into electrons: the image sensor. The lens and other components work together to support the camera’s function, while the sensor ultimately determines the camera’s highest performance.
Many discussions in the industry focus on processing technologies, and whether CMOS or CCD sensors are superior. Both technologies have their advantages and disadvantages, and the processed sensors exhibit different performances. End-users are more concerned with how the sensor performs in the final application rather than how it is manufactured.
In specific applications, three key factors determine the choice of sensor: dynamic range, speed, and responsiveness. The dynamic range determines the quality of the images that the system can capture, also known as the ability to reflect details. The speed of the sensor refers to how many images it can produce per second and the amount of image output the system can receive. Responsiveness refers to the efficiency with which the sensor converts photons into electrons, determining the brightness level needed for the system to capture useful images. The technology and design of the sensor jointly determine these characteristics, so system developers must have their own evaluation criteria when selecting sensors. Detailed research into these features will assist in making the correct judgment.
Good news! The Beginner's Guide to Vision Knowledge Planet is now open to the public👇👇👇
Download 1: OpenCV-Contrib Extension Module Chinese Version Tutorial
Reply "Extension Module Chinese Tutorial" in the backend of the "Beginner's Guide to Vision" public account to download the first Chinese version of the OpenCV extension module tutorial on the internet, covering installation of extension modules, SFM algorithms, stereo vision, target tracking, biological vision, super-resolution processing, and more than twenty chapters.
Download 2: Python Visual Practical Projects 52 Lectures
Reply "Python Visual Practical Projects" in the backend of the "Beginner's Guide to Vision" public account to download 31 visual practical projects including image segmentation, mask detection, lane detection, vehicle counting, eye line addition, license plate recognition, character recognition, emotion detection, text content extraction, and face recognition to help quickly learn computer vision.
Download 3: OpenCV Practical Projects 20 Lectures
Reply "OpenCV Practical Projects 20 Lectures" in the backend of the "Beginner's Guide to Vision" public account to download 20 practical projects based on OpenCV to advance OpenCV learning.
Group Chat
Welcome to join the reader group of the public account to communicate with peers. Currently, there are WeChat groups on SLAM, three-dimensional vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (these will gradually be subdivided). Please scan the WeChat ID below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiaotong University + Vision SLAM". Please follow the format for notes; otherwise, you will not be approved. After successful addition, you will be invited into relevant WeChat groups based on your research direction. Please do not send advertisements in the group; otherwise, you will be removed from the group. Thank you for your understanding~