0. Introduction
For sensor selection, many beginners and even engineers are not very clear about the differences between different price ranges and devices. Here, the author wants to provide some suggestions based on personal experience and some online information, as well as how to use these devices.
1. Camera
1.1 USB Camera (Amateur & Consumer)
A USB camera is a security camera that uses a USB interface, plug-and-play, foolproof operation, no capture card required, no power supply needed, and does not require disassembly, supporting laptops. Compared to traditional surveillance cameras, it is cheaper and allows remote network viewing. Convenient and practical, easy to operate.
-
Pros: Very easy to integrate.
-
Pros: Can perform many offline image tasks (exposure control, frame rate, etc.).
-
Pros: Provides input/interruption functions, saving computation application time (e.g., interrupting on new frames).
-
Cons: Since the USB bus uses CPU time, this will affect your application if using 100% CPU.
-
Cons: Not optimal for using hardware vision pipelines (hardware encoders, etc.).
-
Pros: Can work over long distances (up to USB standards).
-
Pros: Can support larger image sensors (1 inch or more for better image quality and less noise).
1.2 MIPI CSI Bus Camera (Consumer)
The CSI-2 interface specification was released by the MIPI (Mobile Industry Processor Interface) Alliance in 2005 regarding the camera serial interface. The MIPI Alliance was established in 2003 by companies such as ARM, Nokia, ST, and TI, aiming to standardize internal interfaces in mobile phones such as camera, display, RF/baseband interfaces, etc., thereby reducing design complexity and increasing design flexibility. The MIPI Alliance has different Work Groups that define a series of internal mobile phone interface standards, such as camera interface CSI, display interface DSI, RF interface DigRF, microphone/speaker interface SLIMbus, etc.
-
Pros: Optimized based on CPU and memory usage for image processing and storage.
-
Pros: Can fully utilize hardware vision pipelines.
-
Cons: Supports short distances (usually no more than 10cm). Unless you use serialized systems (GMSL, FPD Link, COAXPress, Ambarella), but these systems are currently immature and require customization.
-
Cons: Usually has small sensors like mobile camera modules, or you have to spend more to customize. Reduce extra noise from small sensors through hardware denoising in TX1/2.
-
Pros: Can perform low-level access and control of sensors/cameras.
1.3 Ethernet Camera (Robotics & Automotive)
The American Automation Imaging Association (AIA) introduced the GigE Vision standard, which is developed based on the Gigabit Ethernet communication protocol, achieving fast long-distance image transmission via existing Ethernet cables, capable of real-time, high bandwidth transmission of large images, with a transmission rate of up to 125MB/s and a transmission distance of 100 meters.
GigE industrial cameras can unify the protocols of almost all industrial cameras in machine vision products, while allowing other organizations to develop compatible software and hardware based on them. Currently, many products support the GigE Vision standard.
1.4 GMSL Camera (Automotive)
The LVDS data transmission interface mainly has two protocols: GMSL and FPD Link, both of which are widely used in automotive video transmission, especially in the link between the camera and processor, achieving a link distance of 15m through STP or coaxial cables.
GMSL is a high-speed serial interface launched by Maxim, suitable for transmitting video, audio, and control signals, with a distance of up to 15m or longer when using 50Ω coaxial cables or 100Ω shielded twisted pair (STP) cables.
Taking ADAS cameras as an example, which is also a major application area for GMSL. Generally speaking, cameras not only send captured image data externally but also send frame synchronization signals, line synchronization signals, pixel clock, and other information, along with power supply, etc. It consists of many signal lines forming a parallel bus, and parallel buses do not have advantages in high-speed data transmission, so we need to convert these parallel signals into serial signals and transmit them at a higher frequency.
The PCIE capture card (Figure 7) is specifically designed for PC computers/industrial control machines to access GMSL cameras. The PCIE capture card features high bandwidth and multiple channels, with good universality and high real-time performance, widely used in the research and data collection of autonomous driving.
As shown in Table 2, the data transmission bandwidth of 1M (1 million pixels) – 8M (8 million pixels) cameras ranges from 655Mbps to 4.4Gbps. From the bandwidth limitation perspective, USB3.0 can support up to 2M, Ethernet can support up to 1M, and only PCIE can meet all transmission needs.
1.5 FPD Link (Automotive)
FPD-Link III serial bus solution supports high-speed video data transmission and bidirectional control communication through a single differential link, achieving full-duplex control. Integrating video data and control through a single differential pair can reduce interconnection line size and weight, while also eliminating deviation issues and simplifying system design. FPD Link is a serial bus solution from TI. Like GMSL, it can use 50Ω coaxial cables or 100Ω shielded twisted pair (STP) cables for low-latency transmission of uncompressed video and control data.
With FPD-Link™ serializers and deserializers, it simplifies the design and transmission of high-resolution signals for various video interfaces in automotive systems (including cameras for advanced driver assistance systems (ADAS) and infotainment system displays). Our FPD-Link camera serializer/deserializer supports the transmission of uncompressed video, control signals, and power through a single low-latency cable. Our FPD-Link display serializer/deserializer supports full 1080p, 2k, and 3k HD displays, as well as device-level HDCP keys, while also providing adaptive equalization to compensate for cable aging and temperature changes.
For a detailed history of automotive development, refer to the article “Summary of Automotive Electronic Data Transmission Protocols: CAN, CAN FD, A²B, Ethernet100Base-T1 & Ethernet1000Base-T1, FPD-LINK, GMSL”.
2. IMU
2.1 Inertial Measurement Unit
An Inertial Measurement Unit (IMU) is a device that measures an object’s three-axis attitude angles and acceleration. IMUs typically include gyroscopes, accelerometers, and sometimes magnetometers. Gyroscopes measure the angle/angle velocity in three axes, accelerometers measure acceleration in three axes, and magnetometers provide information about magnetic field orientation.
2.2 IMU Classification
For IMUs, the technical barriers are high and the market share is highly concentrated, mainly divided into three types: laser, fiber optic, and MEMS.
2.2.1 Ring Laser Gyro (RLG) IMU – Ring Laser Gyroscope (Automotive + Usable as Truth)
In 1913, French scientist George Sagnac discovered that when a light beam travels in a circular medium channel, if the circular channel itself has a rotational angular velocity, the time required for the light to travel in the direction of the channel’s rotation is more than that required to travel in the opposite direction. The two types of gyroscopes that utilize the Sagnac effect are the ring laser gyroscope and the fiber optic gyroscope. These two methods produce higher radar accuracy, but the costs are also higher.
The ring laser gyroscope IMU uses a ring laser and fiber optic sensors to measure angular velocity. It is based on the principle of optical rotation and calculates angular velocity by measuring the phase difference of the laser beam along the ring path.
2.2.2 Fiber Optic IMU – Fiber Optic Gyroscope (Automotive + Usable as Truth)
The fiber optic IMU uses fiber optic sensors to measure an object’s acceleration and angular velocity. It is based on the principle of interference in fiber optics, calculating motion parameters by measuring the light path difference in the fiber. This design is more ingenious, directly using fiber optics to create a circular medium channel. Compared to the former, it does not require precision processing of optical mirrors or strict sealing of optical cavities, fundamentally avoiding laser locking issues.
2.2.3 MEMS IMU (Consumer – Automotive)
MEMS (Micro-Electro-Mechanical Systems) is a microelectromechanical system that includes tiny mechanical structures and electronic components. MEMS IMU (Inertial Measurement Unit) is an inertial sensor based on MEMS technology used to measure acceleration and angular velocity.
The working principle of MEMS IMU is based on the micro-mechanical structures and electronic sensors in microelectromechanical systems. An alternating voltage of 14 kHZ is used to make the flexible vibrator resonate periodically in the direction of the red arrow in the figure. When the detection unit experiences rotation along the blue axis, the flexible vibrator in linear motion will be deflected due to the Coriolis force, resulting in 14kHZ oscillation, and using capacitive sensors to detect this amplitude can obtain the Coriolis force, thereby determining the angular velocity of rotation.
It mainly includes the following aspects:
-
Acceleration Measurement: MEMS IMU uses tiny mass blocks (mass) and spring systems in micro-mechanical structures to measure acceleration. When the IMU experiences acceleration, the mass block will be displaced due to inertial force, and this displacement will be converted into an electrical signal through electronic sensors such as capacitance, resistance, or piezoelectric effect, thereby measuring the magnitude and direction of acceleration.
-
Angular Velocity Measurement: MEMS IMU uses tiny gyroscopes (gyroscopes) in micro-mechanical structures to measure angular velocity. Gyroscopes can sense angular velocity through inertial force or rotational force. Generally, MEMS gyroscopes measure changes in angular velocity through tiny vibrating structures, piezoelectric effects, or optical sensors.
-
Signal Processing and Output: The electronic components of MEMS IMU are responsible for converting mechanical motion into electrical signals, processing and amplifying the signals. These processing circuits can convert tiny mechanical movements into measurable and usable electrical signals. Finally, the IMU outputs the measured acceleration and angular velocity data to the system for further applications.
2.3 Major Manufacturers
MEMS-IMU is mainly led by overseas Honeywell, ADI, and Northrop Grumman, with these three accounting for 55% of the global market; domestic manufacturers are catching up, currently represented by Minghao Sensor, Xindong Lian Ke, and Meitai Technology, with automotive IMU modules mainly from Huayi and Daoyuan.
First Tier: Suzhou Gude (Minghao Sensor), Xindong Lian Ke (went public this morning), Huayi Technology (IMU designated by multiple automotive manufacturers), pure MEMS-IMU.
Second Tier: StarNet, Huace Navigation, Beidou Navigation, Ligong Navigation, etc., mainly laser and fiber optic, with less MEMS, having strong military relevance or impacts on expansion to specific customers.
Third Tier: Minxin Co., Ltd. (only accelerometers), Saiwei Electronics, etc.
Some commonly used combination navigation selections can refer to this article: Combination Navigation Selection.
3. LiDAR
The main content has been detailed in the book “From ROS1 to ROS2 Drone Programming Practical Guide”. Here is a brief description, 3D LiDAR mainly includes:
3.1 Mechanical Scanning LiDAR (Consumer)
This three-dimensional LiDAR uses multiple laser beams emitted simultaneously and directs these laser beams through a rotating scanner or optical components. They can simultaneously acquire point cloud data from multiple angles to improve the efficiency and accuracy of three-dimensional perception. Rotating multi-beam LiDAR usually has higher resolution and longer measurement distances, but is relatively complex and expensive.
3.2 Solid-State LiDAR – Flash (Consumer – Automotive)
Flash LiDAR, also known as panoramic LiDAR, emits a large number of laser beams at once. They use an array of photodetectors to record the reflection time of the laser beams and calculate the distance by measuring the time difference. Flash LiDAR has high speed, panoramic scanning, and short measurement times, but usually has lower resolution and shorter measurement distances.
Flash LiDAR is equivalent to using a flash to emit lasers and using detectors to receive signals. Since it is all solid-state, it is much smaller, but the problem is that it will be overexposed at close distances and cannot measure at long distances (insufficient capability), so currently, flash LiDAR measurements are generally within 50 meters.
3.3 OPA LiDAR (Consumer – Automotive)
OPA LiDAR uses Optical Phased Array technology to direct and control laser beams. They use optical components and phase modulators to adjust the phase and direction of the laser beams and calculate the distance by measuring the time difference. OPA LiDAR has high speed, panoramic scanning, and longer measurement distances while having higher resolution and flexibility.
However, OPA (Optical Phased Array) is similar to millimeter-wave radar using RF signals, focusing with phased arrays to scan. It is achieved on the scale of optical wavelengths, making it more difficult than millimeter-wave technology, and there are companies working on solutions.
3.4 MEMS LiDAR (Consumer – Automotive)
MEMS LiDAR uses Micro-Electro-Mechanical Systems (MEMS) technology to achieve miniaturized laser emitters and detectors. They use tiny vibrating mirrors or arrays to direct laser beams and calculate distances by measuring the time difference. MEMS LiDAR has small size, low power consumption, and lower costs, but usually has shorter measurement distances and lower resolution.
MEMS LiDAR achieves 64 lines by using MEMS micromirrors to scan the pulse emitted from a single laser to form a grid of 64 lines, making it very easy to achieve high resolution with a very small size. In the future, the cost of MEMS LiDAR is expected to be controlled within a thousand RMB.
