Learning Notes (Self-Driving System)This article | Feynman Yang Source | Learning Summary
The path of growth is often fraught with challenges; the true beauty of growth lies not in the ultimate height of life, but in whether we can maintain enough calmness within ourselves during the process of growth and whether our actions can maintain enough resilience.
2.1 Resolution
- Definition: Refers to the number of pixels that the camera sensor can capture, usually expressed as “horizontal pixels × vertical pixels”. The “M” in “2M, 3M, 8M” is an abbreviation for “Megapixel”, representing the total number of pixels in the camera (Total Pixels ≈ Horizontal Pixels × Vertical Pixels).
| Total Pixels | Resolution | Calculation Formula | RGB Image | Usage |
| 1M | 1280×720 (720P) | 1280×720≈921,600 pixels (about 1 million) | 2.6MB | Reversing image, basic ADAS functions; in-car personnel monitoring; streaming rearview mirror; |
| 2M | 1920×1080 (1080P) | 1920×1080≈2,073,600 pixels (about 2 million) | 5.9MB | |
| 3M | 2304×1296 (Near 2K) | 2304×1296≈2,985,984 pixels (about 3 million) | 8.5MB | |
| 5M | 2592×1944 | 2592×1944≈5,038,848 pixels (about 5 million) | 14.5MB | Advanced driver assistance |
| 6M | 3264×1836 | 3264×1836≈5,992,704 pixels (about 5.99 million) | 17.1MB | |
| 8M | 3840×2160 (4K) | 3840×2160≈8,294,400 pixels (about 8.29 million) | 23.7MB |
Note:16:9 ratio: Suitable for video recording, in-vehicle imaging, and other scenarios requiring a wide field of view, in line with the horizontal visual habits of the human eye;The data volume of a single frame image is positively correlated with resolution: the higher the resolution (more pixels), the larger the file size of a single frame image. At the same time, the size of uncompressed images (such as BMP format, RAW format) can be directly calculated using resolution and color depth, with the formula: File Size = Horizontal Pixels × Vertical Pixels × Color Depth ÷ 8 (unit: bytes, 1 byte = 8 bits).BEV perception is primarily based on RGB (3 channels, each channel 8 bits), with a few scenarios possibly using RGBA (including transparency, but in practice, camera images rarely include a transparency channel) or grayscale images (1 channel, less common, as color information is helpful for semantic segmentation and object recognition).🔎Color Depth: Refers to the number of bits used to store color information for each pixel, common types include:
- 8 bits: Grayscale (256 shades);
- 24 bits: True color (RGB three channels each 8 bits, 2⁸×2⁸×2⁸=16.7 million colors, most common);
- 32 bits: True color + transparency channel (RGBA);
- 48 bits: High dynamic range (HDR) images, with more delicate colors.
2.2 Frame RateDefinition: The number of frames captured by the camera in a unit of time, measured in frames per second (FPS).Common specifications: 30fps (mainstream), 60fps (high-end).🔎For example, if a vehicle-mounted camera has a frame rate of 30 FPS, it means it can generate 30 consecutive images (frames) per second, and these frames play continuously to form smooth video for real-time perception of the vehicle’s surrounding environment (such as recognizing obstacles, lane lines, traffic lights, etc.)In vehicle-mounted scenarios, the frame rate directly affects:
- Dynamic object capture: A high frame rate (such as 60 FPS) can capture fast-moving objects (such as pedestrians suddenly crossing the road) more clearly, reducing motion blur;
- Real-time performance: The autonomous driving system needs to make decisions based on camera data; a low frame rate may lead to decision delays, posing safety risks;
- Computational load: A high frame rate generates more data, requiring higher computational power from subsequent image processing chips (such as vehicle-mounted GPUs, FPGAs).
The frame rate of a single camera is not fixed but is a dynamic process; the frame rate we usually refer to is more of an average value over a certain period. As long as the frame interval time is controlled within a reasonable range, it basically meets the actual functional usage requirements. We emphasize the stability of the frame rate rather than a high frame rate, especially for BEV models, where fluctuations in frame rate can cause ghosting, separation, and unsuccessful matching of the perception results from multiple cameras.Factors affecting the frame rate of a single camera:
- Hardware performance: The exposure speed of the camera sensor and the computational capability of the image processing chip (ISP) determine the processing efficiency of a single frame image. If the sensor read speed is slow or the ISP computational power is insufficient, it will lead to longer frame intervals and a decrease in frame rate;
- Resolution: High resolution (such as 8 million pixels) results in larger image data volume, longer processing time, and may limit the frame rate (for example, the same camera can support 60 FPS at 1080P resolution, but may drop to 30 FPS when switched to 4K resolution);
- Lighting conditions: In low-light environments, the camera needs to extend the exposure time to obtain clear images, which will increase the processing time for a single frame and reduce the frame rate (some cameras support “automatic frame rate adjustment” to balance image quality and smoothness);
- Interface bandwidth: The transmission interface (such as MIPI-CSI) between the camera and the vehicle’s main unit has limited bandwidth; high frame rate + high resolution may lead to data transmission congestion, forcing the frame rate to drop to avoid frame loss;
2.3 Field of View (FOV) and Focal LengthDefinition: The field of view (FOV) is a core parameter that measures the range of objects that the camera lens can capture, usually expressed in degrees, reflecting the “width of the lens’s view”. It is an inherent property of the lens optical system. In simple terms, the larger the FOV, the wider the range the lens can “see” (similar to the “peripheral vision” of the human eye); the smaller the FOV, the narrower the view, but the ability to capture details of distant objects is stronger (similar to the focusing effect of the human eye).Classification (by horizontal FOV angle):
- Narrow angle (<60°): Long focal length, suitable for long-distance shooting (such as forward long-distance detection, recognizing traffic lights, distant vehicles).
- Wide angle (60°-120°): Balances near and far, commonly used for front-facing main cameras and side cameras.
- Ultra-wide angle (>120°): Large coverage, suitable for reversing images (to avoid blind spots) and surround view cameras (360° panoramic).
The size of the FOV is determined by the lens design and hardware parameters, with the main influencing factors being: lens focal length, sensor size, and lens optical design.🔎The field of view (FOV) is inversely proportional to the lens focal length:
- Shorter focal length → larger field of view (wide coverage);
- Longer focal length → smaller field of view (focus on distant objects).
Under the same sensor size, the relationship between focal length and field of view can be calculated using optical formulas (simplified understanding: halving the focal length approximately doubles the field of view); if the sensor size is different, the same focal length corresponds to different fields of view (a larger sensor size with the same focal length results in a larger field of view). This relationship is the core basis for selecting vehicle-mounted cameras—different scenarios have different requirements for “coverage” and “detail resolution”, which need to be balanced through the combination of focal length and sensor size.The design of the FOV for vehicle-mounted cameras is a balance between “functional requirements” and “physical limitations”: the horizontal FOV determines the lateral coverage range, which needs to match the scene’s blind spot requirements; the vertical FOV determines the vertical coverage range, which needs to consider ground details and aerial targets. At the same time, it needs to be designed in coordination with the camera’s resolution and frame rate (for example: a large FOV requires higher resolution to avoid image blurriness), ultimately achieving “no blind spots, high reliability” in environmental perception.
| Camera Type | Focal Length | Horizontal FOV | Usage |
| Front Main Camera | 8mm – 16mm | Horizontal field of view approximately 30° – 60° | Responsible for detecting distant targets (such as vehicles, pedestrians, traffic signs, traffic lights 100-200 meters away), while also covering mid-range (such as lane lines, obstacles 5-30 meters ahead), serving as the “main eye” for ADAS (Advanced Driver Assistance Systems) and autonomous driving. |
| Rear View Camera | 2.8mm – 6mm | Horizontal field of view approximately 90° – 140° | Ensures the field of view is wide enough (to avoid missing obstacles on both sides when reversing), while also considering clarity at mid-range (beyond 5 meters). |
| Side View Camera | 4mm – 8mm | Horizontal field of view approximately 50° – 90° | Focal length slightly longer than surround view, moderate field of view, capable of covering side blind spots without causing distant vehicle details to be blurred due to excessive angle. |
| Surround View Camera | 2.5mm – 4mm | Horizontal field of view 120° – 170°, mostly fisheye lenses | Captures the 360° environment around the vehicle (front, rear, left, right), used for parking assistance and blind spot monitoring, requiring an ultra-wide field of view to cover the near areas around the vehicle (such as 1 – 5 meters outside the vehicle), and sufficient overlap in the stitching area. |
| In-Cabin Camera | 3mm – 6mm | Horizontal field of view approximately 60° – 100° | Moderate field of view, avoiding distortion caused by too short a focal length (affecting facial feature recognition), while ensuring complete coverage from the driver’s head to shoulders. |
The design of the field of view and focal length for vehicle-mounted cameras is essentially a reverse deduction process of “scene function → coverage range (FOV) → resolution (focal length)”: first clarifying where the camera needs to “see and what to see clearly”, then defining the coverage boundary through FOV, and finally determining the focal length parameters based on the inverse relationship between FOV and focal length, while optimizing based on distortion control, multi-camera collaboration, installation environment, etc., ultimately achieving “no blind spots, high reliability” in environmental perception.2.4 Sensor Type and SizeVehicle-mounted cameras primarily use CMOS sensors, and sensor size is a key parameter affecting imaging quality: larger sensors are suitable for core scenarios such as front view, focusing on low-light performance and dynamic range; medium and small sensors are often used for surround view and rear view scenarios, balancing cost and basic functionality. When selecting, it is necessary to comprehensively consider the functional requirements of specific scenarios, installation space, and cost budget.
- Sensor Types:
- CMOS (Complementary Metal-Oxide-Semiconductor): Converts light signals into electrical signals through pixel points on a semiconductor chip, with each pixel independently completing photoelectric conversion and signal amplification, high integration; mainstream choice, low power consumption, high integration, suitable for vehicle-mounted environments.
- CCD (Charge-Coupled Device): Reads signals line by line through charge transfer, theoretically better imaging consistency and low-light performance; slightly better imaging quality, but higher power consumption and cost, now used less frequently.
Sensor Size: Sensor size refers to the physical size of the light-sensitive element (CMOS chip), usually expressed in inches (in) (Note: the “inches” here is an industry convention approximate value, not a strict length unit).
- 1/4 inch and below: Very small volume, low cost, but small light-sensitive area, poor low-light performance, used only in scenarios with very low image quality requirements (such as some simplified surround view cameras).
- 1/3.2 inch: Commonly used in miniaturized cameras, balancing size and basic image quality, often found in entry-level vehicle surround or rear view cameras.
- 1/2.7 inch: Mainstream medium size, widely used in surround and rear view cameras, can meet clarity requirements under general lighting conditions.
- 1/2.5 inch: Slightly larger than 1/2.7 inch, with slightly stronger light-sensitive capability, commonly used in front or side cameras with certain low-light performance requirements.
- 1/1.8 inch and above: Large size sensors, large light-sensitive area, significantly improved low-light performance and dynamic range, suitable for front main cameras (such as core sensors responsible for lane line detection, target recognition), supporting higher resolutions (such as 8 million pixels and above).
Impact of Size on Imaging
- Light-sensitive capability: The larger the size, the larger the unit pixel area can be designed (or more light can be accommodated under the same pixel), resulting in less noise and clearer images in low-light environments.
- Dynamic range: Larger sensors can better handle scenes with both strong light and shadows (such as driving against the light), avoiding overexposure or underexposure.
- Field of view and focal length: Under the same lens focal length, larger sensors can cover a wider field of view (or use longer focal lengths under the same field of view to enhance distant detail resolution).
- Size and cost: The larger the size, the larger the camera module, the higher the cost, and the stricter the requirements for vehicle installation space (such as front grille, rearview mirror).
-
Autonomous Driving System Learning – Operating Systems;
-
Intelligent Driving System Learning – Basic Concepts Notes;
-
Intelligent Driving System Learning – Basics Module Overview;
References
This article is authored byNon-Architectural Automotive Electronics and Electrical, Author: Feynman Yang.Some images in the text and cover images are sourced from online screenshots. If there are any copyright issues, please contact within 30 days of publication.
• Phone: Not disclosed • Email: [email protected]• Zhihu: Xiaofoye Xiuluodao • LinkedIn: feynman,yang• WeChat: Not disclosed• Twitter: feynman,yang