Evaluation Metrics and Scene Requirements for Vehicle-Mounted Camera ISPs

Study Notes (AI&ADS-Driving System)This article | Feynman Yang Source | Learning Summary

The path of growth is often fraught with challenges; the true beauty of growth lies not in the ultimate height of life, but in whether we can maintain enough calmness within ourselves during the process and whether our actions can maintain sufficient resilience.

Evaluation Metrics and Scene Requirements for Vehicle-Mounted Camera ISPs

3.6. Basic Metrics for Image Quality Output from ISPsAcceptance scenarios for ISP image quality in normal vehicle-mounted scenes and corresponding processing logic🔎When conducting quantitative analysis of images processed by vehicle-mounted camera ISP modules, it is necessary to evaluate key parameters using professional instruments (such as image analyzers, spectrometers) and algorithmic tools (such as MATLAB, OpenCV). These parameters directly reflect the ISP’s image processing capabilities and output quality.1. Basic Optical Quality Parameters

Performance Parameter	Testing Method	Automotive Standards
Dynamic Range (DR)	Capture a standard target with over 20 levels of gray (such as X-Rite ColorChecker HDR), calculate the ratio of the maximum distinguishable brightness value to the minimum brightness value in the image, measured in decibels (dB), using the formula:DR(dB) = 20×log10(Maximum Brightness/Minimum Brightness).	Typically ≥120dB, ensuring clarity of details in both the sky and ground during backlighting.
Signal-to-Noise Ratio (SNR)	Under uniform lighting, capture a neutral gray board, calculate the ratio of the average brightness of the image (signal) to the standard deviation of pixel values (noise), measured in dB, using the formula:SNR(dB) = 10×log10(Signal Power/Noise Power).	In low light conditions (e.g., 5lux) ≥30dB, to avoid noise masking target details (e.g., pedestrian outlines). 0.1lux (moonlight) ≥25dB
Noise Power Spectrum (NPS)	Perform a Fourier transform on images of uniform areas, analyze the distribution of noise energy across different frequencies, and assess the ISP’s noise reduction algorithm’s effectiveness against noise in different frequency bands (e.g., high-frequency noise corresponding to detail areas).

2. Color and White Balance Parameters

Performance Parameter	Testing Method	Automotive Standards
Color Accuracy	Capture a 24-color standard color card (such as X-Rite ColorChecker Classic), calculate the Euclidean distance between the Lab color space coordinates of each color block and the standard values, which isDelta E = √[(ΔL*)² + (Δa*)² + (Δb*)²].	The deviation of the ISP output image’s colors from the real scene, with the core metric being Delta E (color difference value). Automotive standard: Delta E ≤5 (indistinguishable to the human eye), to avoid misjudgment of traffic light and sign colors (e.g., red light appearing orange).
White Balance Error	Under standard light sources (such as D65 daylight, A source incandescent light), capture a white target, calculate the deviation in RGB channel ratios of the white area in the image, or directly measure the color temperature deviation (in K), with a requirement of deviation ≤500K.	ISP’s color temperature correction accuracy for different light sources (daylight, streetlights, tunnel lights), reflecting the true reproduction ability of white objects.
Color Uniformity	Capture a uniform color card, calculate the Delta E difference between the center of the image and the four corner areas, with a requirement of difference ≤3.	Capture a uniform color card, calculate the Delta E difference between the center of the image and the four corner areas, with a requirement of difference ≤3.

3. Sharpness and Detail Retention Parameters

Performance Parameter	Testing Method	Automotive Standards
Modulation Transfer Function (MTF)	Capture a black and white stripe target containing different frequencies (e.g., 10-200 line pairs/mm), calculate the contrast (modulation depth) of the stripes after ISP processing compared to the original contrast, with MTF values closer to 1 indicating better detail retention (e.g., lane line edges, sign text).	ISP output image’s color deviation from the real scene, with the core metric being Delta E (color difference value). Automotive standard: Delta E ≤5 (indistinguishable to the human eye), to avoid misjudgment of traffic light and sign colors (e.g., red light appearing orange).
Over-sharpening/Under-sharpening	Analyze pixel value distribution at high-contrast edges (e.g., black and white junctions); if abnormal bright/dark pixels (halos) appear on both sides of the edge, it indicates over-sharpening; if the edge transition is too smooth (contrast lower than 30% of the original scene), it indicates under-sharpening.	ISP’s color temperature correction accuracy for different light sources (daylight, streetlights, tunnel lights), reflecting the true reproduction ability of white objects.
Artifact Suppression	Ghosting: Capture HDR images of moving targets (e.g., moving vehicles), calculate the length of ghosting at the edges of the target after multi-frame fusion (requirement ≤1 pixel). Color artifacts: Detect the proportion of abnormal colored pixels in pure color areas (e.g., red billboards) (requirement ≤0.1%).

4. Brightness and Contrast Parameters

Performance Parameter	Testing Method	Automotive Standards
Luminance Uniformity	Capture a uniform white board, calculate the ratio of average brightness in the corner areas of the image to the center area, with a requirement of ratio ≥85% (corner brightness not lower than 85% of the center).	N/A
Auto Exposure Accuracy	Under preset lighting conditions (e.g., 10lux, 1000lux, 10000lux), calculate the deviation rate of the average brightness of the ISP output image from the target brightness (e.g., 128/255), with a requirement of deviation ≤10%.	N/A
Local Contrast	Select typical targets (e.g., pedestrians, vehicles) and their background areas in the image, calculate the brightness difference between the two and the background brightness ratio, with a requirement of ≥30% (to ensure target distinguishability).	N/A

5. Temporal and Dynamic Performance Parameters

Performance Parameter	Testing Method	Automotive Standards
Frame Rate Stability	Continuously capture 1000 frames of images, record the timestamp of each frame, calculate the standard deviation of the frame rate, with a requirement of fluctuation ≤±1fps (e.g., 30fps output must stabilize between 29-31fps).	N/A
Single Frame Processing Delay	Using an oscilloscope or dedicated testing tool, measure the time from when the ISP receives the sensor RAW data to when the processed image is output.	For 30fps processing events, it must be ≤30ms; some L4 autonomous driving requirements ≤10ms.
Motion Blur	Capture a test card moving at a constant speed (e.g., simulating 120km/h), calculate the width of edge diffusion in the direction of motion (in pixels), with a requirement of ≤2 pixels (to avoid blurring of distant vehicle outlines).	At a vehicle speed of 120km/h, motion target edge blur ≤2 pixels.
Exposure Transition Time	Simulate sudden changes in brightness (e.g., from 10000lux to 10lux), record the number of frames for the ISP output image brightness to stabilize within 90% of the target value, with a requirement of ≤3 frames (calculated at 30fps, response time ≤100ms).	During sudden lighting changes (e.g., entering/exiting a tunnel), brightness stabilization time ≤100ms (3 frames @30fps).

6. Reliability and Functional Safety

Performance Parameter	Testing Method	Automotive Standards
Resource Occupation	Evaluate the CPU/GPU occupancy rate of the ISP under full load ≤50%, memory bandwidth ≤70% (must match sensor data throughput, e.g., 4MP@30fps RAW data bandwidth is about 1.5Gbps) and power consumption ≤5W.	Automotive standard chips typically require ≤5W to avoid performance degradation due to high temperatures.
Environmental Tolerance	Temperature cycling test: Cycle 1000 times within a temperature range of -40℃ to 85℃, testing the change rate of image quality metrics (e.g., SNR, Delta E) after each cycle (requirement ≤10%). Vibration test: Simulate vibrations during vehicle operation (e.g., random vibrations of 10-2000Hz), test the stability of ISP hardware connections and image processing (no data loss, no image anomalies).	Pass ISO 16750 (Environmental Conditions for Electrical and Electronic Equipment in Road Vehicles) testing to ensure that mechanical vibrations (e.g., bumpy roads) do not cause communication interruptions between the ISP and camera sensor (e.g., MIPI interface data packet loss).
Long-term Stability	Continuously operate at full load for 1000 hours (simulating 1 year of usage intensity), monitoring the degree of degradation in image quality metrics (e.g., noise increase ≤2dB).	MTBF (Mean Time Between Failures): ≥100000 hours (approximately 11 years).
Electromagnetic Interference Resistance (EMC)	Must not exhibit stripes or flickering in an electromagnetic interference environment.	Comply with ISO 11452 (Electromagnetic Radiation Immunity for Vehicles) standards to prevent electromagnetic signals from vehicle motors, radars, etc., from interfering with the ISP’s image processing (e.g., avoiding stripes and flickering in images).
Functional Safety Compliance	Comply with ISO 26262 functional safety standards, assess the ISP’s “failure modes” (e.g., hardware failures leading to frozen images, color distortion) and their impact on downstream systems, requiring safety mechanisms (e.g., redundancy processing, error checking) to reduce risk levels to ASIL B/D (based on autonomous driving level requirements).	According to the safety risks of application scenarios (e.g., urban roads vs. highways), the ISP must meet the corresponding ASIL (Automotive Safety Integrity Level), typically requiring ≥ASIL B (some high-risk scenarios may require ASIL D).

🔎Requirements for Major Vehicle-Mounted Scenarios:

1. Lighting Change Scenarios

• Backlight Testing: During sunrise/sunset, capture high-contrast scenes of “sun – ground – vehicle” to verify whether the ISP can simultaneously retain details around the sun (without overexposed white spots), ground shadows (without losing pedestrians/vehicles), and vehicle details (without color bias).
• Strong light direct exposure leading to local overexposure should still allow recognition of the edges of overexposed areas (e.g., when facing headlights, retain the outline of the vehicle behind the headlights).
• Tunnel Transition Testing: Simulate the process of a vehicle entering/exiting a tunnel (brightness dropping from 10000lux to 10lux within 10 seconds, then rising back to 10000lux), assess the ISP’s exposure adjustment response speed (requirement ≤3 frames to stabilize) and the image quality of intermediate frames (no flickering, no overexposure/underexposure).
• Low Light/Night Vision Testing: In a 0.1lux (no streetlight at night) environment, capture pedestrians, vehicles, and traffic signs to verify the ISP’s noise reduction capability (no significant color noise) and detail retention (pedestrian outlines, vehicle license plates discernible).

Adverse Weather Scenarios

• Rainy Day Testing: In moderate to heavy rain, assess the ISP’s “rain drop removal” effect (no significant rain drop highlights in the image), and the enhancement effect of road markings (contrast between markings and wet road surface ≥30%).
• Foggy Day Testing: In foggy conditions with visibility of 50-100 meters, verify the ISP’s de-fogging algorithm (clarity of distant vehicles/signs improved by ≥50%, without “over-de-fogging leading to color distortion”).
• Snowy Day Testing: In snowy road conditions (high reflectivity), assess the ISP’s ability to suppress reflections from snow (no large areas of overexposure) and the recognition of targets in the snow (e.g., pedestrians, obstacles by the roadside).

Motion Scenarios

• High-Speed Motion Testing: When the vehicle is traveling at 120km/h, capture vehicles/signs 50-200 meters ahead, assess the ISP’s suppression of motion blur (no trailing edges of targets), and dynamic focusing capability (both near and far targets are clear).
• Sharp Turn Scenarios: When the vehicle makes a sharp turn at 60km/h, verify the ISP’s anti-shake processing (no significant motion blur) and the clarity of edge targets (e.g., pedestrians on the inside of the turn).

• A Brief Overview of the Image Data Processing Process of Vehicle-Mounted Cameras

• A Detailed Explanation of the CMOS Image Sensor of Vehicle-Mounted Cameras

• Optical Components of Vehicle-Mounted Cameras and Conventional Selection

• Camera Models for BEV Perception of Various Cameras

• Continuation of the Overview of Vehicle-Mounted Camera Parameters

• Overview of Vehicle-Mounted Camera Parameters

• Parameter Matching of Camera Sensors Based on BEV Perception Models

• Field of View Design of Camera Sensors Based on BEV Perception Models

• Deployment and Installation Plans for Camera Sensors Based on BEV Perception Models

References

This article is authored byNon-Architectural Automotive Electronics and Electrical, Author: Feynman Yang.Some images and cover images in this article are sourced from online screenshots. If there are any copyright issues, please contact us within 30 days of publication.

© Non-Architectural Automotive Electronics and Electrical WeChat Public Account All Rights ReservedAuthor: A 1992-born man from the North, an engineer in automotive electronic and electrical systems, a 90s generation with three parts romanticism and seven parts pragmatism, hoping to meet more friends and partners in this field and encounter more experts and specialists.

•Phone: Not disclosed                       •Email: [email protected]•Zhihu: Xiaofoye Xiuluodao                            •LinkedIn: feynman,yang•WeChat: Not disclosed•Twitter: feynman,yang