ISP Image Signal Processing
-
1. Introduction to ISP Image Signal Processing
-
2. What is the Purpose of ISP?
-
3. ISP Processing Flow and Algorithms
-
3.1 Geometric Distortion of Lenses
-
-
3.3 Exposure Control: Underexposure
-
3.4 Optical Black Clamping
-
-
4. Internal Composition of ISP
-
-
-
1. Introduction to ISP Image Signal Processing
ISP (Image Signal Processing) is primarily used for processing the output signals from front-end image sensors to match different manufacturers’ image sensors. The camera uses an Image Signal Processor (ISP). The pipelined image processing engine can process image signals at high speed. It also includes dedicated circuits for evaluating Auto Exposure / Auto Focus / Auto White Balance. Furthermore, image processing modules developed by THine, such as noise reduction, can achieve the highest image quality for various CMOS sensors.
2. What is the Purpose of ISP?
Pixels are sensitive to certain wavelength groups and are essentially color-agnostic. The method for obtaining a color image is to place a filter (usually a Bayer pattern color filter) on top and then interpolate the colors of adjacent pixels. An 8 MP CMOS sensor does not sense red, green, and blue for each pixel; instead, it senses one color for each pixel, and the ISP guesses the color based on the surrounding colors. This is known as demosaicing, which may be the primary function of the ISP and involves many secret seasoning methods to compute this interpolated image. Additionally, the ISP is responsible for all other internal management tasks, controlling the camera system’s auto-focus, exposure, and white balance. Recently, corrections for lens defects (such as vignetting or chromatic aberration introduced by imperfect lens systems in Instagram) have been added, as well as HDR reconstruction, noise reduction, other filters, face correction, object detection, and color space conversions. There are differences in the functionalities of ISPs, but they essentially act as controllers to convert Bayer data into a workable image array.
3. ISP Processing Flow and Algorithms
The following is a typical digital camera pipeline processing flow:
The ISP processing algorithms generally include: Bayer, Black Level Compensation, Lens Shading Correction, Bad Pixel Correction, Demosaicing, Bayer Noise Reduction, Automatic White Balance Correction, Color Correction, Gamma Correction, Color Space Conversion (RGB to YUV), noise reduction and edge enhancement in YUV color space, color and contrast enhancement, along with automatic exposure control and image compression.
3.1 Geometric Distortion of Lenses
3.2 Lens Vignetting
3.3 Exposure Control: Underexposure
3.4 Optical Black Clamping
3.5 Image Compression
4. Internal Composition of ISP
As shown in the figure below, the ISP internally contains devices such as CPU, SUP IP, and IF. In fact, the ISP can be considered a SOC (System on Chip) that can run various algorithm programs to process image signals in real-time.
CPU: The CPU, or Central Processing Unit, can run various image processing algorithms such as AF (Auto Focus), LSC (Lens Shading Correction), and control peripheral devices. Modern ISPs generally use ARM Cortex-A series CPUs, such as Cortex-A5 and Cortex-A7. SUB IP: SUB IP is a general term for various functional modules that perform specialized processing on images. Common SUB IPs include DIS (Digital Image Stabilization), CSC (Color Space Conversion), VRA (Video Rate Adjustment), etc. Image Transmission Interfaces: There are two main types of image transmission interfaces: parallel ITU and serial CSI (Camera Serial Interface). CSI stands for MIPI CSI, and due to its numerous advantages, MIPI-CSI interfaces are widely used in mobile camera applications for transmitting image data and various custom data. External ISPs generally include both MIPI-CSIS and MIPI-CSIM interfaces, while internal ISPs typically only require the MIPI-CSIS interface. General Peripheral Devices: General peripheral devices refer to I2C, SPI, PWM, UART, WATCHDOG, etc. The ISP includes an I2C controller for reading OTP (One-Time Programmable) information and controlling VCM (Voice Coil Motor), among others. For external ISPs, the ISP itself acts as an I2C slave device. The AP (Application Processor) can control the ISP’s operating mode and retrieve its operational status via I2C.
5. IC ISP Architecture
5.1 NuCORE Sip1270 DBE
5.2 TI TMS320DM270
5.3 DM270 CCD
There are numerous groups in the automotive industry, including complete vehicles, key components, new energy vehicles, intelligent connected vehicles, aftermarket, automotive investment, autonomous driving, vehicle networking, etc. To join a group, please scan the administrator’s WeChat (Please indicate your company name) There is also a financing group for startups; angel round and Series A companies are welcome to join;
