Understanding Automotive Camera Technology, Market, and Future Prospects in 40,000 Words

Source | Abao1990

Join the “Electronic and Electrical Group” by adding WeChat 13636581676, note architecture

1. Pinhole Imaging

In the early Warring States period, Chinese scholar Mozi (468 BC – 376 BC) and his disciples completed the world’s first experiment on pinhole imaging, recorded in the “Mo Jing”: “The scenery arrives, at noon there is an end, and with scenery length. It is said to be at the end.” “Scenery. The light person, warm as shooting, the lower person is high; the higher person is low. The foot covers the lower light, so the scenery is formed above; the head covers the upper light, so the scenery is formed below. At a distance and near there is an end, with light, so the scenery is also in the library.”

This article explains the reason for the formation of inverted images by pinholes and points out the property of light propagating in a straight line. This is the first scientific explanation of the straight propagation of light.

In the Western world, until 350 BC, the ancient Greek scholar Aristotle proposed optical laws, and from then on, Westerners understood the optical principles of pinhole imaging.

2. Camera Obscura

At the end of the 15th century, people created the camera obscura based on the principle of pinhole imaging, which is essentially the prototype of the camera. The Italian Leonardo da Vinci described this camera obscura in his writings, describing how people used this tool for sketching and painting.

In 1553, the Italian Porta published a book titled “Natural Magic” which detailed the use of the camera obscura. Using this tool, one could simply reflect the image onto paper with a pencil, outline it, and then color it to complete a realistic image that conforms to true proportions.

3. Silver Plate Photography

In 1822, the Frenchman Niepce produced the world’s first photograph on light-sensitive material, but the image was not very clear and required an exposure time of 8 hours. In 1826, he took the earliest surviving photograph on a tin base plate coated with photosensitive asphalt through a camera obscura.

In 1838, French physicist Daguerre invented the daguerreotype silver plate photography method, which used a steel plate coated with silver iodide exposed in a camera obscura, then developed with mercury vapor and fixed with ordinary salt. The method produced a metal negative that was very clear and could be permanently preserved. Subsequently, Daguerre made the world’s first camera based on this method, requiring an exposure time of 20-30 minutes.

On August 19, 1839, the French government announced the abandonment of the patent for the silver plate photography invention and made it public. This day is commonly regarded as the beginning of photography.

4. Film and Digital Cameras

In 1866, German chemist Schott and optician Ague invented barium crown optical glass at Zeiss, which produced positive optical photography lenses, leading to rapid development in the design and manufacture of photographic lenses.

With the development of light-sensitive materials, in 1871, dry plates coated with silver bromide appeared, and in 1884, film made of nitrocellulose as the substrate was introduced.

In 1888, Kodak produced a new type of light-sensitive material—a soft, rollable “film.” This was a leap in light-sensitive materials. In the same year, Kodak invented the world’s first portable box camera that installed film. Kodak No. 1 was launched as a household camera, with the slogan “Press the shutter, and I’ll do the rest.” This camera can be said to be the ancestor of consumer cameras.

In 1969, CCD chips were used as light-sensitive materials in cameras installed on the Apollo lunar module in the United States, laying the technical foundation for the electronicization of photographic light-sensitive materials.

In 1981, Sony produced the world’s first camera using CCD electronic sensors as light-sensitive materials after years of research, laying the foundation for the replacement of film with electronic sensors. Following closely, Panasonic, Copal, Fuji, and some electronic chip manufacturers in the United States and Europe invested in the research and development of CCD chip technology, laying the technical foundation for the development of digital cameras.

In 1987, a camera using CMOS chips as light-sensitive materials was born at Casio.

Looking back at the development history of cameras:

5. Explanation of the Internal Structure and Working Mechanism of Camera Modules:

Lens: gathers light and projects the scene onto the imaging medium’s surface. Some are single lenses, while others require multi-layer glass lenses for better imaging effects.

Color filter: The visible light spectrum is what the human eye perceives, while the light spectrum that the imaging sensor can recognize is much larger than what the human eye can see. Therefore, a color filter is added to filter out excess light spectrum, allowing the imaging sensor to capture the actual scene that humans see.

Image COMS sensing chip: the imaging medium that converts the image (light signal) projected by the lens onto its surface into an electrical signal.

PCB substrate: transmits the electrical signals from the image sensor to the backend. For automotive cameras, the PCB here will have more circuits, needing to convert parallel camera signals into serial transmission, which enhances interference resistance.

The working principle of the camera module is that the lens focuses light, then the IR filter removes unnecessary infrared light. At this point, the analog signal enters the sensor COMS chip, which outputs a digital signal through AD conversion. Sometimes an ISP image processing chip is placed on the camera side to process the signal before transmitting it to the host; other times, there is no ISP chip, and the built-in ISP chip on the host processes the image, which greatly improves heat dissipation and reduces radiation.

6. Automotive Camera Architecture:

The structure of the automotive camera is shown in the figure above. When placed outside the vehicle, it needs to form a complete camera. If it is in the vehicle’s DVR, waterproofing is not considered, and it can be assembled into the above camera module.

The above figure is a dissection of the camera module commonly used in automobiles. Besides the outer aluminum shell, sealing ring, and lens, the middle is actually a simple design of several layers of boards, usually including the sensor board, a small board for the image processor, and a serial board. Why is a serial board needed? Because the camera sensor or ISP’s image data output bus usually follows the MIPI CSI standard, which is characterized by high-speed transmission but short transmission distances; otherwise, signal integrity cannot be guaranteed.

Therefore, in vehicles, we need to convert it into high-speed bus standards suitable for long-distance transmission, such as GMSL, so the camera module typically uses a serial board for bus conversion. Additionally, coaxial cables can be used to provide power to the module and transmit image data.

7. Differences Between CCD and CMOS Image Sensors:

CCD (Charge Coupled Device) is a light-sensitive coupling component primarily made of silicon crystal semiconductors. Its basic principle is similar to the solar cells on CASIO calculators, which convert light into stored electric charge through the photoelectric effect. In simple terms, when the CCD surface receives light during shutter opening, the energy of the light is converted into charge. The stronger the light, the more charge is generated, which serves as the basis for judging the intensity of light.

CMOS (Complementary Metal-Oxide Semiconductor) is made of silicon and germanium semiconductors, with N (negatively charged) and P (positively charged) semiconductors coexisting, allowing the current generated by these complementary effects to be recorded and interpreted by the processing chip.

The most significant differences between CCD and CMOS are the location and quantity of amplifiers. Comparing the structures of CCD and CMOS, the position and quantity of amplifiers are the biggest differences.

When CCD is exposed, after the shutter closes or the internal frequency automatically disconnects (electronic shutter), it processes the pixel transfer, transmitting the electric charge signals from each pixel sequentially to the “buffer” (charge storage device), which is then guided to the amplifier next to the CCD for amplification and subsequently sent to the ADC (Analog-to-Digital Converter) for output.

In CMOS design, each pixel is directly connected to an “amplifier,” allowing the photoelectric signal to be amplified before being moved to the ADC for conversion into digital data. Due to structural differences, CCD and CMOS exhibit different performance characteristics. CCD is characterized by maintaining signal integrity during transmission (dedicated channel design), allowing each pixel’s signal to be processed uniformly through a single amplifier, preserving data integrity. In contrast, CMOS has a simpler manufacturing process without dedicated channel designs, necessitating amplification before integrating data from various pixels.

1. Sensitivity Differences: Due to the inclusion of amplifiers and A/D conversion circuits in each pixel of CMOS, the additional devices compress the photo-sensitive area of a single pixel, leading to lower sensitivity compared to CCD at the same pixel size.

2. Resolution Differences: As mentioned in the first point, the complex structure of CMOS pixels results in smaller apertures compared to CCDs of the same size, leading to generally superior resolution for CCD sensors.

3. Noise Differences: Each photo-sensitive diode in CMOS is paired with an ADC amplifier, leading to a higher noise level due to the variability of each amplifier, making it harder to achieve synchronized amplification. In contrast, CCD’s single amplifier setup results in lower noise levels.

4. Power Consumption Differences: CMOS operates in an active mode, where the charge generated by the photo-sensitive diode is amplified directly by the adjacent transistor, while CCD operates in a passive mode, requiring external voltage to move charges to the transmission channel, typically needing over 12 volts, leading to higher power consumption.

5. Cost Differences: CMOS manufacturing processes are more aligned with semiconductor industry standards, allowing for integration of all peripheral components onto a single chip, reducing manufacturing costs and yield losses. In contrast, CCD requires separate transmission channels, which can lead to defects causing signal blockages, resulting in higher manufacturing costs.

6. Other Differences: IPA (Individual Pixel Addressing) is commonly used in digital zooming. CMOS must rely on x, y positioning for zoom processing; otherwise, individual pixel amplifier discrepancies can lead to uneven images. CCD requires specially customized equipment for manufacturing, making it difficult to produce high-pixel CCD components.

In summary, the characteristics of CCD and CMOS suggest that CMOS is more suitable for portable devices like smartphones, while CCD is better suited for professional equipment like DSLRs.

8. Structure and Key Parameters of CMOS Sensors

The function of an image sensor is photoelectric conversion. Key parameters include pixel count, individual pixel size, chip size, and power consumption. Technological processes include front-side illumination (FSI), back-side illumination (BSI), and stacked designs. Below is a brief introduction.

The image sensor externally features a light-sensitive area (Pixel Array), bonding pads, inner circuit, and substrate. The light-sensitive area consists of an array of individual pixels, and when the light signals captured by each pixel are aggregated, they form a complete image.

CMOS chips consist of layers of microlenses, color filters, circuit layers, photo-sensitive elements, and substrate layers.

A cross-section of a CMOS chip.

Due to the varying angles at which light enters each pixel, a microlens is added to each pixel’s surface to correct the light angle, ensuring that light enters the photo-sensitive element surface perpendicularly. This is the concept of chip CRA, which needs to stay within a certain deviation range from the lens CRA.

In terms of circuit architecture, we add the image sensor as a dark box that converts light signals into electrical signals. The dark box’s exterior typically includes power, data, clock, communication, control, and synchronization circuits. It can be simply understood that the light-sensitive area (Pixel Array) converts light signals into electrical signals, which are then processed and encoded by logic circuits in the dark box before being output through data interfaces.

9. Key Parameters of Image Sensors

1. Pixels: Refers to the number of individual pixels within the light-sensitive area, such as 5 Megapixels, 8M, 13M, 16M, 20M; the more pixels, the larger the captured image area and the more detail in the image.

2. Chip Size: Refers to the diagonal distance of the light-sensitive area, usually expressed in imperial units, such as 1/4 inch, 1/3 inch, 1/2.3 inch, etc. The larger the chip size, the higher the material costs.

3. Individual Pixel Size: Refers to the length and width of each photo-sensitive element, also known as the aperture size of a single pixel, e.g., 1.12 microns, 1.34 microns, 1.5 microns, etc. The larger the aperture size, the greater the amount of light energy entering per unit of time, leading to better overall performance and image quality. Individual pixel size is a crucial parameter for image sensors.

Front-Side (FSI) vs. Back-Side (BSI)

Traditional CMOS image sensors are structured in a front-side manner, with layers arranged from top to bottom, including lens layer, color filter layer, circuit layer, and photo-sensitive element layer. When using this structure, light must pass through the circuit layer’s openings to reach the photo-sensitive element layer, which can lead to light loss.

In contrast, back-side illumination moves the photo-sensitive element layer above the circuit layer, retaining only the essential logic circuits of the photo-sensitive layer, allowing light to enter the photo-sensitive layer more directly, reducing light loss (e.g., reflection). Thus, within the same unit time, the single pixel can capture a larger amount of light energy, significantly improving image quality. However, the production process for this structure is more challenging, leading to lower yields and higher costs.

10. Sensor Dynamic Range

The dynamic range of the sensor is the ability of the sensor to reflect both highlight and shadow areas within a single image, indicating the capability of the camera to simultaneously capture bright and dark details. In practical terms, this is the capability to see the brightest and darkest parts of an image.

In summary, the characteristics of CCD and CMOS suggest that CMOS is more suitable for portable devices like smartphones, while CCD is better suited for professional equipment like DSLRs.

The function of an image sensor is photoelectric conversion. Key parameters include pixel count, individual pixel size, chip size, and power consumption. Technological processes include front-side illumination (FSI), back-side illumination (BSI), and stacked designs. Below is a brief introduction.

Image sensors are crucial components in modern automotive camera systems, enabling various functionalities such as lane detection, obstacle recognition, and driver monitoring, contributing significantly to the advancement of autonomous driving technologies.

In conclusion, the automotive camera industry is rapidly evolving, driven by technological advancements and increasing demand for safety and automation in vehicles. With the integration of advanced imaging technologies and AI algorithms, automotive cameras are set to play a pivotal role in the future of transportation.