Reflecting on these years of work, having tasted the hardships of society, from having nothing at first to 300,000, from 300,000 to 2 million, from 2 million to 13 million, this is not a boast, I just want to tell my friends through my own experience that “the higher the pixel of the phone, the clearer the photos taken“

Camera Structure

When I was in middle school, once I ranked 7th in my grade, and the school rewarded me with 200 yuan. I was very happy and immediately wanted to buy a camera. I wanted to record everything I saw at that time, but unfortunately, the cheapest one I liked cost more than 200. Then my mom told me to save the money with her, and when I had money later, she would buy it for me. Then, you might want to know what happened next, but there was no next. The first camera I had was the bulky phone my brother bought for me in college, and when the ringtone rang, even a sleeping pig could be awakened.

The first camera was the manual focus type, which means rotating the motor to allow the photosensitive chip to receive the best signal, commonly known as focusing.

The Original Camera Imaging Principle

Speaking of imaging principles, we must mention pinhole imaging, which is how we have such a magical thing as a camera.

Pinhole imaging is a relatively old method. With the development of technology today, new methods have replaced pinhole imaging because the hole in pinhole imaging is very small, which also causes many limitations in shooting.

Modern Camera Working Principle

The working principle of a camera is that external light passes through the lens, is filtered by a color filter, and then illuminates the sensor. The sensor converts the light transmitted from the lens into an electrical signal, which is then converted into a digital signal through an internal AD converter. If the sensor does not integrate DSP, it is transmitted to the baseband via DVP, and the data format at this time is RAW DATA. If DSP is integrated, the RAW DATA is processed through AWB, color matrix, lens shading, gamma, sharpness, AE, and de-noise (usually done at the CPU ISP end), and then outputs YUV or RGB format data. Finally, it will be sent to the framebuffer by the CPU for display, so we see the scene captured by the camera.

First, we use our eyes to determine whether the person we are photographing is a good-looking person, then let this good-looking person pose a bit more handsomely, and then press the shutter, the light shield lifts, and the light shines on the photosensitive element. The photosensitive element is an analog image, converted into a digital signal through ADC, and stored in the storage chip.

Camera Hardware Schematic

Hardware schematic analysis

POWER:

VDD_CAMA28 —-2.8v analog voltage of the camera

VDD_CAMD28 —-1.5v working voltage of the camera

VDD_CAMIO28 —-2.8v digital voltage of the camera’s GPIO port

OUTPUT:

CAM_DATA: data pin of the camera. The output formats can be YUV, RGB, JPEG

CAM_VSYNC: frame synchronization signal pin of the camera. One VYSNC signal ending indicates that one frame (i.e., one image) of data has been output.

CAM_HSYNC: line synchronization signal pin of the camera. One HSYNC signal ending indicates that one line of data has been output.

CAM_PCLK: pixel synchronization signal pin. One PCLK signal ending indicates that one data has been output.

INPUT:

CAM_PWDN: enable pin of the camera. When the camera is in PWDN mode, all operations on the camera are ineffective. Therefore, before RST, the PWDN pin must be set to normal mode.

CAM_RST: reset pin of the camera. This is a hard reset mode, generally the pin is set to low, and the camera is in a hard reset state, restoring all IO ports of the camera to factory default state. Only after XCLK is turned on and RESET is set to low, is the hard reset effective; otherwise, the reset is ineffective.

CAM_MCLK: working clock pin of the camera. This pin provides the working clock for the camera to the main control.

CAM_I2C: communication pin between the camera and the main control. The main communication interface of the camera is I2C, and the data transmission is MIPI.

During software debugging, the general process of the camera is:

The first step: power on the sensor and output clock signals;

The second step: then communicate via I2C, sending addresses for the CPU to locate the camera;

The third step: when opening the camera, issue commands via I2C to adapt and prepare the sensor registers;

The fourth step: output camera data, collected by the CPU via MIPI;

Types of Sensors

4.1 Common types of camera sensors mainly include two types

CCD (Charge Coupled Device), which is widely used in most digital cameras. This is a special semiconductor material composed of many independent photosensitive elements arranged in a matrix. Light passes through the lens and strikes the CCD, converting it into charge. The amount of charge on each element depends on the intensity of the light it receives. When the photographer presses the shutter, the CCD transmits the information from each element to the analog/digital converter, converting the analog electrical signal into a digital signal, which is then compressed in a certain format and stored in the cache, thus completing the entire process of taking a digital photo.

CMOS (Complementary Metal-Oxide-Semiconductor), which occupies an important position in semiconductor technologies such as microprocessors and flash memory, is also a type of semiconductor that can sense changes in light. Its component elements are mainly silicon and germanium, and it uses negatively and positively charged transistors to achieve basic functions. The current generated by these two complementary effects can be recorded and interpreted as images by processing chips. Due to the relatively simple structure of CMOS, and the same production process as existing large-scale integrated circuits, production costs can be reduced. Theoretically, CMOS signals are charge signals at the point level, while CCD signals are current signals at the line level. In comparison, the former is more sensitive, faster, and more energy-efficient. Currently, the development of CMOS technology is still immature, and high-quality CMOS is only used in some professional digital cameras, while low-end digital cameras often use cheap low-end CMOS, resulting in generally poor imaging quality. Therefore, when purchasing consumer-grade digital cameras, it is recommended that users choose products with CCD as the image sensor.

4.2 There are two modes of image acquisition data

Color Filter Array—CFA. Image sensors use certain modes to acquire image data, commonly used are BGR mode and CFA mode.

BGR mode: BGR mode is a type of image data mode that can be directly displayed and processed for compression, determined by the values of R (Red), G (Green), and B (Blue) to define one pixel. For example, the SUPER CCD image sensor used by Fujifilm digital cameras adopts this mode. Its advantage is that the image data generated by the image sensor can be directly displayed and processed without interpolation, resulting in the best image effect, but it is costly and commonly used in professional cameras.

CFA mode: To reduce costs and shrink size, most digital cameras on the market use CFA mode, which covers a layer of Color Filter Array (CFA) over the pixel array surface. The most widely used CFA format is the Bayer format, satisfying the GRBG rule, where the number of green pixels is twice that of red or blue pixels. This is because the peak sensitivity of human eyes to visible light spectrum is located in the mid-band, corresponding to the green light spectrum component. In this mode, image data is represented by one value among R, G, and B, while the other two color values are missing, resulting in a mosaic image. To obtain a full-color image, the color information of surrounding pixels is used to estimate the missing two colors, a process known as color interpolation.

Conclusion

This is an introduction to the working principle of camera hardware. In the following articles, we will gradually analyze how to drive the camera to work in conjunction with code.