The series of articles on dual cameras has received a lot of attention from readers. Some readers hope to understand the principles behind dual cameras. I, the editor, have read numerous articles on dual camera principles, most of which have been quite boring, leading to a journey from beginner to abandonment. Here, I will use a simple and straightforward way to discuss the principles of dual cameras, and everyone is welcome to discuss in the comments section. As introduced in previous articles, the applications of dual cameras mainly fall into categories such as distance-related applications, optical zoom, low-light compensation, and 3D shooting and modeling. Each application has different principles, and we will introduce the relevant principles respectively: Original | In-Depth Analysis of Dual Cameras (1): What Can Dual Cameras Do? Original | In-Depth Analysis of Dual Cameras (2): The Ecosystem of Dual Cameras

Distance-Related Applications The human eye can easily locate the distance of an object, but when one eye is closed, the ability to locate significantly decreases.

Dual cameras simulate the application of human eyes.

Simply put, to measure distance, the algorithm calculates the angles θ1 and θ2 between the object being photographed and the left/right cameras, plus a fixed y-value (the distance between the two cameras’ centers), making it quite easy to calculate the z-value (the distance from the object to the camera).

However, this can be easily deduced; if the distance between the two cameras is too small, the measurable distance will be very close. If you want to calculate a long distance, you must increase the distance between the left and right cameras. Optical Zoom Optical zoom mainly uses different FOV (field of view) for the left and right cameras, allowing them to capture different scenes. When the user needs a wide-angle photo, the left camera with an 85-degree field of view is used to achieve a wide-angle effect. When the user needs a telephoto photo, the right camera with a 45-degree field of view is used to achieve a telephoto effect.

To ensure a high overlap of objects captured by the left and right cameras, the optical zoom dual camera module cannot have a large distance like that of the distance application cameras, but rather needs to have the left and right cameras placed as close as possible. Low-Light Enhancement In the second article, I briefly introduced the principle of low-light enhancement. Generally speaking, low-light enhancement involves using one RGBG standard camera and one black-and-white camera without the RGBG filter. The RGBG camera captures the object’s color, while the black-and-white camera captures more light to determine the light intensity of the object being photographed. The two images are then fused to achieve better low-light enhancement. There are generally two fusion methods: 1. Using the black-and-white image as the main image, the color of each pixel obtained from the color image is pasted onto the black-and-white image to fuse the two images. 2. Using the color image as the main image, compensating the light intensity obtained from the black-and-white image onto the color photo to fuse the two images. As for which method is more suitable for fusion, it may vary from person to person, and we will not discuss it here. Similarly, for low-light enhancement, to ensure a high overlap of the objects captured by the left and right cameras, this type of dual camera module also requires them to be as close as possible.

It should be noted that the Huawei P9 actually uses this type of module.

Of course, some industry insiders also say that the effects of this algorithm are not very significant at present. Low-light compensation is indeed very helpful for users, especially when taking night scenes. However, some customers believe that Sony and Samsung’s Dual PD technology is very good, and they prefer to use Dual PD cameras for low-light compensation. Whether dual cameras or Dual PD have better low-light compensation effects can be compared by looking at the Huawei P9 and Samsung’s Galaxy S7 edge, and the answer will be clear.

3D Shooting and 3D Modeling The algorithms for 3D shooting and 3D modeling are somewhat similar to distance applications, but they require higher accuracy and sometimes need to use infrared distance measurement for more accurate distance judgment. I will not go into detail here. ISP Requirements When discussing dual camera algorithms, we must mention ISP (Image Signal Processing). The main function of ISP is to process the signals output from the front-end image sensor, including linear correction, noise removal, bad pixel removal, interpolation, white balance, automatic exposure control, etc. It relies on ISP to restore scene details well under different optical conditions, and ISP technology largely determines the imaging quality of mobile phones. In the feature phone era, ISP was built into the camera, with different pixel cameras paired with ISPs of varying performance. As smartphone camera pixels increase, the demand for ISP performance also rises. If ISP is integrated into the camera sensor, it will inevitably lead to a larger camera module, which may affect the photo quality. Therefore, in the smartphone era, ISP is generally on the main chip SoC. Some brand customers, to achieve better results, do not hesitate to add an additional ISP for better and more professional photography effects. Good photography algorithms need to be paired with good ISPs, and ISP and algorithms complement each other, both being indispensable. Dual cameras have even higher requirements for ISP performance. First, to ensure that the signals from the left and right cameras can be processed simultaneously, a single ISP can no longer meet the needs of dual cameras. This requires a dual-path ISP to achieve this function.

Taking low-light enhancement as an example, the color/black-and-white images enter their respective ISP channels and calibration channels, and then the two images are matched (for example, extracting the same parts of the two images and removing the parts captured by only one camera), and then processed through algorithms such as occlusion, detection, and compensation. Finally, the two images are fused to achieve color enhancement. Of course, in reality, the tasks performed by ISP in conjunction with algorithms go far beyond what is described in this image. I truly do not know, so I will not mislead everyone here. Of course, there is also a small episode in this. After all, there are two ISPs, and there are some differences in processing speed and capacity. To ensure that the two ISPs can sample at the same time, the images captured by the dual camera must be taken at the same time. One solution is to have a synchronization signal pin on the sensor. By connecting the synchronization signals of the two cameras, each time an image is read, a timestamp is added to the image, and the ISP uses the timestamp to ensure that the photos taken by the left and right cameras are captured at the same time and then fused.

Camera Interfaces Generally speaking, the camera interfaces of current smartphones are MIPI interfaces. Previously, mobile platforms only had two MIPI interfaces, one for the front camera and one for the rear camera. To implement dual cameras, the platform must support at least three MIPI interfaces. In fact, on some high-end platforms, to achieve higher pixels, dual-path ISP has already been used (for example, to support a 16M camera, two 8M capable ISPs are used), and such platforms may only have two MIPI interfaces. However, this does not stop engineers from implementing a front single camera + rear dual camera setup.

Indeed, by adding a small switch, dual cameras can be easily achieved. Postscript: After writing three articles on dual cameras, I finally finished. If readers have any questions, please leave your valuable opinions and suggestions in the comments section, or any questions you want to know. If there is an opportunity, I would also like to continue discussing the wonderful world of cameras with you.

Click to read the original text and see more high-paying positions in the semiconductor industry.