Lei Feng Network: This article is authored by 7Tens, the initiator of the WeChat public account: ICshare.

| Dual-Camera Structure

High-end smartphones have fewer highlights, and the Lei Tu Tu score can no longer showcase the advantages of a smartphone. Thus, the more powerful dual-camera has made its appearance. The dual-camera mentioned here is not like previous smartphones with one camera in the front and one in the back; instead, it uses two rear cameras to simulate a pair of human eyes, achieving more photographic functions and better photographic effects.

Generally speaking, the current smartphone camera interfaces are MIPI (Mobile Industry Processor Interface) interfaces. Previously, mobile platforms only had 2 MIPI interfaces, one for the front camera and one for the rear camera. Implementing dual cameras requires the platform to support at least three MIPI interfaces. In fact, on previous high-end platforms, to achieve higher pixel counts, dual ISP (Image Signal Processing) has already been used; for example, to support a 16M camera, two 8M capable ISPs would be used. These platforms might only have 2 MIPI interfaces, but this does not prevent engineers from implementing a single front camera + dual rear cameras.

| Applications of Dual-Cameras

So the question arises, what can dual cameras do?

1. Dual cameras can measure distance and create distance-related applications

The human eye can easily locate the distance of an object, but when one eye is closed, the ability to locate decreases significantly. Dual cameras simulate the human eye’s functionality. Simply put, to measure distance, algorithms calculate the angles θ1 and θ2 between the object being photographed and the left/right cameras, and by adding a fixed y value (the distance between the two cameras), it becomes easy to calculate the z value (the distance from the object to the camera).

However, it is easy to deduce that if the distance between the two cameras is too small, the calculable object distance will be very close. To calculate further distances, the left and right cameras must be spaced further apart.

As shown in the image above, dual cameras can determine the distance of the object being photographed through algorithms, making it easy to create special effects such as:

A. Background Blur

One of the most outstanding features of DSLR cameras is the large aperture. Since dual cameras can measure the distances of different objects being photographed, focusing on objects needing a large aperture while blurring other objects at different distances can easily achieve a large aperture effect.

(Original Image)

(Focusing on the beautiful woman with a blurred background and mushroom in hand)

(Focusing on the mushroom with a blurred background and beautiful woman)

B. Background Replacement

By measuring distances, it is possible to extract the subject from the photograph and change the background, making it easier than Photoshop to cut out images.

C. Background Effects

By measuring distances and distinguishing between the subject and background, it is easy to apply any processing to the background, which will not be elaborated here.

D. Distance Measurement

This image clearly indicates the distances of different objects, with the distance information marked in different colors. When the AP obtains the distance information of different objects, it can perform the various functions mentioned above.

2. Dual cameras can achieve optical zoom

Optical zoom mainly involves using different FOVs (field of view) for the left and right cameras, allowing the two cameras to capture different perspectives. When users need wide-angle photos, the left camera with an 85-degree view is used to capture the wide-angle effect. When users need telephoto photos, the right camera with a 45-degree view is used to capture the telephoto effect.

To ensure high overlap of the objects captured by the left and right cameras, the optical zoom dual-camera module cannot have too much distance like the distance-measuring camera module; instead, the left and right cameras should be as close as possible.

If the FOVs of the two cameras are different, one being large and the other small, then by using algorithms to achieve the effect between the two optical lenses, optical zoom can be easily accomplished.

If not using dual cameras, enlarging the image results in unclear text

If using dual cameras, enlarging the image still keeps the text clear

This image merges wide-angle and telephoto images, calculating the intermediate state photo to avoid detail distortion

3. Enhanced low-light performance

Generally speaking, enhancing low-light performance involves using two cameras: one standard RGBG camera and one monochrome camera without RGBG filters. The RGBG camera is used to capture the color of the object, while the monochrome camera captures more light to judge the light intensity of the object being photographed. The two images are then merged to achieve better low-light enhancement.

Currently, there are two fusion methods:

Using the monochrome image as the main image, the color of each pixel obtained from the colored image is pasted onto the monochrome image, merging the two images.

Using the colored image as the main image, compensating the light intensity obtained from the monochrome image onto the colored photo, merging the two images.

As for which method is more suitable for fusion, it may vary from person to person, so it will not be discussed here.

Similarly, to enhance low-light performance, the dual-camera module should also be as close as possible to ensure high overlap of the objects captured.

It should be noted that the Huawei P9 actually uses this type of module.

Of course, some industry insiders also state that the effects of this algorithm are not very obvious at present. Low-light compensation is indeed very helpful for users, especially when shooting night scenes. However, some customers believe that Sony and Samsung’s Dual PD technology is very good, preferring to use Dual PD cameras for low-light compensation.

Whether dual cameras or Dual PD provides better low-light compensation can be compared by looking at the Huawei P9 and Samsung Galaxy S7 edge.

This generally uses a colored + monochrome camera. The monochrome camera captures the light intensity of the image to provide low-light compensation.

4. 3D photography and 3D modeling

3D photography and 3D modeling algorithms are somewhat similar to distance applications, but they require higher precision and sometimes even need to use infrared distance measurement for more accurate distance judgment.

When discussing dual-camera algorithms, one cannot forget about ISP (Image Signal Processing). The ISP mainly processes the signals output from the front image sensor, with primary functions including linear correction, noise removal, bad pixel removal, interpolation, white balance, and automatic exposure control. Relying on ISP is essential to restore scene details under various optical conditions, and ISP technology largely determines the imaging quality of smartphones.

In the feature phone era, ISPs were embedded in the cameras, with different pixel cameras paired with different performance ISPs. As smartphone camera pixels increase, the performance requirements for ISPs also rise. If ISPs are integrated into the camera sensors, it inevitably leads to larger camera modules, potentially affecting photo quality. Therefore, in the smartphone era, ISPs are generally on the main chip SoC. Some brand customers even spare no expense to add an ISP to achieve better and more professional photography effects.

Good photography algorithms need to be paired with good ISPs; ISPs and algorithms complement each other and are both indispensable. Dual cameras have even higher demands on ISP performance. Firstly, to ensure that the signals from the left and right cameras can be processed simultaneously, a single ISP can no longer meet the needs of dual cameras. This requires dual ISPs to achieve this functionality.

Taking low-light enhancement as an example, the colored/monochrome images enter their respective ISP channels and calibration channels, and then the two images are matched (e.g., extracting the same parts from both images and removing parts captured by only one camera), followed by processing related images through algorithms such as occlusion, detection, and compensation. Finally, the two images are merged to enhance color. Of course, the actual tasks performed by ISP in conjunction with algorithms go far beyond what is described in this image.

Of course, there is also a small interlude here. Since there are two ISPs, there can be some differences in processing speed and capability. To ensure that the two ISPs sample at the same time, the images captured by the dual cameras must be taken at the same time. One solution is to have a synchronization signal pin on the sensor. By connecting the synchronization signals of the two cameras, each time an image is read, a timestamp is added to the images, and the ISP ensures that the photos from the left and right cameras are taken simultaneously based on the timestamps, and then merges them.

Unlike ordinary 3D movie shooting, the two cameras on smartphones cannot create sufficient visual disparity during the image capture process due to the distance between the two cameras being different from that of the human eye. To achieve a more pronounced 3D visual effect, algorithms often need to enhance the effect.

Since distance can be measured, subsequent dual cameras can not only achieve 3D photography but also 3D modeling. At that point, I believe the importance of dual cameras will become even more significant.

Other enhancement effects, such as HDR and increased resolution, can also be achieved by single cameras, but dual cameras can improve these effects, so they will not be listed here.

Conclusion:

Currently, these several functions are the most common functionalities of dual-camera smartphones. Background blurring/replacement and low-light effects bring users more photographic effects; optical zoom allows us to experience the zoom capabilities of cameras; but personally, I believe the most exciting future feature will be the 3D functionality.

This year, with VR being so popular, where do the VR materials come from? It still relies on the optimization of dual-camera algorithms. If the algorithms for 3D photography and modeling mature, dual cameras will become even more popular.

| The Ecosystem of Dual Cameras

In the previous text, we discussed the applications and principles of dual cameras. Now let’s focus on the dual-camera industry chain.

1. Dual-camera Algorithm Suppliers

Since algorithms need to work in conjunction with ISPs, algorithms and ISPs complement each other. To develop good algorithms, good ISPs are also necessary.

As main platform suppliers, Qualcomm/MediaTek have their own ISPs and have also developed dual-camera algorithms. However, the effectiveness of these algorithms remains to be tested by the market.

As sensor suppliers, Sony, Samsung, and OV are also actively developing dual-camera algorithms, but there have not yet been any mass-produced products seen. However, in the feature phone era, ISPs did not exist. When these camera sensor suppliers produced 2M/5M cameras, they had to pair them with their own ISPs, so these suppliers have experience in developing ISPs. Therefore, developing dual-camera algorithms is relatively easier for them.

Last year, Apple acquired Linx, which also holds patents and algorithms for multi-camera systems. Whether they will use them in their dual-camera models will depend on this year’s iPhone 7. Theoretically, Linx has enough algorithms, so Apple does not need to buy dual-camera IP licenses from any major manufacturers.

In addition to these platform and sensor suppliers, smartphone brand manufacturers will also develop their own dual-camera algorithms. Other suppliers will be listed one by one:

Founded in Taiwan in 1996, Huajing mainly develops independent ISP chips. Some high-end smartphones, cameras, and automotive applications have used their ISPs. Dual-camera smartphones also utilize their ISPs. They have made achievements in distance applications, optical zoom, and low-light compensation.

An Israeli company. Its algorithm advantages mainly lie in optical zoom and low-light compensation. In terms of depth perception, they also have some research. According to media reports, HTC has models that use their algorithms.

ArcSoft was founded in 1994, headquartered in the United States, with technical centers in Shanghai, Hangzhou, and Nanjing. ArcSoft’s strengths lie in optical zoom and low-light compensation. The second image of optical zoom and low-light compensation mentioned in the previous text comes from ArcSoft.

Shanghai X-Chip is a company established in 2011, mainly engaged in the research and development of image processors, currently focusing on automotive applications. As one of the few companies developing independent ISPs, they are also developing dual-camera algorithms and ISPs. Once the dual-camera market takes off, X-Chip could become a very promising dark horse.

Dual cameras are still relatively new, so different companies have their own strengths and weaknesses in terms of algorithms. However, based on the effects of currently released dual-camera smartphones, all algorithms still have room for improvement. Once various algorithms improve, dual cameras will inevitably become standard features in smartphones.

2. Dual-camera Sensor Suppliers

Since dual cameras need to synchronize timestamps when taking pictures, the camera sensor must have a synchronization signal. Currently, sensor suppliers with this synchronization signal include Sony, Samsung, OV, and GeKong Micro. Therefore, dual-camera sensors mainly use products from these suppliers.

3. Dual-camera Module Suppliers

Currently, many manufacturers can produce dual-camera modules, including Lite-On, Sunny Optical, Lens Technology, Namuga, O-Film, Samsung Electro-Mechanics, and Q Technology. However, only Lite-On, Sunny Optical, and Lens Technology have mass production experience. Huawei’s models mainly use Lite-On and Sunny Optical modules. O-Film and Samsung Electro-Mechanics, with their strong factory capabilities, are also aggressively entering the dual-camera module market. Namuga maintains good communication with various algorithm companies and is a supplier for Samsung smartphones, gradually gaining strength in the dual-camera module field.

However, different functionalities have different requirements for modules. Continuing with the four functionalities mentioned earlier:

Distance-related applications generally involve using different sizes for cameras, such as common camera specifications like 13M+2M or 13M+5M. As shown below:

Or by placing the two cameras further apart, as shown below:

Through these two methods, it becomes easier to calculate the distance from the object being photographed to the lens.

Optical zoom dual-camera modules primarily need the two cameras to have different FOVs, as shown below:

Different FOVs produce different focus points, and through algorithms, optical zoom functionality can be achieved.

General low-light enhancement is typically achieved through one RGB full-color camera and one monochrome camera, as shown below:

In terms of algorithms, the right monochrome camera reads the amount of light entering, which compensates for the color of the left RGB camera.

3D photography and modeling are somewhat similar to distance-related modules, but 3D requires higher precision in distance measurement. In this case, the two cameras should be spaced further apart, and some may use external infrared assistance for distance measurement to ultimately achieve 3D photography and modeling.

Challenges in Dual-Camera Module Manufacturing:

There are generally two methods for dual-camera construction: shared substrate or shared bracket, as shown below:

If a shared substrate is used, both camera sensors can be placed on the same substrate, and one FPC can be drawn from this substrate. If a shared bracket is used, as shown in the image, the sensors are fixed by the bracket, and each sensor has its own substrate and FPC.

Advantages of shared substrates: both sensors can sit on the same panel, and they are drop-resistant.

Disadvantages of shared substrates: low yield, resulting in high costs.

Advantages of shared brackets: higher yield, better price.

Disadvantages of shared brackets: as two independent sensor modules, they require AA calibration to ensure they are on the same plane, which is challenging and less drop-resistant.

In summary, both methods have their pros and cons, and currently, only Huawei uses the shared substrate method. Regardless of the method, the yield remains low, leading to high costs.

| The Future of Dual Cameras

If algorithms improve and module yields increase, dual cameras still have many advantages worth pursuing.

However, due to different functionalities requiring different module placements, dual cameras may currently be unable to fully meet everyone’s camera requirements.

When Google launched Project Tango, they cleverly proposed a three-camera concept: when distance measurement and 3D modeling are needed, two cameras can be used that are spaced far apart; for optical zoom and low-light compensation, two cameras can be used that are spaced close together. This depth sensor can even incorporate infrared for more accurate distance measurement.

Regardless, with the evolution of dual-camera algorithms and the increasing demand for VR, the role of cameras is becoming more significant. Dual cameras, and even smartphones with dual front cameras + triple rear cameras, are likely to become increasingly common in the coming years.

Click“Read Original” to register.