The camera module is an essential part of AI-based embedded systems. There are so many camera modules available on the market that it can be difficult to choose. This article will detail this process to help you make the right choice for embedded applications, including NVIDIA Jetson.
Considerations for Choosing a Camera
Choosing a camera module requires consideration of three key aspects: the sensor, interface (connector), and optical components.
Sensor
Electronic image sensors are divided into two main categories: Charge-Coupled Device (CCD) and Complementary Metal-Oxide-Semiconductor (CMOS). CCD sensors can only read pixel values line by line, shifting each row of pixels to the readout register one by one. CMOS sensors can read each pixel individually and in parallel.
In most cases, CMOS is cheaper and consumes less power without compromising image quality. Because pixels can be read in parallel, CMOS also achieves higher frame rates. However, in certain specialized fields, CCD sensors still dominate, such as in astronomy, where long exposure times and extremely low noise images are required.
Electronic Shutter
There are two options for electronic shutters: global shutter and rolling shutter. A global shutter exposes each pixel to incoming light simultaneously. A rolling shutter exposes rows of pixels sequentially (e.g., from top to bottom), which can lead to distortion (Figure 1).
Figure 1. Distortion of Propeller Blades Caused by Rolling Shutter
A global shutter is not affected by motion blur and distortion caused by moving objects. Because there is only one start time for exposure, it is easier to synchronize multiple cameras using a global shutter. However, sensors with a global shutter are much more expensive than those with a rolling shutter.
Color or Monochrome
In most cases, monochrome image sensors are sufficient for typical machine vision tasks, such as defect detection, presence monitoring, and measurement recording.
In monochrome sensors, each pixel is typically described with 8 bits. In color sensors, each pixel consists of 8 bits for red, 8 bits for green, and 8 bits for blue. The amount of data that color sensors need to process is three times that of monochrome sensors, resulting in longer processing times and slower frame rates.
Dynamic Range
The dynamic range is the ratio between the maximum and minimum signals captured by the sensor. At the upper limit, pixels appear white at higher intensity values (saturation); at the lower limit and below, pixels appear black. Indoor applications require at least 80 dB HDR, while outdoor applications require up to 140 dB HDR.
Resolution
Resolution is the ability of the sensor to reproduce object details. It is influenced by various factors, such as the type of lighting used, sensor pixel size, and optical component performance. The smaller the details of the object, the higher the required resolution.
Pixel resolution refers to how many millimeters each pixel on the image represents. The higher the resolution, the clearer the image. The resolution of the camera or sensor should be able to cover at least two pixels of the feature area.
High-resolution CMOS sensors often have lower frame rates. Even if the sensor can achieve the resolution you need, if it cannot reach enough frames per second, it will not be able to capture the high-quality images you need. Therefore, evaluating the speed of the sensor is very important.
A general rule of thumb for determining the required resolution for a use case is shown in Figure 2. The multiplier “2” indicates that it is generally desired to have at least two pixels on the object for successful detection.
Figure 2. Required Sensor Resolution
Depends on the field of view and the size of the target features
For example, suppose there is an image of a boxer’s eye being injured.
-
Resolution = 2 × (2000/4)
-
Field of View (mm) = 2000 mm
-
Target Feature (eye) Size (mm) = 4 mm
Based on the calculation of 1000 x 1000, a 1 million pixel camera should be sufficient to detect the eye using CV or AI algorithms.
Note that the sensor consists of multiple rows of pixels. These pixels are also referred to as “photons.” The number of photons collected by the pixels is proportional to the pixel size. Choosing larger pixels may seem tempting, but it is not always the best choice.
Table 1. Advantages and Disadvantages of Small Pixels and Large Pixels
Back-illuminated sensors maximize the amount of light captured and converted by each photodiode. In front-illuminated sensors, metal wires above the photodiode block some photons, reducing the amount of light captured.
Figure 3. Cross-Section of Front-Illuminated Structure (Left) and Back-Illuminated Structure (Right)
Frame Rate and Shutter Speed
Frame rate refers to the number of frames captured per second (or images captured) (FPS). Frame rate should be determined based on the number of detections required per second. This is related to shutter speed (or exposure time), which is the time the camera sensor is exposed to capture an image.
Theoretically, the maximum frame rate equals the reciprocal of the exposure time. However, due to delays caused by frame reading, sensor resolution, and interface data transmission rates, including cables, the achievable FPS will be lower than this value.
By increasing additional lighting and binning pixels, the demand for a large amount of exposure time can be reduced, thereby increasing FPS.
CMOS sensors read each pixel faster than CCD sensors transfer charge in the shift register, allowing for higher FPS.
Interface
There are various methods to connect camera modules to embedded systems. Typically, cameras with USB and Ethernet interfaces are used for evaluation, as this eliminates the need for custom driver development.
Other important parameters when choosing an interface include transmission length, data transmission rate, and operating conditions. Table 2 lists the most commonly used interfaces. Each interface has its pros and cons.
Table 2. Comparison of Various Camera Interfaces
Optical Components
The basic role of optical lenses is to collect scattered light from the object and reproduce the image of the object on the photosensitive image sensor (CCD or CMOS). When selecting optimized lenses, the following factors should be considered: focal length, sensor format, field of view, aperture, principal ray angle, resolving power, and distortion.
Lenses come in fixed standard focal lengths. Common lens focal lengths include 6 mm, 8 mm, 12.5 mm, 25 mm, and 50 mm.
After selecting the lens with the focal length closest to the imaging system’s required focal length, the working distance must be adjusted to focus on the object being measured. Short focal length lenses (less than 12 mm) produce greater image distortion.
If your application is sensitive to image distortion, try increasing the working distance and using a longer focal length lens. If the working distance cannot be changed, then there will be certain limitations when selecting optimized lenses.
Table 3. Main Types of Camera Lenses
When mounting a lens onto a camera, some mounting system must be used, and the mechanical stability (loose lenses will produce out-of-focus images) and distance to the sensor must be determined.
To ensure compatibility between different lenses and cameras, we define the following standard lens mounts.
Table 4. Common Lens Mounts in Embedded Space
NVIDIA Camera Module Partners
NVIDIA has established a rich ecosystem of partnerships with powerful camera module manufacturers worldwide. For details, please see Jetson Partner Supported Cameras:https://developer.nvidia.com/embedded/jetson-partner-supported-cameras?t1_supported-jetson-products=Orin
These partners can assist you in completing the entire imaging system design process from concept to production for NVIDIA Jetson applications.
Figure 4. Combination of NVIDIA Jetson with Camera Modules
Meets the diverse needs of various industries
Conclusion
This article discusses the most important features to consider when selecting a camera for embedded applications. This selection process may seem daunting, and you first need to understand the main constraints in design, performance, environment, and cost.
After understanding the constraints, focus on the features most relevant to your use case. For example, if the camera will be deployed far from the computing center or in harsh environments, consider using the GMSL interface; if the camera will be used in low-light conditions, consider camera modules with larger pixels and sensor sizes; if the camera is to be used for motion applications, consider using a camera with a global shutter.
For more information, please watch:
Optimizing Edge Applications: Unveiling the Right Combination of Jetson Processors and Cameras
https://www.e-consystems.com/webinars/optimize-your-edge-application-unveiling-the-right-combination-of-jetson-processors-and-cameras.asp
For detailed specifications of AI performance, GPU, CPU, etc., based on Xavier and Orin Jetson modules, please visit Jetson Modules:
https://developer.nvidia.com/embedded/jetson-modules
