Follow us by clicking the blue text
Abstract: Hyperspectral imaging has recently received increasing attention in various applications, including agricultural surveys, ground tracking, remote sensing, etc. However, the high cost, large physical size, and complex operation process hinder the application of hyperspectral cameras in various applications and research fields. In this paper, we introduce an economical, compact, and easy-to-use active lighting camera that may benefit many applications. We have developed a fully functional prototype of such a camera. With the hope of assisting agricultural research, we tested the camera’s imaging capabilities for plant roots. Additionally, we trained a U-Net model for spectral reconstruction by using data from a reference hyperspectral camera as ground truth and our camera’s data as input. We demonstrate that our camera can obtain more information than a typical RGB camera. Moreover, the ability to reconstruct hyperspectral data from multispectral input allows our device to be compatible with models and algorithms developed for hyperspectral applications without modification.
1. Introduction
In recent years, hyperspectral imaging has garnered increasing interest. The ability to capture rich spectral information beyond traditional RGB channels has proven beneficial in revealing more information about subjects of interest, including agricultural and plant studies [1-5]. Compared to RGB cameras, hyperspectral imaging can provide richer information within the visible light bandwidth (380-700nm) as well as additional information beyond the visible bandwidth, which would otherwise be unavailable. However, the cost of hyperspectral cameras is often significantly higher than that of traditional cameras, and their physical size is larger. These drawbacks limit the opportunities for integrating hyperspectral imaging into broader industrial applications or research projects. For example, minirhizotrons are common tools for studying plant roots [6-9]. They are tubular inspection tools made of transparent materials. Their inner diameters range from 5 centimeters to 20–30 centimeters. They penetrate the soil near the plants to be inspected, allowing researchers to visually examine plant roots non-destructively. A significant amount of research has been conducted with the help of minirhizotrons and compact monochrome or RGB cameras [10-12]. Although these studies could significantly benefit from hyperspectral imaging, to our knowledge, no feasible method exists to consider the potential benefits of performing hyperspectral imaging in narrow spaces like minirhizotrons. We investigated a new method for acquiring spectral information. Our approach does not perform full-spectrum scanning but focuses on a few selected discrete bands that carry the most promising information. Our device utilizes off-the-shelf components from the consumer market, and our design features a super-compact size, allowing it to fit into applications with very narrow imaging operating spaces. In addition to being compact, our device is also low-cost and easy to use. These advantages make large-scale deployment and autonomous operation possible.
2. Related Work
Hyperspectral imaging (HSI) has had a significant impact on agriculture and remote sensing. For example, classification allows for the separation of diseased plants from healthy ones using additional color band information [13-17]. Furthermore, detecting crop maturity and identifying stressed crops are important tasks for HSI [18-23]. The increasing accessibility of HSI has had a wide-ranging impact [24-29], with studies conducted on hundreds of spectral bands [30-33]. These bands provide rich information that can be analyzed and identified to differentiate objects in a scene [34,35]. HSI has influenced various fields, including agriculture (plant disease detection and classification), medical tissue analysis, forest detection, mining and mineral studies, vegetation estimation, environmental protection, and bioanalysis [36-41]. However, the large data footprint of spectral bands also has drawbacks, such as high computational time complexity, transmission, storage, and analysis [32,42–45]. Therefore, minimizing redundant information and time complexity is crucial for effectively reconstructing hyperspectral data [43,46–48]. Techniques for compressed sensing include clustering [49,50], sorting methods [51-54], greedy methods [55,56], and evolutionary methods [57,58]. Dimensionality reduction for HS samples: a series of band selection methods have been proposed in the existing literature. Typically, to reduce the dimensionality of hyperspectral images, these band selection methods can be divided into two specific groups, including feature extraction [59,60] and feature selection (band selection) [46,61–65]. Interestingly, both of the aforementioned specific methods extract or select data from all HSI bands corresponding to the entire spectral cube, resulting in outputs that are nearly or approximately equivalent to the entire HSI bands. Currently, traditional feature selection methods are regarded as the most commonly used techniques, including Principal Component Analysis (PCA) [53], Maximum Noise Fraction (MNF) [66], Genetic Algorithms, and FICA (Fast Independent Component Analysis) [59]. For feature extraction methods, high-dimensional spatial data is mapped to low-dimensional spatial data, considering specific criteria in dimensional space, thereby extracting a complete new subset of features that represent the original HS data. Unfortunately, during the spatial transformation process, the physical representation of the original HS data cannot remain the same, and it is also possible that some key or primary information may be permanently lost. On the other hand, for band selection methods, a unique and representative subset is selected from the unique hyperspectral data, which retains the physical representation and information without loss. Additionally, it preserves the inherent characteristics of the HS data. In this paper, we focus on feature (band) selection rather than feature (band) extraction. Supervised and semi-supervised labeling: Supervised methods require labeled samples to select the most advantageous bands during training and learning, where similarity metrics are used in class labels. These techniques require many evaluation conditions, which can be categorized as: information divergence [68], maximum ellipsoid volume [69], Euclidean distance [70], etc. For semi-supervised methods, graph-based models are used to label and select appropriate spectral bands from labeled and unlabeled data samples [71,72], but they are hindered by the lack of contextual information. Attempts have been made to integrate multispectral cameras into minirhizotron devices [73]. This work achieved an autonomously operated multispectral camera that can operate within a minirhizotron. Their camera was also equipped with multi-bandwidth light sources. However, this work did not explore the information contained in the data obtained from such devices, focusing primarily on autonomous operation and remote deployment.
Figure 1. (a) Photo of the 0-version camera designed for data collection; (b) The layout of the LEDs minimizes the intensity distribution of different colors. Left: LED module; Middle: All LEDs are on, excluding UV; Right: Monochrome lighting.
3. Hardware Design
We propose a camera that uses LEDs of different bands instead of color filters to capture spectral information. In our setup, the camera sensor does not need to be equipped with any type of optical filter, not even the Bayer filter commonly used for RGB sensors or infrared filters on the back of the lens. This setup is very effective when the camera LEDs are the only light source. Additionally, it can handle static scenes with moderate ambient light by treating ambient light as a “dark field” and only tracking the incremental light when each LED is lit. Our prototype design accommodates 8 different types of LEDs, each with a unique bandwidth we selected. Table 1 lists the optical characteristics of each LED model, and the selection of bandwidth will be discussed further in the next section. We installed 4 LEDs of each color, as shown in Figure 1b, with LEDs of the same color arranged in a rectangular pattern for as even illumination of the subject as possible. This arrangement minimizes the differences in light distribution among different colors. Since all components of our camera are standard off-the-shelf parts, we can achieve extremely low unit costs compared to currently available hyperspectral cameras on the market. Additionally, the constraints of the camera’s operating environment are another important aspect we should consider. In our case, our goal is to be widely used in the study of plant minirhizotrons. A minirhizotron is a specialized tool used for biological and agricultural research to study roots in natural environments. It is a transparent tube inserted into the soil, allowing observation of root growth over time. The challenge lies in the geometry of the minirhizotron. Typical minirhizotron diameters range from 5 to 10 centimeters, while the diameter used by our collaborating research group is only 5 centimeters. Even for our camera, installing it into such a narrow tube is a significant engineering challenge. Most existing hyperspectral cameras on the market have larger sizes, making them unsuitable for deployment in such environments. Our design integrates LEDs by utilizing small monochrome sensor modules and custom-designed printed circuit boards (PCBs). The camera module and LED module are stacked together, with only 5 millimeters of distance between them. There is an opening at the center of the LED module to allow the wide-angle lens located on the camera module PCB to pass through. Each module has an upstream USB port for connection to the host (Raspberry Pi), and the USB port handles both power and communication. The design of the first camera is intended to work with the flat surface of the root box, but the same core components can easily be adapted to different diameters of 5-centimeter minirhizotrons (Figure 2).
Figure 2. The same components can be installed into minirhizotrons with dedicated shell designs.
4. Bandwidth Selection
We analyzed the data collected from the reference HSI camera using the same rhizobox samples as our camera (analysis conducted by T.M. Sazzad). Among all 8 bandwidths, three of the selected bandwidths match the optimal bands (blue, green, and red) perfectly. Other bandwidths were limited by stock availability during the production of the prototype, so they do not match the computed optimal values exactly.
5. Data Collection and Processing
We collected a dataset to train a U-Net-based spectral reconstruction model. Since ground truth is needed to train the model, and our reference data must be captured by a reference HSI device unsuitable for minirhizotrons, the initial version of our developed camera (as shown in Figure 1a) was specifically designed to match results from the reference camera. Both cameras capture hyperspectral images of plant roots in the root box. The rhizobox is a container similar to a minirhizotron, with the difference being that the root box has a flat transparent surface, making it easier to take pictures with all types of cameras. Additionally, the compact size of the root box makes it easy to move. Due to distortions and uneven intensity distributions introduced by the hardware, the raw data captured by our camera (Figure 4a) must undergo preprocessing before continuing. In our calibration process, the raw image is first mapped by a “reference white” calibration image, eliminating the uneven intensity distribution introduced by the light source and lens projection. The reference white is captured and applied individually for each color. Then, geometric correction is applied based on checkerboard images captured by the same camera. The brightness distribution calibrated in the previous step is preserved during this process (the calibration process is illustrated in Figure 3, and sample calibration images are shown in Figure 4b). The final step of data processing is to match the results of our camera with those of the reference camera. (1) The sizes of the root box, (2) the imaging area of the reference camera, and (3) the imaging area of our camera are shown in Figure 5. We first manually resize the output of the camera (after calibration at 1024×1024) to match the pixel density of the reference camera (286×286). Then we were able to run template matching on the reference image (using the resized image as a template). Figure 6 shows examples of successful matches.
Figure 4. (a) The raw images captured by our camera, showing 8 different bands of the same area, each rendered with corresponding pseudo colors; (b) Processed results after calibration, where distortions are corrected, camera frames are cropped to the field of view, and vignetting is adjusted to make the brightness uniform across the entire image.
Figure 5. Comparison of sizes between the root box, the reference camera’s imaging area, and our camera’s imaging area. The sizes shown in the figure are conceptual sizes and are not strictly proportional to actual sizes.
Figure 6. Examples of successful matches between the reference camera and our camera.
6. Reconstruction Model
We implemented and trained a U-Net-based model using the data we collected. The model aims to eliminate the bright LED hotspots introduced by our custom light module and to extend the number of bands from 8 to 299.
7. Conclusion
In this work, we demonstrated the feasibility of constructing a compact hyperspectral camera in a cost-effective manner. We conducted spectral reconstruction experiments from downsized spectral images to full-sized hyperspectral images. The reconstruction results showed higher correlation with plant root pixels but performed poorly on soil pixels. Such results indicate that our active lighting camera setup can indeed acquire additional useful data. The next step will involve capturing data from real minirhizotrons and verifying our model’s ability to transfer its knowledge to these data. This is expected to be more challenging, as the plants may belong to different species, and the soil types may differ significantly from those of the rhizobox from which we obtained the training data.
Follow for more information
