Follow us by clicking the blue text
Abstract: Hyperspectral imaging has recently gained increasing attention in various applications, including agricultural surveys, ground tracking, remote sensing, etc. However, the high cost, large physical size, and complex operating procedures hinder the application of hyperspectral cameras in various applications and research fields. In this paper, we introduce a cost-effective, compact, and easy-to-use active lighting camera that could benefit many applications. We developed a fully functional prototype of such a camera. With the hope of aiding agricultural research, we tested the camera’s imaging capabilities for plant roots. Furthermore, a U-Net model for spectral reconstruction was trained using data from a reference hyperspectral camera as ground truth and our camera’s data as input. We demonstrated that our camera can capture more information than a typical RGB camera. Additionally, the ability to reconstruct hyperspectral data from multispectral inputs allows our device to be compatible with models and algorithms developed for hyperspectral applications without modification.
1. Introduction
In recent years, hyperspectral imaging has attracted increasing interest. The ability to capture rich spectral information beyond traditional RGB channels has proven helpful in revealing more information about subjects of interest, including agriculture and plant research [1-5]. Compared to RGB cameras, hyperspectral imaging can provide richer information within the visible light bandwidth (380-700nm) as well as additional information outside the visible bandwidth that would otherwise be unavailable. However, hyperspectral cameras are often significantly more expensive than traditional cameras and have larger physical sizes. These disadvantages limit the opportunities to integrate hyperspectral imaging into broader industrial applications or research projects. For example, minirhizotrons are common tools for studying plant roots [6-9]. They are tubular inspection tools made of transparent materials, with an inner diameter ranging from 5 to 20–30 centimeters. They are inserted into the soil near the plants to allow researchers to visually inspect plant roots non-destructively. A significant amount of research has been conducted with the help of minirhizotrons and compact monochrome or RGB cameras [10-12]. Although these studies could significantly benefit from hyperspectral imaging, to our knowledge, there is currently no feasible method that considers the potential benefits of performing hyperspectral imaging in narrow spaces such as minirhizotrons. We explored a new method for obtaining spectral information. Our approach does not perform full spectral scans but focuses on a few selected discrete bands that carry the most promising information. Our device utilizes off-the-shelf components available in the consumer market, and our design features ultra-compact dimensions, allowing it to fit applications with very narrow imaging operational spaces. Besides being compact, our device is also low-cost and easy to use, making large-scale deployment and autonomous operation possible.
2. Related Work
Hyperspectral imaging (HSI) has had a significant impact on agriculture and remote sensing. For instance, classification allows the separation of diseased plants from healthy ones using additional color band information [13-17]. Moreover, detecting crop maturity and identifying stressed crops are important tasks of HSI [18-23]. The increasing accessibility of HSI has had a broad impact [24-29], with hundreds of spectral bands studied [30-33]. These bands provide rich information that can be analyzed and identified to differentiate objects in a scene [34,35]. HSI has implications across various fields, including agriculture (plant disease detection and classification), medical tissue analysis, forest detection, mining and mineral studies, vegetation estimation, environmental protection, and bioanalysis [36-41]. However, the big data footprint of spectral bands also has drawbacks, such as high computational time complexity, transmission, storage, and analysis [32,42–45]. Therefore, minimizing redundant information and time complexity is crucial for effective reconstruction of hyperspectral data [43,46–48]. Techniques for compressed sensing include clustering [49,50], sorting methods [51-54], greedy methods [55,56], and evolutionary methods [57,58]. Dimensionality reduction of HS samples: A series of band selection methods have been proposed in the existing literature. Generally, to reduce the dimensionality of hyperspectral images, these band selection methods can be categorized into two specific groups, including feature extraction [59,60] and feature selection (band selection) [46,61–65]. Interestingly, both specific methods extract or select data from all HSI bands corresponding to the entire spectral cube, resulting in results that are almost or roughly equivalent to the entire HSI bands. Currently, traditional feature selection methods are considered the most commonly used techniques, including Principal Component Analysis (PCA) [53], Maximum Noise Fraction (MNF) [66], Genetic Algorithms, and FICA (Fast Independent Component Analysis) [59]. For feature extraction methods, high-dimensional data is mapped to low-dimensional data, considering specific criteria in dimensional space, thereby extracting a complete new subset of features representing the original HS data. Unfortunately, during the spatial transformation process, the physical representation of the original HS data may not be the same, and it is also possible that some critical or major information may be permanently lost. On the other hand, for band selection methods, a unique and representative subset is selected from the unique hyperspectral data, retaining physical representation and information without loss. Additionally, it retains the inherent characteristics of HS data. In this paper, we focus on feature (band) selection rather than feature (band) extraction. Supervised and semi-supervised labeling: Supervised methods require labeled samples to select the most advantageous bands during training and learning [67], where similarity measures are used in class labels. These techniques require many evaluation conditions, which can be categorized as: information divergence [68], maximum ellipsoid volume [69], Euclidean distance [70], etc. For semi-supervised methods, graph-based models are used to label and select appropriate spectral bands for labeled and unlabeled data samples [71,72], but are hindered by the context information provided. Attempts have been made to integrate multispectral cameras into minirhizotron devices [73]. This work achieved an automatically operated multispectral camera that can operate within a minirhizotron. Their camera was also equipped with multi-bandwidth light sources. However, this work did not explore the information contained in the data obtained from such devices, but primarily focused on autonomous operation and remote deployment.
Figure 1. (a) Photo of the 0-version camera designed for data acquisition; (b) The layout of LEDs maximizes the intensity distribution of different colors. Left: LED module; Middle: All LEDs on, excluding UV; Right: Monochrome lighting.
3. Hardware Design
We propose a camera that uses LEDs of different bands instead of color filters to capture spectral information. In our setup, the camera sensor does not need to be equipped with any type of optical filter, not even the Bayer filter commonly used for RGB sensors or infrared filters at the back of the lens. This setup is very effective when the camera LEDs are the only light source. Additionally, it can handle static scenes with moderate ambient light by treating the ambient light as a “dark field” and only tracking the incremental light when each LED is illuminated. Our prototype design accommodates 8 different types of LEDs, each with a unique bandwidth we selected. Table 1 lists the optical characteristics of each LED model, and the selection of bandwidths will be further discussed in the next section. We installed 4 LEDs of each color, as shown in Figure 1b, with the same color LEDs arranged in a rectangular pattern to illuminate the subject as evenly as possible. This arrangement minimizes the differences in light distribution among different colors. Since all the components of our camera are standard market-ready components, we can achieve an extremely low unit cost compared to currently available hyperspectral cameras. Additionally, the constraints of the camera operating environment are another important aspect we should pay attention to. In our case, our goal is to be widely used in the study of plants in minirhizotrons. Minirhizotrons are specialized tools used for biological and agricultural research to study roots in natural environments. They are transparent tubes inserted into the soil to observe the growth of roots over time. The challenge lies in the geometry of the minirhizotron. Typical minirhizotrons have diameters ranging from 5 to 10 centimeters, while the diameter used by our collaborating research group is only 5 centimeters. Even for our camera, mounting it into such a narrow tube presents a significant engineering challenge. Most existing hyperspectral cameras on the market are large in size, making them unsuitable for deployment in such environments. Our design integrates LEDs by utilizing small monochrome sensor modules and custom-designed printed circuit boards (PCBs). The camera module and LED module are stacked together with only 5 millimeters of distance. There is an opening in the center of the LED module to allow the wide-angle lens located on the camera module PCB to pass through. Each module has an upstream USB port for connecting to the host (Raspberry Pi), and the USB port handles both power and communication. The design of the first camera is intended to work with the flat surface of the root box, but the same core components can be easily adapted to different diameters of 5-centimeter minirhizotrons (Figure 2).
Figure 2. The same components can be installed into minirhizotrons with dedicated housing design.
4. Bandwidth Selection
We analyzed the data collected by the reference HSI camera using the same rhizobox samples as our camera (analysis performed by T.M. Sazzad). Among all 8 bandwidths, the three bandwidths we selected perfectly matched the optimal bands (blue, green, and red). The other bandwidths were limited by the availability of stock during the production of the prototype, thus not fully matching the calculated optimal values.
5. Data Collection and Processing
We collected a dataset to train a U-Net-based spectral reconstruction model. Since ground truth is needed to train the model, and our reference data needs to be captured by a reference HSI device unsuitable for minirhizotrons, the initial version of our developed camera (as shown in Figure 1a) is specifically designed to match the results from the reference camera. Both cameras captured hyperspectral images of plant roots in the root box. The rhizobox is a container similar to a minirhizotron, except that the root box has a flat transparent surface, making it easier to take pictures with all types of cameras. Additionally, the compact size of the root box makes it easy to move. Due to distortions and uneven intensity distribution introduced by the hardware, the raw data captured by our camera (Figure 4a) must be preprocessed before proceeding. In our calibration process, the raw images are first mapped by a “reference white” calibration image, eliminating the uneven intensity distribution introduced by the light source and lens projection. The reference white is captured and applied separately to each color. Then geometric corrections are applied based on checkerboard images captured by the same camera. The brightness distribution from the previous calibration step is preserved during this process (the calibration process is shown in Figure 3, with an example calibration image shown in Figure 4b). The last step in data processing is to match the results of our camera with the results of the reference camera. (1) Root box, (2) imaging area of the reference camera, and (3) imaging area of our camera are sized as shown in Figure 5. We first manually resized the output from the camera (calibrated to 1024×1024) to match the pixel density of the reference camera (286×286). Then we were able to run template matching on the reference image (using the resized image as a template). Figure 6 shows examples of successful matches.
Figure 4. (a) The raw images captured by our camera, showing 8 different bands of the same area, each band rendered with the corresponding pseudo-color; (b) Processed post-calibration results, where distortions were corrected, camera frames cropped out of the field of view, and dark corners adjusted to make the brightness uniform across the entire image.
Figure 5. Size comparison of the root box, reference camera imaging area, and our camera imaging area. The sizes shown are conceptual sizes and are not strictly to scale with actual sizes.
Figure 6. Example of successful matching between the reference camera and our camera.
6. Reconstruction Model
Using the data we collected, we implemented and trained a U-Net-based model. The model aims to eliminate the bright LED light spots introduced by our custom light module and expand the number of bands from 8 to 299.
7. Conclusion
In this work, we demonstrated the feasibility of constructing a cost-effective compact hyperspectral camera. We conducted spectral reconstruction experiments from reduced spectral images to full-sized hyperspectral images. The reconstruction results showed higher correlation with plant root pixels but performed poorly on soil pixels. Such results indicate that our active lighting camera setup can indeed obtain additional useful data. The next step will involve capturing data from real minirhizotrons and validating our model’s ability to transfer its knowledge to these data. This is expected to be more challenging, as the plants may belong to different species, and the soil types may differ significantly from the rhizoboxes from which we obtained training data.
Follow for more information
