Edge AI algorithms often require the combination of various features and signals as input. For example, calculating moving averages of time series over multiple different windows and passing them together to a machine learning model. Creative slicing and dicing of data can yield unexpected results. This article presents a general framework for data fusion in edge AI applications, in conjunction with the book “AI at the Edge”.
Sensor Data Fusion
In addition to combining features from the same signal, sensor fusion also involves integrating data from multiple sensors. For instance, an edge AI fitness tracker can combine information from accelerometers, gyroscopes, and heart rate sensors to attempt to detect the activity the wearer is engaged in.
In more complex edge AI scenarios, sensors do not even have to be integrated within the same device. For example, smart climate control systems utilize temperature and occupancy sensors distributed throughout a building to optimize air conditioning usage.
Sensor fusion can be categorized into three types:
Complementary
Compared to a single sensor, multiple sensors combined can provide a more comprehensive understanding of the situation, such as various sensors on a fitness tracker.
Competitive
Multiple sensors measure the same object to reduce the likelihood of erroneous measurements, such as multiple redundant sensors monitoring the temperature of critical equipment.
Cooperative
Information from multiple sensors is combined to produce a signal that could not be obtained otherwise, for example, two cameras generating a stereo image that provides depth information.
The challenge of sensor fusion is how to combine multiple signals that may occur at different frequencies. The following points need to be considered:
1. Timely alignment of signals. For many algorithms, it is important that all signals we intend to fuse are sampled at the same frequency and that this data reflects simultaneous measurements. This can be achieved through resampling, for example, upsampling low-frequency signals to match the rate of high-frequency signals being fused. Specific methods can refer to “Common Digital Signal Processing Methods for Edge AI”.
2. Scaling signals. The numerical values of the signals must be on the same scale so that signals with larger values do not overwhelm those with smaller values.
3. Digital combination of signals. This can be accomplished using simple mathematical operations such as addition, multiplication, or averaging, or more complex algorithms like Kalman filtering, or simply concatenating the data and passing it as a single matrix to the algorithm.
Sensor fusion can also be performed at other stages of feature engineering. For example, when fusing two time series, one time series can be low-pass filtered first, then scaled to the same ratio, combined by averaging, and the combined value transformed into the frequency domain.
Feature Scaling
Data fusion also involves feature scaling. The data streams from sensors can have a wide range of numerical values. For instance, if a sensor returns measurements in the form of a 16-bit unsigned integer, its values may range from 0 to 65,535.
This large range can make certain AI algorithms tricky. For example, when the input values for deep learning models are large, they may be difficult to train.
Moreover, when incoming features with drastically different scales are present, it can be challenging to obtain good results from machine learning models. Larger values overshadow smaller ones, which undermines the advantages of multiple feature inputs. This is also one of the issues that sensor fusion needs to address.
To solve this problem, inputs need to be scaled before combining or sending them to AI algorithms. A common method to achieve this is called normalization. There are several different types of normalization. The simplest method is rescaling, which first determines the maximum and minimum values of a specific feature in the input data sample, where the input data typically refers to training data in machine learning. The normalized value can then be calculated using the following formula:
The calculated result is a value between 0 and 1, which can be conveniently compared and combined with other normalized values on the same scale.
Other common scaling methods include mean normalization and standardization. An overview can be found in the Wikipedia article “Feature Scaling”.
It is important to note that in practical applications, the data encountered may have a different range than the training data. To avoid issues, all data exceeding the expected range should be trimmed.
In summary, this article introduces data fusion methods for edge AI algorithms, including sensor data fusion and feature scaling.
The content related to digital signal processing methods discussed in this article can refer to the following article.
Common Digital Signal Processing Methods for Edge AI