There are thousands of different types of sensors on the market. This article discusses the classification of commonly used sensors in edge AI based on the book AI at the Edge.
A good way to group sensors is by their modalities, which refer to the ways in which something occurs or is experienced. From a human perspective, our vision, hearing, or touch have different modalities. There is no strict definition for the classification of sensors, and their optimal applications may vary by industry. Below is a general classification from the perspective of edge AI industry applications:
-
Acoustic and Vibration
-
Visual and Scene
-
Motion and Position
-
Force and Touch
-
Optical, Electromagnetic, and Radiation
-
Environmental, Biological, and Chemical
Acoustic and Vibration
The ability to “hear” vibrations allows edge AI devices to detect the movement and vibrations of distant objects, as well as communication between humans and animals. This is accomplished through acoustic sensors, which measure vibrations transmitted through a medium, which can range from air to water and even ground. Sensors with air as the medium include microphones, as shown in the figure below, while sensors with water as the medium include hydrophones, and sensors with ground as the medium include seismic detectors and seismographs. Some vibration sensors are specifically designed for heavy industrial machinery.
3D rendering of a surface-mounted Micro-Electro-Mechanical Systems (MEMS) microphone. From the book AI at the Edge.
Acoustic sensors typically provide time-series data describing pressure changes in their medium. Acoustic signals contain information of various frequencies, such as the highs and lows of a song. Acoustic sensors usually operate within a certain frequency range, and even within that range, they may not respond linearly to frequencies.
In addition to non-linear frequency response, the ability of acoustic sensors to capture high frequencies also depends on their sampling rate. To accurately capture high-frequency signals, acoustic sensors must have a sufficiently high sampling rate. When building acoustic edge AI applications, it is important to understand the properties of the signals to be measured and select appropriate sensor hardware.
Visual and Scene
Edge AI applications often require a passive understanding of the surrounding environment without the need to reach out and touch it. The most common sensors used for this task are image sensors, ranging from miniature low-power cameras, as shown in the figure below, to ultra-high-quality, multi-megapixel sensors. Data obtained from image sensors is represented as an array of pixel values.
Miniature image sensors, possibly used in embedded devices. From the book AI at the Edge.
Image sensors capture light using a grid of sensor elements. In a camera, light from the scene is focused onto the sensor through a lens. The area that a camera can image is called its field of view, which depends on the lens and the size of the image sensor.
Some common characteristics of image sensors include:
Color Channels
For visible light, sensors can typically capture grayscale or color (red, green, and blue, or RGB) data.
Spectral Response
Image sensors are sensitive to light wavelengths that may extend beyond the human visual range. This can include infrared radiation, for example, thermal cameras can “see” heat.
Pixel Size
Larger sensors can capture more light at each pixel point, thus increasing their sensitivity.
Sensor Resolution
The more elements on the sensor, the finer the details that can be captured.
Frame Rate
The frequency at which the sensor captures images, usually measured in frames per second.
Since sometimes lighting is required for the scene before image capture, image sensors are often paired with light emitters in the visible and non-visible spectrum. For example, infrared LEDs can be used with infrared-sensitive cameras to illuminate dark scenes without producing visible light that could disturb humans or animals.
Larger, higher-resolution sensors typically require more energy. High-resolution sensors generate large amounts of data, which may be difficult to process on smaller edge AI devices.
A relatively new class of image sensors, called event cameras, operates slightly differently. Instead of capturing the entire field of view at a specific frame rate, each pixel in the camera responds individually to changes in brightness, remaining silent if there is no response. The result is a time series of changes at individual pixels, making this data easier for edge AI devices to process than traditional large amounts of full-frame data.
Another interesting type of image sensor is the distance imaging sensor. It can image the surrounding environment in three-dimensional space, typically by emitting light and measuring the time it takes for the light to bounce back. A common time-of-flight detection sensor technology is LiDAR (Light Detection and Ranging).
LiDAR sensors work by scanning the surrounding environment with a laser beam and measuring how much light is reflected back to the sensor. This allows them to visualize areas in three-dimensional space, as shown in the figure below.
This image from the open-source PandaSet LiDAR dataset shows a typical LiDAR “point cloud,” where each point in the 3D visualization represents a distance measured by the laser; the inset in the upper right shows a visual sensor capture of the same scene.
Compared to image sensors, LiDAR and other time-of-flight sensors are larger, more complex, more expensive, and consume more energy. The large amounts of data they generate can be difficult to process and store on edge devices, which also limits their practicality. LiDAR is often used for mapping environments, including assisting autonomous vehicles in navigation. In the future, as the computational capabilities of edge devices improve, the limitations of LiDAR applications will gradually decrease.
Radar or radio detection and ranging is also used by edge devices to understand the position of surrounding objects in three-dimensional space, typically over longer distances. Like LiDAR, it is complex and energy-intensive, but it is also a viable option depending on the application scenario.
Motion and Position
One of the common functions for edge AI devices is to understand their position and orientation. There are many different types of sensors that can achieve this function. This is a broad category, ranging from the simplest mechanical tilt switches to the most complex satellite GPS (Global Positioning System). Their role is to help devices understand their position and movement in the world.
Below are typical motion and position sensors used in edge AI applications:
Tilt Sensors
A mechanical switch that opens or closes based on its orientation. They are inexpensive and easy to use.
Accelerometers
Measure the acceleration of an object along one or more axes, i.e., the change in speed over time, usually at a high frequency. Accelerometers are commonly used for motion sensing; in smartwatches, they can be used to identify characteristic movements of sports activities, and in predictive maintenance of devices, they can sense vibrations in industrial equipment. Due to gravity, they always sense which direction is down.
Gyroscopes
Measure the rate of rotation of an object. They are typically paired with accelerometers to provide a description of the object’s motion in 3D space.
Rotary or Linear Encoders
Measure the precise position of rotating or linear mechanisms. They can be used for positioning inkjet print heads. Commonly used in robotics to capture the position of a robot’s wheels, limbs, and other appendages.
Time-of-Flight
A sensor that uses electromagnetic radiation (light or radio) to measure the distance between the sensor and any object within its line of sight.
Real-Time Location Systems (RTLS)
A system that uses multiple transceivers at fixed locations around a building or site to track the position of a single object. Can be used for locating goods in a warehouse.
Inertial Measurement Units (IMU)
A system collection that uses multiple sensors to estimate the current position of a device based on its motion measured from an internal reference frame.
Global Positioning System (GPS)
A passive system that uses radio signals from satellites to determine the position of a device, with a maximum range of several meters. Requires signal transmission from the device to multiple satellites.
Motion and position are typically represented as time-series readings from the sensors. Given the number of sensor types in this category, there are viable options for every cost and energy budget. Generally, the higher the confidence required for absolute positioning, the higher the cost and complexity involved.
Force and Touch
From mechanical switches to load cells, force and touch sensors help edge AI devices measure the physical characteristics of their environment. They can be used for user interaction, understanding the flow of liquids and gases, or measuring mechanical strain on objects. Below are some typical force and touch sensors:
Buttons and Switches
Traditional switches can serve as simple buttons for human-machine interaction or as sensors providing binary signals indicating when a device has collided with something.
Capacitive Touch Sensors
Measure the number of conductive objects touching the surface. For example, modern touch screens use a person’s finger as a conductor, calculating coordinates by measuring changes in capacitance.
Strain Gauges and Flexible Sensors
Measure the degree of deformation of an object, which can be used to detect the extent of damage to an object or to build human-machine tactile interaction systems.
Load Cells
Measure the precise amount of physical load applied to them. They come in various sizes, from tiny to massive. Small ones can measure down to the milligram level, while large ones can measure strain in bridges and skyscrapers.
Flow Sensors
Used to measure the flow rate of liquids and gases, such as measuring the flow rate of water in pipes.
Pressure Sensors
Used to measure the pressure of gases or liquids, whether environmental pressure, such as atmospheric pressure, or internal system pressure, such as tire pressure in cars.
Force and touch sensors are typically simple, low-power, and easy to use. Their measurements are easily represented as time series. They are particularly useful when building tactile interactions or detecting when mobile devices like robots collide with something.
Optical, Electromagnetic, and Radiation
This category includes sensors that measure electromagnetic radiation, magnetic fields, and high-energy particles, as well as basic electrical characteristics such as current and voltage, and measurements of light color. Below are some typical optical, electromagnetic, and radiation sensors:
Photoelectric Sensors
They can detect various wavelengths of light, both visible and invisible to the human eye. This is useful for many scenarios, measuring ambient light intensity or detecting when a light beam is interrupted.
Color Sensors
Use photoelectric sensors to measure the precise color of surfaces, aiding in the identification of different types of objects.
Spectral Sensors
Use photoelectric sensors to measure the absorption and reflection of various wavelengths of light by materials, enabling edge AI systems to gain insights into their composition.
Magnetometers
Measure the strength and direction of magnetic fields. One application of magnetometers is in digital compasses, which can indicate the direction of north.
Inductive Proximity Sensors
Use electromagnetic fields to detect nearby metals. They can be used to detect vehicles for traffic monitoring.
Electromagnetic Field (EMF) Meters
Measure the strength of electromagnetic fields. They can detect signals emitted sporadically by industrial equipment or continuously emitted by radio transmitters.
Current Sensors
Measure the current flowing through a conductor. They are very useful for monitoring industrial equipment, as current fluctuations can provide information about the operation of the equipment.
Voltage Sensors
Measure the amount of voltage across an object.
Semiconductor Detectors
Measure ionizing radiation, which consists of fast-moving particles typically produced by radioactive material decay.
Like many other sensors, this category typically provides time-series measurement data. While useful for measuring environmental conditions, the sensors described here can also be used to detect emissions intentionally produced by devices. For example, photoelectric sensors can be paired with a light emitter on the other side of a channel to detect when someone passes by.
Environmental, Biological, and Chemical
This is a looser category that includes many different types of sensors. Environmental, biological, and chemical sensors enable edge AI devices to sniff out the composition of the surrounding world. Some common types of sensors include:
Temperature Sensors
Measure the temperature of the device itself or a distant infrared emitting source.
Gas Sensors
These sensors can measure the concentration of different gases. Common gas sensors include humidity sensors, which measure water vapor, volatile organic compound sensors, and carbon dioxide sensors, which measure various common organic compounds.
Particulate Matter Sensors
Measure the concentration of tiny particles in air samples, often used to monitor pollution levels.
Biological Signal Sensors
Cover various signals present within living organisms, such as measuring electrical activity in the human heart (ECG) and brain (EEG).
Chemical Sensors
These sensors can measure the presence or concentration of specific chemicals.
These types of sensors typically provide time-series readings. Because they require chemical and physical interactions with the environment, they can sometimes be challenging to use; for example, they often require calibration with known quantities of chemicals, and sometimes sensors need to be preheated for reliable readings. Environmental sensors often degrade over time and need to be replaced periodically.
Other Signals
In addition to collecting signals from the physical world, many edge AI devices can also access rich virtual data sources. This can be broadly divided into two groups: internal data about the state of the device itself and external data about the systems and networks to which the device is connected. Depending on the device, there are various types of internal state signals. These include:
Device Logs
Track the lifecycle of the device since power-up. They can provide information on many different contents: configuration changes, duty cycles, interrupts, errors, or any other recorded selections.
Internal Resource Utilization
This may include available memory, power consumption, clock speed, operating system resources, and peripheral usage.
Communication
The device can track its physical connections, radio communications, network configurations and activities, and the resulting energy usage. Internal sensors
Some devices have internal sensors; for example, many system-on-chip devices include a temperature sensor to monitor their CPU operating temperature.
Internal data can also be used to extend battery life. If lithium batteries are continuously kept at 100% charge when plugged in, they may lose capacity. Apple‘s iPhone uses an edge AI feature to optimize battery charging to avoid this issue. It uses a machine learning model on the device to learn the user’s charging patterns and then uses this model to minimize the time the battery is fully charged while ensuring the battery is still charged when the user needs it.
The outward data streams from external sources also contain extremely rich information. Here are some possible sources:
Data from Interconnected Systems
Edge AI devices are often deployed in networks, and data forwarded from adjacent devices can serve as input for AI algorithms. For example, IoT gateways can process and make decisions based on data collected from their nodes using edge AI.
Remote Commands
Edge AI devices may receive control commands from other systems or users. For example, a user of a drone can request it to move to a specific coordinate in 3D space.
Data from APIs
Edge AI devices can request data from remote servers to input into their algorithms. For example, a home heating system equipped with edge AI may request weather forecast data from an online API and use this information to help decide when to turn on the heating.
Network Data
Includes network structure, routing information, network activity, and the content of data packets.
Some of the most interesting edge AI systems simultaneously utilize all of these data streams. For example, an agricultural technology system can help farmers care for crops. It may include remote sensors in the field, important online data sources such as weather forecasts or fertilizer prices, and a control interface used by farmers. As an edge AI system, it may operate without an internet connection, but if it has an internet connection, it can leverage valuable information. In more complex system architectures, edge AI can also pair with server-side AI.
In summary, this article summarizes the common types of sensors used in edge AI applications, including acoustic and vibration, visual and scene, motion and position, force and touch, optical, electromagnetic, and radiation, and environmental, biological, and chemical.