Introduction
INTRODUCTION
Bringing the intelligent world back to its roots, sensors are waiting here. The foundation of intelligence is perception, and sensors are the gateway to perception. Sensors are evolving towards intelligence, cognition, analysis, and diagnostics. As an increasingly evident intelligent microsystem, sensors are becoming more independent and capable of self-correction. So, what capabilities will sensors use to ignite the future and drive digital transformation?
Author: Lin Xueping Global Industrial Observer
General Manager of Beijing Lianxun Power Consulting Co., Ltd., and part-time professor at Tianjin University, School of Precision Instrument. Some content is sourced from the sensor report “Transformational Trends: Paradigm Shift from 2021 to 2023” by Singapore consulting firm Twimbit. This article has been restructured in terms of viewpoints and content.
Figure 1: The Top Twenty Trends in Sensor Development
(Source: Singapore startup consulting firm Twimbit)
Where are 3D Sensors Headed?
3D depth sensor technology can create three-dimensional images, fully meeting human visual needs. It can use Time of Flight (ToF), structured light, 3D interference, etc., to obtain three-dimensional visual data. Among these, ToF sensors have attracted much attention in the mobile phone sector, providing excitement for phones that have been busy enhancing camera functions. It utilizes the time difference of laser pulses emitted and reflected back from each pixel of the infrared sensor to obtain three-dimensional depth, forming imaging technology for 3D models.
However, ToF lenses have experienced a rollercoaster ride in the mobile phone industry, “coming with excitement and leaving with disappointment.” In 2018, its application peaked, with mid-to-high-end models from Samsung, Huawei, OPPO, Xiaomi, etc., all equipped with ToF lenses. Just when everyone thought this was the future direction of mobile imaging, these phones suddenly disappeared quickly. The ToF lens came quickly, just like a delivery person, and left just as fast.
The reason is simple: ToF technology lacks essential application support, and there is no widely used killer application to drive the further development of ToF lenses. ToF can be used to scan the shape of objects and automatically create a 3D model. However, for ordinary consumers, creating a model has no practical use. A phone with a ToF lens can be used as a ruler, but the accuracy is insufficient. Besides, who would take measuring with a phone seriously?
In today’s intelligent world, any hard technology must rely on software support. Without software applications, it is difficult to support the iterative progress of hardware technology. Unless the rise of the metaverse, that is, AR/VR applications, may save the application of ToF lenses in mobile phones.
On the other hand, a robotic vacuum cleaner can replace single-line mechanical scanning laser radar with a wide-angle ToF camera, making it easier to create a “battlefield map” of the room, better planning its path, and making the robotic vacuum cleaner appear less foolish. Otherwise, the robotic vacuum cleaner would either bump into table legs or get tangled in socks and cables, with poor obstacle avoidance. But the premise remains that it cannot be too expensive.
In autonomous driving, distance measurement and perception of the driving environment, human-machine collaboration in industrial collaborative robots, and intelligent logistics vehicles can also allow ToF sensors to truly play a role. However, for the industry, ToF sensor lenses are still too expensive. From the perspective of the industrial chain, the current infrared sensors for ToF lenses are mainly controlled by Sony, Infineon, ON Semiconductor, Texas Instruments, and Panasonic, while optical lenses are dominated by Largan Precision, Zhejiang Sunny Optical, and AAC Technologies. The core part, CMOS image sensors, mainly comes from three long-time rivals: Sony, Samsung, and OmniVision under Will Semiconductor.
3D depth sensors have even greater ambitions in the military. The U.S. Department of Defense’s DARPA is developing military 3D sensing technology for covert operations at night.
Generally speaking, any autonomous driving system usually requires some form of active illumination to achieve autonomous navigation at night. However, turning on headlights or the laser radar (LiDAR) emission system will emit radiation signals. In military situations, this can allow the enemy to detect these vehicles from a distance. DARPA is attempting to develop 3D visual sensors using the weak thermal signals of various living and non-living objects in the wild environment. This “invisible headlight” program will explore the information contained in various thermal radiations in the environment, as all objects emit thermal energy. DARPA’s goal is to explore how to capture information even from minimal thermal radiation to develop passive sensors that generate 3D maps for navigation. This will greatly expand the capabilities of autonomous driving systems to operate covertly.
A New Face for Acoustic Sensors
The biggest feature of acoustic technology is its relative affordability compared to other sensor technologies, allowing for exploration of various applications. Surface Acoustic Wave (SAW) technology is widely used in filter signal processing and has shone brightly in smartphone speakers. Globally, the SAW filter market is mainly dominated by Japanese companies, including Murata, TDK, and Taiyo Yuden, which together account for about 82% of the market share. Murata alone accounts for nearly half of the global SAW filter market. Among domestic RF filter companies, Shenzhen Micron Technology is a leader in the domestic replacement of filters and integrated inductors, having entered Huawei’s supply chain with a revenue of 1.3 billion in 2021.
SAW technology is relatively used for low frequencies and is sensitive to temperature, taking full advantage during the 4G era. However, in high-frequency fields and under conditions where multiple signal processing must avoid interference, Bulk Acoustic Wave (BAW) resonator technology has broader applications, catering to the 5G and IoT era. In this area, American technology is more advanced. Qorvo and Texas Instruments hold advantages. This is also a bottleneck that China finds difficult to break through, as Texas Instruments applied this technology to integrated clock functions in 2019. With the increasing speed of big data transmission, there are stringent requirements for clock signals. A data processing system (DPS) requiring 18G capacity per second has become a pressing issue for many chip manufacturers. BAW can effectively achieve clock technology under high-frequency unified communication. Although SAW is relatively inexpensive, from the overall technological development trend, BAW is replacing SAW, and it has already started to be used in high-end mobile devices like Apple phones.
Sensors as Power Stations
Sensors have two directions. One is integrated sensors, which are integrated with other devices to share energy input; the other is independent sensors. The latter is like Robinson Crusoe on a deserted island, ideally able to survive on its own without care. Self-powering is the primary challenge.
Self-powered sensors are gaining widespread attention, as they are very suitable for remote monitoring, wireless connectivity, and continuous monitoring scenarios.
This often requires deploying energy harvesters for sensors, which can generate microcurrents from solar energy, vibrations, and thermal energy for self-use. In other words, a sensor is both an energy generation device and a storage device. Since an electric vehicle can be a storage system, why can’t a small sensor?
In 2019, American motion and power control systems supplier Parker Hannifin acquired adhesive and vibration management equipment supplier Lord for $3.7 billion. Lord has been providing precise measurement wireless sensors and pressure sensors for aerospace and petrochemical industries. Parker is diversifying from its traditional power systems business, especially to strengthen the technological advantages of its engineering materials department to fully embrace emerging trends like electrification and lightweighting. Lord’s accumulated expertise in coatings, springs, sensor hardware, and sensor cloud, with annual sales of $1.1 billion, perfectly meets Parker’s needs.
Figure 2: The New Favorite of Traditional Power – Sensors (Source: Parker Lord Official Website)
Lord’s MicroStrain sensing business has begun using piezoelectric materials to convert material strain energy into electrical energy. The era of independent power stations for sensors has begun.
From Intelligent Robots to Intelligent Sensors
Machine learning is ubiquitous. If algorithms can not only be placed in machines but also in the smallest perception units—sensors, then embedded artificial intelligence will greatly promote the development of intelligent sensors.
Of course, machine learning is just one direction. Most sensors have shifted from interactive to predictive, partially transferring the initiative of machine intelligence to sensor intelligence. Sensors with intelligent real-time data analysis and process correction capabilities will significantly enhance the interactivity of devices. This also means that edge computing will delve deeper at the edge of machines.
However, the parameters that a sensor can measure are limited; why not integrate multiple sensors together? Thus, various combination sensors have emerged, such as temperature + humidity, pressure + flow, vibration + acceleration + deceleration, etc., becoming the most commonly used combinations. A mixed sensor fleet achieves multi-parameter detection with “one device serving multiple purposes,” forming closed-loop automation applications by detecting various parameters. This has welcomed a broad development space in the field of intelligent manufacturing.
Another direction is fusion sensors. Intelligent sensors are accelerating the development of autonomous vehicles, and the multifunctional fusion of sensors will leverage the advantages of different sensors to provide data analysis and control capabilities, thus possessing embedded intelligence. This is especially important in military aircraft. The U.S. F35 fighter jet has been conducting multi-domain data connectivity and analysis, with the core being the use of fusion sensors to achieve high-speed analysis of multidimensional data, capable of processing heterogeneous data from various platforms, whether from sea, air, space, or land.
OmniVision, a subsidiary of Will Semiconductor, has recently launched a composite sensor that combines CMOS Image Sensors (CIS) with Event-based Vision Sensors (EVS), which is a biomimetic sensor that does not require exposure time limitations. This will be a significant advantage for AR headset players, and the imaging capabilities of mobile phones will be further enhanced. This characteristic of integrating image and vision sensors on a single chip represents a pixel-level sensor fusion, undoubtedly an important direction for the future.
Where Do Pulse Flow, Blood Flow, and Heartbeat Flow Go?
Health has become a hotspot for future development, and health prevention and diagnosis can be achieved through widely used sensors. Whether it is life-supporting implanted devices, long-term monitoring of critically ill patients, or robotic surgery. The Da Vinci surgical robot, born in 2000, is currently the most successful surgical robot internationally. Initially just a stabilizer to assist doctors in surgery, its capabilities have grown stronger, and it has taken the operating table, becoming the best companion for surgeons. The Da Vinci robot has ignited robotic surgery.
A report from the Journal of the American Medical Association (JAMA) mentioned that from 2016 to 2021, the proportion of robotic surgeries among surgical procedures increased from 2% to 15% over the past five years and is accelerating. Currently, there should be over 150 units installed in China, with 40,000 surgeries completed annually using the Da Vinci surgical robot.
It has significant advantages in 3D imaging and precise control. With its assistance, doctors can easily create 3D images of diseased areas magnified 10-15 times (traditional imaging systems can only provide 2-3 times two-dimensional images) and then operate the robot precisely for surgery. Behind this success are nearly 500 sensors. The Da Vinci robot is expensive, with a single machine costing millions. In the cost composition of surgical robots, torque sensors account for about 5%.
For medical and home diagnostics, this is just the beginning. The popularity of the Internet of Things and wearable sensors has already flourished in health applications. These sensors can monitor elderly care and daily health monitoring in a non-invasive manner. Sleep monitoring on watches is just the tip of the iceberg, while the prevention of diseases like diabetes is becoming a hotspot.
Undoubtedly, innovations in wearable sensors have changed the way health monitoring is conducted. Wearable and implantable sensors transmit health data in real-time, providing quantified exercise data and various physiological data for precise diagnosis. These key devices use different sensors, including CMOS image sensors used in mobile imaging, vibration, blood glucose, and optical sensors. The four ring sensors behind the Apple Watch iWatch determine vascular movement and pulse by reflecting LED light onto the skin.
Where do pulse flow, blood flow, and heartbeat flow go? All human health data flows into a digital channel. All data from the human body is stored on an IoT digital health platform. Big data analysis automatically provides a comprehensive portrait of people’s health: the existing health prevention model is about to undergo profound changes.
According to a report by CB Insights in 2020, there was a total of $80.6 billion in financing and 55,000 venture capital deals in this field. Nearly 200 medical financings of over $100 million in 2020 set a new record. Medical giants have over $550 billion in cash waiting to delve into this digital healthcare field. Sensors are at the forefront of this medical investment.
Without Sensor Breakthroughs, the Industrial Internet Will Always Be a Supporting Role
The development of the industrial internet places higher demands on sensors. The performance of these sensors and control systems largely determines the ability of managers to achieve critical control functions remotely using industrial internet platforms. For automation companies like Honeywell, while consolidating its dominant position in control systems, it has an obsession with sensors—this is also the reason for its recent frenzied acquisitions of sensors, including temperature and humidity sensors. A typical example is its acquisition of the British Citeytech sensor, a pioneer in gas sensors, in 2019. Only by strengthening the core technologies of control and sensors can the industrial internet platform truly be realized.
It is expected that by 2023, the number of devices connected to the industrial internet will reach 21.5 billion. The industrial internet needs to solve six problems: connection, perception, control, analysis, communication, and application. Connection is merely physical work, analysis is not yet of great use, communication will become a common technology, and applications are just flashy products before analysis and mechanism models are established. Who controls the devices and who perceives the data are crucial points. This is the crux of the industrial internet platform. Unfortunately, in China, many industrial internet initiatives have strayed from this core battlefield, playing tricks at the application level, only stirring up small waves on a sea of massive garbage data.
Soft Sensors Are Also Sensors
Soft sensors are an interesting concept. Unlike the soft bodies of soft robots, soft sensors are actually virtual sensors, essentially software. They can process multiple measurements simultaneously.
Soft sensors are based on control theory and can indirectly process dozens or even hundreds of measurements simultaneously. In terms of data fusion, soft sensors play a significant role. They combine different static data and dynamic measurement data, making them useful for fault diagnosis and control applications. The classic soft sensor can start with the Kalman filter. It is a data processing technique that removes noise to restore real data, which can be seen as data filtering computed by software. This can also be considered a type of soft sensor. Of course, the latest soft sensors may use neural networks or fuzzy computing. Soft sensors are software programs that estimate physical quantities using information from other sensors rather than direct measurement.
This trend is most evident in process automation, where many control functions are activated by software and assisted by computers. High reliability and high precision are hallmarks of soft sensors; for example, a soft sensor based on pH values can conveniently perform water treatment and peak load detection.
Soft sensors can also be seen as integrators of digital technology, combining advanced automation, IoT, big data real-time analysis, and sensors.
More Sensor Technologies
Optical technology combined with sensors is benefiting from the emerging optoelectronic technology. Chip development is primarily made from silicon, and both optical and electronic components can be integrated on silicon wafers, forming photonics. This adds the rapid upgrade capabilities of electronics to the slowly progressing optical instruments over the years, thus invigorating optical sensors, which will also greatly benefit the solar energy industry and the industrial internet.
IO-Link enables digital connections, directly transmitting data from sensors to IoT interfaces and programmable logic controllers (PLC). Compared to traditional independent module-connected sensor technologies, IO-Link technology offers outstanding cost-effectiveness and technical improvements. Traditional independent modules often represent a single network node. Deploying each node requires a set of chips, which significantly increases system costs when the number of control points is high. IO-Link adopts a master-slave approach, meaning one master can expand to 128 control points, reducing network burden and improving efficiency. Additionally, thanks to the standardization of IO-Link communication, it is more convenient to configure components such as RFID, valve islands, and sensors, allowing not only status monitoring but also parameter configuration and maintenance.
Tianjin Yike’s IO/Link is being widely applied in power lithium battery factories. The production lines in the lithium battery industry are relatively long, with many dense detection points requiring certain dust protection, while high automation levels necessitate collaboration among various components, and continuous production with hot-swappable capabilities and cost-effectiveness is also necessary. Based on these requirements, the IO-Link solution is very suitable for the characteristics of the lithium battery industry. The rapid development of the domestic power battery industry has also brought significant opportunities for domestic sensors.
Wireless sensor networks are also developing rapidly, with very high requirements for low energy consumption. One can imagine many development scenarios within this.
Figure 3: Different Wireless Standards
(ISM Industrial, Scientific, Medical Bands)
On farms, daily monitoring of the body temperature of thousands of dairy cows is needed to prevent diseases like foot-and-mouth disease. If wireless network technology is utilized, simply installing a temperature sensor with a wireless transmitter on each cow can easily achieve real-time temperature readings and transmissions to a main terminal. This low-energy sensor cluster can significantly improve the efficiency and speed of wireless networks, but such sensor networks must ensure low power consumption for widespread application. The two crucial characteristics of Wireless Sensor Networks (WSN) are reliability and low power, with cost being a third priority.
Non-contact sensing technologies, whether optical, wave, magnetic, laser, or acoustic, are also rapidly developing. Infrared temperature sensors have shone brightly in temperature detection during the recent two years of the pandemic. With higher accuracy and more application scenarios, non-contact sensors will also see significant growth. Similarly, rapid detection biosensors are developing quickly to facilitate timely diagnosis and in vitro diagnostics.
Sensors in drones are also a focal point. In fact, drones can be seen as a collection of sensors, essentially a flying sensor cluster. Drones widely use LiDAR, tilt sensors, inertial measurement units, etc. Shenzhen DJI is undoubtedly a leader, while American arms suppliers Lockheed Martin and Boeing are also actively participating. The drone market has exploded from photography and entertainment to military use, significantly increasing its attention. This market is still in its infancy, and various sensors present enormous opportunities.
LiDAR, as a non-contact sensor, is gaining popularity. It can instantly capture millions of data points, making it directly applicable in autonomous vehicles. Currently, domestic progress is not bad, with Huawei, SUTENG, and Hesai Technology having good products. In the field of handling robots (AGV), intelligent warehouse management has also driven the application of LiDAR, including widespread applications from Germany’s largest sensor manufacturer Sick and Japan’s largest LiDAR manufacturer Hokuyo. Drones and robots are also excellent application areas.
Another direction for sensors is friendly natural interaction interfaces, developed around human sensory capture capabilities. Since the birth of computers, the interaction interface has primarily relied on keyboards, with the mouse triggering the second wave of graphical interfaces. However, since then, human-computer interaction has not seen decisive progress. Who can become the third major player in human-computer interaction interfaces? Natural interfaces such as voice, touch, and gesture, which highly conform to human biological characteristics, all have potential. However, intelligent voice control has disappointed in its performance in speaker control, and voice control has not ignited the home as expected. The era of smart speakers initiated by Amazon’s Echo, followed by Google’s Nest and various domestic brands like Xiaomi, has proven to be merely a fleeting celebration of interface interaction. The interface of smart homes is still waiting for a new ruler. Natural interface sensors still have vast development space.
Conclusion: Hidden Pits
The role of sensors cannot be overstated. Sensors are the sensory organs of automation systems and the vanguard of digital technology. They are also a super shortcoming that China has not paid attention to, one reason being that their application scenarios are too scattered. There are over 30,000 types of sensors globally, with applications being small and narrow. However, the hidden pitfalls left by sensors in the intelligent world are subtle. Focusing solely on reaching for the stars can easily lead to falling into these hidden pits.
Video RecommendationFollow the “Digital Enterprise” video account for frontline insights
IPD Method and Practice Advanced Training Course ● Online
Long press the QR code for details or click Read the original text to register↓
