▲ On April 1, 2022, at the Qingke Jia Robot Joint Research Institute in Yantai, Shandong, a throat swab nucleic acid testing sampling robot (demonstration machine) collected samples from a staff member. (People’s Vision/Photo)
The full text contains4597words, reading time approximately10minutes
-
At least 10 research institutes and companies in China have announced fully automated nucleic acid sampling robot products. Some companies completed product initiation and design within a month and iterated the finished product multiple times.
-
To improve data collection levels, considerations must be made regarding how the camera captures images during sampling, how to illuminate the dark oral cavity, and what to do if the lens is accidentally touched during operation. In addition to the visual system, another critical technology is the force control system, which is closely related to safety.
-
Due to the involvement in medical procedures, nucleic acid sampling robots must undergo medical device registration approval, and regulatory authorities are still researching which category of device this robot falls under.
This article was first published inSouthern Weekend. Unauthorized reproduction is prohibited.
Written by｜Southern Weekend reporter Hai Yang
Southern Weekend intern Gong Xinran
Editor｜Cao Haidong
This is a pair of white robotic arms connected to a body adorned with eye patterns, obediently “waiting” in a white sampling vehicle. Upon detecting someone approaching the vehicle, the left arm swiftly retrieves a test tube and unscrews the cap; the right arm, holding a cotton swab, gently probes into the throat of the person being tested, rotating slightly before placing the swab into the test tube, the arm’s scissors then cut it, and the cap is screwed back on and placed in the test tube rack. In 30 seconds, the sampling is complete; all the person being tested has to do is open their mouth, close it, and put on or take off their mask. This process is no different from the countless nucleic acid tests many have experienced during the pandemic, conducted by healthcare workers. Like the all-powerful white robot in the animated film “Big Hero 6,” a batch of robots known as “Big Whites” is taking over the nucleic acid sampling work, ensuring precision and safety through machine vision and force control systems. According to incomplete statistics from Southern Weekend reporters, as of the end of June 2022, at least 10 research institutes and companies in China have announced fully automated nucleic acid sampling robot products. Some companies completed product initiation and design within a month and iterated the finished product multiple times. In terms of sampling speed, most robots currently cannot match skilled healthcare workers. The initial advantage of robotic sampling lies in reducing infection risks and alleviating the burden on healthcare workers, but with the implementation of regular nucleic acid testing, nucleic acid sampling robots are being assigned potential cost advantages. Several interviewed companies told Southern Weekend reporters that due to the involvement in medical procedures, nucleic acid sampling robots must undergo medical device registration approval, but regulatory authorities are still researching which category of device this robot falls under.1
Universities and Companies Collaborate, Design Details Vary
Rather than being labeled as “black technology,” sampling robots are more like an integration of existing technologies. Their two core technologies—the machine vision system for image recognition and the force control system for adjusting force—are already quite mature. Compared to other applications, the operational area has simply shifted from automotive components to the more delicate human throat. Currently, the developers of sampling robots come from diverse backgrounds, including traditional robotics companies like Zhongke Xinsong Co., Ltd. (hereinafter referred to as Zhongke Xinsong) and universities such as Shanghai Jiao Tong University and Tsinghua University. Multi-party collaboration is a common model. For example, the “Pengcheng Qinggeng” robot was jointly developed by Hangzhou Huxi Yunbaisheng Technology Co., Ltd. and the Shenzhen Luohu Hospital Group. Southern Medical University Shenzhen Hospital has also collaborated with Shenzhen Zhihulin Network Technology Company to develop a self-service nucleic acid testing sampling machine. Despite the maturity of the technology, there are subtle differences in the solutions proposed by various research teams from the perspectives of human-machine interaction, detection accuracy, and safety assurance. According to Southern Weekend reporters, based on different usage scenarios, the currently developed sampling robots come in two forms. One is mounted on a negative pressure designed vehicle, where air flows unidirectionally outward, achieving mobility and safety with the concept of “the vehicle moves while the person remains still,” such as the “Sairui” sampling vehicle from the Shanghai Artificial Intelligence Research Institute; the other adopts a fixed box structure similar to a sampling booth, with the robotic arm inside and the tested person outside, such as the “Duoke Convenient Nucleic Acid Sampling Booth” developed by Zhongke Xinsong. A major difference among various products lies in the design of the sampling port’s bite device. The bite device resembles a short plastic tube installed on the outer wall of the sampling booth. The tested person bites it, effectively providing a “sight guide” for the sampling swab, allowing the robotic arm to advance through the hollow tube and perform visual analysis and sampling on the exposed throat. Of course, there are simpler designs as well. The product from Jiangyin Furun High-tech Co., Ltd. replaces the bite device with a disposable paper cup for the tested person to bite. Lin Yuan, who participated in the development of the nucleic acid sampling robot, told Southern Weekend reporters that the bite device physically confines the tested person to a certain range, “technically speaking, it is relatively crude and not intelligent, relying on human cooperation with the machine.” However, to expedite the product’s market entry, such designs are also necessary. Currently, he and his team are researching more intelligent solutions that would allow the machine to cooperate with humans through facial recognition and voice guidance, and automatically adjust for human movements. In fact, nucleic acid sampling robots are not a new phenomenon. In 2020, Japanese robotics giant Kawasaki Heavy Industries collaborated with System Cooperation to develop a nucleic acid testing robot system. This system consists of three parts: nasal swab sampling, PCR testing, and medical care. The nasal swab sampling part is operated remotely by a doctor using the “duAro2” collaborative robot, allowing the tested person to communicate with the doctor through a screen on the robot. Domestic nucleic acid sampling robots emerged prominently during the outbreak of COVID-19. In March 2020, the Shenyang Institute of Automation collaborated with the First Affiliated Hospital of Guangzhou Medical University to complete a solution for intelligent robotic throat swab sampling, conducting trial sampling in Shenyang. At that time, the sampling robot, named “Lingcai,” was mounted on a small vehicle and operated remotely by a person, aiming at the sampling area on a computer screen.2
Who is Purchasing Sampling Robots
Currently, many “Big White” robots have received enthusiastic support from local governments and epidemic prevention departments. In January 2021, the second generation of “Lingcai” made its debut during a mass nucleic acid testing in Shenyang. Compared to the first generation of “Lingcai,” the upgraded product features a closed box structure, operated by medical staff who control the robotic arm and communicate with the tested person through a screen. According to reports from Guizhou Daily, the first “Lingcai” in Guizhou Province was put into use at the Third People’s Hospital of Bijie City in mid-June 2022. The latest version of the product requires “no human intervention throughout the process.” Compared to most new generation sampling robots that are still in the debugging phase, Beijing Boao Jingdian Biotechnology Co., Ltd. (hereinafter referred to as “Boao Jingdian”) has commercialized its “Light Cavalry” mobile laboratory for COVID-19 nucleic acid testing, which includes a mature nucleic acid sampling robotic arm. The product resembles a lightweight passenger vehicle, with some models equipped with robotic arms developed by the Mechanical Engineering School of Tsinghua University. Medical staff inside the vehicle control the robotic arm to aim at the sampling area and click a button, allowing the robotic arm to sample through the side window. A staff member from Boao Jingdian told Southern Weekend reporters that the value of the robotic arm and the negative pressure design of the vehicle lies in protecting the sampler. When sampling personnel face viruses more dangerous than COVID-19, an additional layer of protection is provided. The staff member also emphasized that compared to the robotic arm, the highlight of the “Light Cavalry” lies in its rapid testing capabilities. Through the fully integrated chip laboratory system onboard, the entire process from sampling to reporting positive results can be completed within 45 minutes. The entire laboratory system is integrated into a lightweight cargo vehicle, balancing portability and mobility. Currently, the “Light Cavalry” has been applied in mountainous and border areas in the southwest. Previously, the Qingke Jia Research Institute also stated in media reports that since the product’s development began, they have continuously received inquiries from government agencies, disease control centers, and universities regarding purchasing and agency opportunities. Pan Chun, director of the Intelligent Manufacturing Division of Zhongke Xinsong, told Southern Weekend reporters that a larger sampling site requires four medical staff, and if working in shifts, the labor costs add up significantly throughout the day, “this part of the expense can be replaced by robots.” Currently, the “Duoke” sampling booth has been undergoing fixed-point testing in areas such as Pudong, Xuhui, and Jing’an in Shanghai. Regarding product pricing, Zhongke Xinsong stated that it has not been formally determined yet, “we haven’t reached the pricing stage. Our original intention is to serve society and reduce the burden on healthcare workers.” Lin Yuan stated that the sampling robot he participated in developing has custom-made core components, and although the complexity is high and requires continuous debugging, they can keep the price under 1 million yuan. “Some sampling robots are priced high because they use off-the-shelf solutions, such as repurposing industrial assembly line robots. In fact, sampling work is not that complicated; some are over-engineered for the task.”3
How to Achieve Fully Automated Sampling
Replacing human doctors with robots to perform surgeries is a scene found only in science fiction and represents the ultimate pursuit in the field of medical robotics. In a sense, the newly emerged fully automated nucleic acid sampling robots have inadvertently achieved this leap. To fully realize the goal of “machines replacing humans,” an automatically operating sampling robot first needs “eyes.” A researcher from a Shanghai robotics company told Southern Weekend that when designing the visual recognition system for nucleic acid sampling robots, the company mobilized employees to collect throat photos, resulting in over 5,000 samples. The researcher stated that different individuals and shooting angles result in varying appearances of the throat wall in the photos, “some people have swollen tonsils, while others have none at all.” The development team annotated the positions of the tonsils and throat wall on each photo, allowing the machine to learn until it could accurately identify the sampling area. Once the visual system was resolved, another technology closely related to safety is the force control system. To ensure the safety of the robotic arm during the sampling process, the development team first set the safe working range of the robotic arm through research—approximately 12 centimeters for adults and shorter for children. Additionally, during the sampling process, force control technology is employed throughout, sensing the sampling force in real-time. After multiple validations, the sampling force is controlled between 0.2 to 0.3 Newtons, which is not difficult for a robotic arm capable of achieving motion precision of 0.02mm. This ensures both the accuracy of the sampling and the comfort of the sampled individual, achieving gentle sampling. Lin Yuan stated that the standard documents for nucleic acid sampling—the “Nucleic Acid Sampling Training Program for COVID-19” and the “Work Manual for Nucleic Acid Testing in Medical Institutions (Trial Second Edition)”—only outline the general area and methods for sampling, without specifying the force magnitude. When setting the force of the robotic arm, they referenced the clinical research results of the first generation of “Lingcai” robots. Southern Weekend reporters noted that the aforementioned clinical research results were published in the August 2020 issue of the “European Respiratory Journal.” Researchers manipulated the robot to sample from subjects using four different force levels, finding that the best results occurred with a force between 20 to 40 grams, while subjects reported fewer side effects such as throat pain, thus recommending a force of 40 grams for sampling. Professor Huang Tiejun from the Computer Science Department of Peking University told Southern Weekend reporters that the machine vision capabilities required for nucleic acid sampling robots can now be achieved by current artificial intelligence, “it is not a mysterious technology,” but one should not fully trust machine vision. He suggested implementing a verification method to identify erroneous visual recognition results, then adjusting the lens or changing angles. Only when a high confidence level is confirmed should the robotic arm continue operating. Professor Shan Shiguang from the Institute of Computing Technology, Chinese Academy of Sciences, also told Southern Weekend reporters that as long as sufficient data is collected, visual recognition is not difficult to achieve. To enhance data collection levels, considerations must be made regarding how the camera captures images, how to illuminate the dark oral cavity, and what to do if the lens is accidentally touched during operation. To address lighting issues, the aforementioned Shanghai company’s team integrated a constant light source into the robotic arm. The heat generated by the light source can also solve the lens fogging problem. To address the issue of camera lifespan reduction due to hydrogen peroxide spray disinfection after each sampling, the research team selected an anti-fog industrial camera. To secure the tested person’s oral cavity without causing discomfort from the tongue depressor, the team also improved the diameter and length of the bite device, producing three custom versions.
On January 13, 2021, in Shenyang, the “Lingcai” throat swab sampling robot made its debut at a nucleic acid testing site, with medical staff observing the robot’s sampling situation. (People’s Vision/Photo)4
Awaiting Medical Device License Approval
Similar to how autonomous driving is classified from L1 to L5 based on the level of intelligence, the field of medical robotics has also proposed a standard for automation grading. In December 2021, in the inaugural issue of the journal “Science Robotics,” Professor Yang Guangzhong from Imperial College London, one of the journal’s founders, proposed classifying medical robots into six levels based on their degree of automation. The lowest level, Level 0, indicates no automation, where the robot’s actions are entirely controlled by the operator’s intentions; the globally most widely used surgical robot, “Da Vinci,” falls into this category. The highest level, Level 5, indicates that the robot can independently perform surgeries without supervision. Currently, research teams both domestically and internationally have completed animal experiments with fully automated robots, but no related products are available on the market. From remote control to autonomous operation without supervision, sampling robots seem to follow a similar evolutionary path to surgical robots. However, as Lin Yuan stated, “nucleic acid sampling only involves contact with the body surface, unlike surgical robots that need to excise human tissue, deal with bleeding, and suture wounds,” making sampling robots unsuitable for the surgical robot classification proposed by Yang Guangzhong. Because inserting a cotton swab into the nasal cavity constitutes a medical procedure, sampling robots must obtain medical device certification before being put into use. After developing the nucleic acid testing robot system, the mayor of Kobe City, Japan, clearly stated at a press conference in June 2020, “We will absolutely not deploy this robot until we obtain medical certification.” An analyst from a medical industry brokerage firm told Southern Weekend reporters that since sampling robots have already touched sensitive areas of the human body, they should normally be classified as Class III medical devices. Previously, a few surgical robot products have passed innovative medical device reviews and followed a “green channel.” However, this review only applies to devices with significant innovations that address disease needs, “I personally feel that nucleic acid sampling robots face considerable challenges.” In the medical device registration certificate database of the National Medical Products Administration, Southern Weekend reporters searched for “robots” and found no products clearly related to nucleic acid sampling. Some interviewed companies stated that they have maintained communication with local governments since the design phase and are currently awaiting medical device license approval. However, due to the lack of precedents, “the Health Commission has not yet determined which category of equipment it belongs to.” The aforementioned Shanghai manufacturer stated that after the prototype is produced, it will first be trialed within the factory, and they have already iterated to the third generation, awaiting approval for medical device-related licenses. While waiting for approval, companies have begun to proactively establish industry standards for nucleic acid sampling robots. Pan Chun stated that Zhongke Xinsong has recently established corporate standards for sampling robots in collaboration with Shanghai Electric Science Research Institute to regulate the design and production of their products. In the next step, the Shanghai Electric Science Research Institute will lead industry experts to jointly discuss the group standards for nucleic acid sampling robots.(Lin Yuan is a pseudonym at the request of the interviewee)Others are watching:
