Advancements in Agricultural Robotics Research

By Zhao Chunjiang

1 Agricultural Robots

Scientists around the world are continuously exploring robotics. In 1920, Czech writer Karel Čapek’s science fiction play “R.U.R.” introduced the term ‘robot’, derived from the Czech word ‘Robota’, meaning slave. At the first robotics conference in Japan in 1967, Ichiro Kato proposed that robots consist of three essential components: brain, hands, and feet; they possess both non-contact and contact sensors; and they have sensors for balance and proprioception. Chinese scientists define robots as automated machines that possess certain intelligent capabilities similar to those of humans or biological organisms.

The definition of agricultural robots (Agribots) on Wikipedia states that they are robots used for agricultural production activities. From a fundamental perspective, agricultural robots refer to multi-degree-of-freedom autonomous equipment used in agricultural production, which possess perception, decision-making, control, and execution capabilities. This mainly includes information perception systems, decision control systems, operational execution mechanisms, and autonomous mobility platforms, akin to “eyes, brain, hands, and feet.” The information perception part is based on a multimodal information perception system (human senses) that includes visual, tactile, auditory, and gustatory technologies to perceive the spatial environment, target locations, and forms; the decision control part is reflected in the machine’s brain, as well as in object recognition, scene analysis, path judgment, and task planning (human brain); the operational execution part is reflected in efficient and robust robot-specific drives and end effectors (human hands and feet). In practical engineering applications, agricultural robots are integrated with artificial intelligence, big data, cloud computing, and the Internet of Things, forming an agricultural robot application system that enriches the connotation and extension of the agricultural robot concept.

Currently, the labor force engaged in agricultural production in China is continuously decreasing, and labor costs are rising, leading to a rapid increase in the demand for agricultural robots. In 1991, agricultural labor in China accounted for 61% of the total labor force, now it is below 25%; in developed countries that have achieved agricultural modernization, like the United States, it is about 1%. In the future, with urbanization, rural labor will continue to decrease, leading to an increasing demand for robots. At the same time, China also faces an aging problem; since 2016, the proportion of elderly people has rapidly increased, which has also led to a rapid growth in the demand for robots in agriculture. This trend is evident not only in China but also globally, especially in Japan. According to the “World Population Prospects: 2019 Revision,” by 2050, one in every six people in the world will be aged 65 or older (16%), and in Europe and North America, one in every four people will be aged 65 or older. In 2018, the global population aged 65 and older surpassed the number of children under five for the first time in history. Furthermore, the population aged 80 and older is expected to double, increasing from 143 million in 2019 to 426 million by 2050.

2 Current International Status of Agricultural Robots

Agricultural robots are developing rapidly, and the diversity and complexity of agricultural tasks determine the diversity of agricultural robots. They can be classified by application areas, such as field agricultural robots, greenhouse agricultural robots, forestry robots, livestock robots, and aquaculture robots.

The international academic community is increasingly focusing on agricultural robot research. In 2017, “Time Magazine” awarded its annual Best Invention Award to the Hortirobot weeding robot; in 2008, the IEEE Robotics and Automation Society (IEEE RAS) established a dedicated academic committee for agricultural robotics and automation; in 2009, the leading journal in ground mobile robotics, Field Robotics, organized a special issue on agricultural robots; in 2010, IEEE RAS and the International Federation of Automatic Control (IFAC) co-hosted a discussion on agricultural robots called Agicontrol; the EU funded a series of agricultural robot projects under “Horizon 2020,” such as CROPS, Sweeper, and MARS; in 2017, the UK launched the world’s first unmanned farm project, HandsFreeHectare; in 2018, the World Robot Conference (WRC) featured global experts focusing on agricultural robots, and the same year, the top robotics conference (IROS) organized three formal sessions on agricultural robots, marking a shift in focus towards agricultural robotics; in 2019, the World Robot Conference established a forum for agricultural robot applications; in 2021, the International Conference on Agricultural Robotics and Automation (ICARA) was held in the UK. Since 2018, many experts have discussed agricultural robots in various relevant venues. In recent years, at the annual World Robot Conference held in Beijing, there has been a dedicated section for agricultural robots, which has received widespread attention from academia.

According to our analysis of keywords from top international conferences ICRA and IROS in recent years, the hot topics in frontier robotics research mainly concentrate on deep learning, mobile path planning, autonomous learning and adaptation, simultaneous localization and mapping (SLAM), multi-robot systems, optimization computation, and optimal control, representing the international robotics research trend towards autonomy, intelligence, integration, and collaboration. Globally renowned agricultural robot companies and their representative products have emerged like mushrooms after rain, including those for transportation, harvesting, weeding, and sorting agricultural products…

Agricultural robots play an important role in harsh environments unsuitable for human operation and in scenarios with high labor demand. For instance, foreign weeding robots can prevent the poisoning risks associated with manual pesticide application, while also being significant for environmental safety and agricultural product safety. Although the application scenarios of such robots are relatively simple, they connect the entire process from perception to decision-making, control, and operation, presenting a promising blueprint and prospect for the future.

Currently, from a global perspective, the market status of agricultural robots shows that unmanned tractors, spraying drones, and milking robots are the three largest market shares. Agricultural robot companies in Europe and the United States focus on various aspects of complex operations such as harvesting and weeding, such as the top robot vision institution CMU (Carnegie Mellon) with its Robotanist, and the Rowesys weeding robot from ETH (Swiss Federal Institute of Technology Zurich). Internationally, active research institutions in agricultural robotics include Washington State University in the USA, Wageningen University in the Netherlands, Harper Adams University in the UK, and the University of Bonn in Germany.

The EU specifically supports the AgROBOfood project, which divides regions according to different production characteristics and farming types, and develops agricultural robots suitable for agricultural production in those areas; AgROBOfood connects robot research and agricultural business by establishing a sustainable digital innovation center network; AgROBOfood will closely cooperate with a broader European robotics team to ensure synergy through programs like EU-Robotics, enhancing the efficiency and competitiveness of European agriculture and food while reducing dependence on human labor.

The EU has also funded groups of agricultural robot projects, including sweet pepper harvesting robots (AI&ROB), data-driven dairy farmer decision-making (IOT), food and farm IoT (IOT), and data-driven bio-economy (BigData), to promote more efficient and sustainable agricultural development in Europe, demonstrating a high level of attention to agricultural robots.

Some international consulting agencies, such as American Business Insights, have predicted the agricultural robot market, estimating that the global agricultural robot market size will be $7.4 billion in 2020 and will reach $20.6 billion by 2025, with a compound annual growth rate of 22.8%; the Sandler Research Institute has also made predictions, estimating that the market size will be $4.6 billion in 2020 and can reach a market value of $20.3 billion by 2025, with a compound annual growth rate of 34.5%.

In the current pandemic context, the development of agricultural robots is accelerating. Figure 1 shows Tractica’s forecast for the revenue and export global market of agricultural robots, which indicates that by 2025, agricultural robots will reach $87.9 billion, with 720,000 agricultural robots being put into production.

Advancements in Agricultural Robotics Research

3 Development Status of Agricultural Robots in China

1. Overall Situation of Agricultural Robot Development in China

Research on agricultural robots in China began in 1990.

Over the years, Chinese researchers have conducted extensive research on agricultural robots based on the characteristics and demands of agriculture. Significant research work has been carried out in areas such as autonomous tractors, small agricultural mobile platforms, grafting robots, transfer robots, fruit sorting robots, harvesting robots, field information monitoring robots, and plant protection drones, with good applications already achieved in autonomous agricultural machines, plant protection drones, and milking robots. On July 8, 2017, China’s first agricultural unmanned system application demonstration base was launched in Xinghua, Jiangsu. In 2017, the sales scale of civilian drones in China reached 6.2 billion yuan. According to research data from the Huifei Drone Application Technology Training Center, there are currently over 100,000 drones, with an annual operational area of approximately 1 billion mu, and the relevant industry market size is nearly 100 billion yuan.

2. Current Status of Typical Agricultural Robots in China

1. Harvesting Robots

Harvesting robots can be used for fruit picking under different height levels in high-rise cultivation modes, improving the practicality of intelligent harvesting robots and facilitating further development of smart harvesting equipment for industrial applications. From visual positioning to flexible harvesting, relevant technologies for cucumber harvesting robots have been developed; tomato harvesting robots utilize vision servo technology for laser measurement of target space, employing a dual-threaded “eye-hand” cooperative operation mechanism, aimed at robotic operational system integration in non-mechanized agricultural environments. However, the current harvesting efficiency of harvesting robots is still relatively low, with the fastest cucumber and tomato picking speed being approximately 11 seconds per fruit, while in China it is 13 seconds. In the future, if artificial intelligence technology, especially through algorithm enhancements, can quickly identify and locate targets, it will further improve harvesting operation speeds.

2. Seedling Bed Monitoring and Management Robots

Seedling bed monitoring and management robots obtain seedling leaf image information based on environmental light fluctuation compensation, employing an active gripping-type plug tray seedling transplanting manipulator to separate good seedlings from bad ones through an intelligent management operation platform.

3. Apple Harvesting Robots

Apples are one of the largest fruits produced globally, with China’s apple production accounting for over 55% of the total, covering a planting area of 2 million hectares and an annual output of approximately 43 million tons, accounting for 20% of the total fruit production in the country. The labor cost of apple production accounts for as much as 66% of the total cost, and due to the low mechanization rate in the harvesting process (less than 3%), the comprehensive manual harvesting efficiency is 200 kg/day. There is an urgent need for an integrated “harvest-transport” apple harvesting robot.

The typical tree shape in standardized orchards in China is a low-density, high-spindle shape. Currently, we have made significant progress in key technologies for target recognition and harvesting execution mechanisms, but there is a need to strengthen the integrated research and development of efficient harvesting robots suitable for China’s agronomic conditions.

In practice, the dense foliage and fruit clustering within tree canopies, as well as the obstruction caused by overlapping fruits, limit the robot’s information perception. During dynamic operations, the random displacement of fruit poses challenges for the robot’s precise operations. Understanding the motion coupling relationship between target fruit and the execution mechanism in complex near-far vision fields is a necessary prerequisite for the robot’s visual servo dynamic tracking control. In the past three to four years, our team has conducted extensive research on apple harvesting robots and has achieved good progress, but there is still some distance to commercialize. Additionally, we are exploring harvesting robots for kiwifruit, but it is quite challenging, and currently, there are no commercialized products.

For apple harvesting robots, countries around the world have explored different paths and models. For instance, the USA has made significant changes to the structure of apple tree canopies, flattening the entire trunk so that apple trees resemble a fence, greatly reducing canopy thickness, making it easier for robots to identify and harvest apples. At the same time, some countries are also exploring unmanned aerial vehicle harvesting models.

4. Other Robot Systems

Other robot systems, such as health inspection robots for poultry houses in the livestock industry, feature unattended operation and possess multi-modal information fusion analysis capabilities, including visible light, thermal imaging, and voiceprint information. They can detect harmful gases such as carbon dioxide, ammonia, and methane in chicken farms and also monitor animal vitality. For example, the health status of chickens can be assessed based on their vocalizations, enabling abnormal inspections of egg-laying hens from physiological and behavioral perspectives. Additionally, there are robots for disease prevention and disinfection in livestock farming environments, supporting remote control and automated operation modes, capable of intelligent, timed, targeted, and quantitative disinfection operations, which are already in use in many farms across the country.

Overall, there is a significant gap between the development of agricultural robots in China and that in developed countries. Firstly, the integration of machinery and agronomy is not tight enough; secondly, indicators such as stability, failure rate, and ease of use need improvement; thirdly, costs are high, and production efficiency is low; fourthly, the degree of intelligence is not high, and there is a significant gap in core algorithms.

4 Prospects for Agricultural Robots

(1) Characteristics of Agricultural Robots

Agricultural robots have unique characteristics compared to industrial robots. Firstly, agriculture involves unstructured environments, as there are no identical fruit tree structures globally. Additionally, crops obstruct each other, and the color and texture are complex, with variations in lighting conditions between sunny and rainy days, as well as between day and night; secondly, agricultural objects have biological characteristics—mobility, softness, and marketability; thirdly, high technology is required, but costs must remain low, as high costs hinder widespread adoption due to low agricultural profits. These combined factors create significant challenges for the development of agricultural robots.

In the future, robots will not merely be standalone entities; they will integrate with big data, cloud intelligence, and the Internet of Things to form intelligent agricultural robot systems. Unmanned farms fall into this category, where highly intelligent machines combined with network-based control create autonomous operational systems, representing the future direction of agricultural robot development.

To develop agricultural robots in the future, research should focus on different components such as the “eyes” (intelligent perception), “brain” (decision control), “hands” (dexterous execution), and “feet” (autonomous mobility), as shown in Figure 2, which includes many key core technologies. Furthermore, scene perception is very important, as many objects are constantly moving within the environment, creating a unique scenario where robots must predict, decide, and control in advance, rather than simply targeting specific objects, necessitating corresponding predictive decision-making capabilities.

The core of the new generation of artificial intelligence is data, algorithms, and computing power. As the new generation of artificial intelligence continues to develop, future research on agricultural robots should focus on the new three elements—scene, system, and computing power. These new three elements are very complex, especially the scene aspect, where perception and advance prediction need well-developed algorithms. Agricultural robots represent an excellent application scenario for the new generation of artificial intelligence, and efforts should be made to transform the soft algorithms of AI into the hard capabilities of robot intelligence. Future high-level agricultural robots will certainly require technological support from artificial intelligence.

(2) Future Bottlenecks in Agricultural Robot Development

1. Operational Scene Challenges

(1) Complex and variable environments. Agricultural environments have typical unstructured characteristics and many uncertain interference factors, such as diverse lighting conditions and weather disturbances like wind, rain, and dust, posing challenges to the adaptability of agricultural robots.

(2) Complex and variable objects. Compared to standardized industrial objects, agricultural objects have their own characteristics, posing challenges to the robot’s object recognition/positioning capabilities and flexible/soft operation capabilities.

2. Theoretical/Technical Bottlenecks

(1) Scene perception. Deep learning has achieved great success in image and point cloud recognition, but there are still bottlenecks in addressing agricultural object recognition. Strengthening research on deep learning for image and point cloud recognition will facilitate breakthroughs in perception under complex scenarios.

(2) Behavioral planning. Agricultural robots face certain uncertainties in perception observation, action execution, and cooperative coordination, posing challenges for efficient and safe planning under multi-source uncertainty conditions. For example, when injecting chickens in a farm, they are moving, and we should have prior knowledge of their next actions and directions to accurately target during the injection.

(3) Execution systems. The execution system of agricultural robots faces bottlenecks in materials, configurations, power, and control, making it difficult to meet operational task requirements.

3. Market Bottlenecks

(1) High costs and expensive prices lead to low acceptance among farmers.

(2) Lack of comparative advantages in operational efficiency poses a significant challenge. Leveraging the advantages of robots and achieving integration between agricultural machinery and agronomy, as well as parallel operation of multiple machines, is the future direction of development.

(3) Strengthening Technological Innovation in Agricultural Robots

Introducing new technologies is a potential variable for achieving high-level agricultural robot innovation and development.

By enhancing the perception and decision-making capabilities of agricultural robots through deep learning and core algorithms, especially using deep learning methods and technologies for recognizing phenotypic features, scenes, and crop diseases, optimizing movement paths, operational postures, and sequences. Research on tactile feedback control can enhance the perception and execution capabilities of agricultural robots, preventing excessive gripping force from damaging product quality during harvesting. Human-robot collaboration can improve operational efficiency and significantly enhance machine intelligence, greatly reducing costs, which is a method to address current challenges under complex scene conditions and further improve machine quality. Multi-agent theory is expected to enhance the collaborative operational efficiency of agricultural robots, as the autonomy, distribution, and coordination characteristics of multi-agents are conducive to maximizing the efficiency of agricultural robot collaboration and multi-machine operations. Multiple robots can simultaneously complete a common task, and if one robot encounters an issue, others can continue the task. Multi-arm parallel operations can improve the efficiency of harvesting robots, as seen in the USA’s Harvest Croo strawberry harvesting robot and Spain’s Agrobot strawberry harvesting robot, which adopt this approach. Although multi-arm parallel operations can significantly enhance the efficiency of harvesting robots, they also face challenges such as improving agronomic processes, complex auxiliary equipment for power/control, and task coordination and planning among multiple arms. Research into new materials can improve the execution capabilities of agricultural robots, such as the potential of nanomaterials, composite materials, and 3D printing to support the future development of agricultural robots. The key areas for future innovation research in agricultural robots are as follows.

1. Intelligent Perception Technology (“Eyes”)

There is a need to strengthen perception technologies, including sensor devices, feature extraction, and achieving information fusion (see Figure 3). Information fusion is the basis for our decision-making.

2. Intelligent Decision-Making Technology (“Brain”)

To improve the level of intelligent decision-making, it is necessary to combine algorithmic tools with operational decision-making (see Figure 4), and a bus control platform is also needed to enable robots to operate quickly according to human requirements under controllable environmental conditions. In algorithmic tools, task coordination and optimization, deep learning, knowledge representation and reasoning, and knowledge-driven methods are all very important.

3. Dexterous Operation Technology (“Arms”)

As shown in Figure 5, the key technologies for dexterous operations include arm design, operational planning, and control. In arm design, research should focus on the physical characteristics of materials and objects; integrating dynamic planning, multi-arm task planning, as well as trajectory and collision avoidance planning, while using control methods and techniques to enable agricultural robots to work quickly, effectively, and accurately.

4. Mobile Platform Technology (“Legs”)

The common key technologies for autonomous mobility include mobile platforms, positioning navigation, and autonomous movement (see Figure 6). It is essential to develop mobile platforms suitable for agricultural non-structured walking surfaces (such as platforms for “four-wheeled robots”). Combining various technologies such as satellite positioning, inertial navigation, and visual navigation, including laser and radar, is necessary to ensure real-time control and understanding of trajectories and positions for the robots.

5 Conclusion

This report introduces some basic situations of agricultural robots and hopes that scientists engaged in artificial intelligence research, along with friends from the industry, will venture into the field of agricultural robots to jointly promote the development of modern agriculture in our country and enhance the level of modern agriculture.

(This article is based on shorthand notes)

Excerpted from “Communications of the Chinese Association for Artificial Intelligence”

Volume 12, Issue 10, 2022

Speech Transcript

↓↓↓ Welcome to join our association for more exclusive member benefits

Scan to Join Us

Related posts

Leave a Comment Cancel reply