July 22-23, the “2020 First Software Defined Vehicle Summit Forum” hosted by Gaishi Automotive and supported by Shanghai International Automotive City was officially held. This forum mainly discusses the latest innovative concepts, technological trends, and practical challenges in the field of software-defined vehicles, aiming to explore the future development of the industry. Below is the speech by Ying Yichen, ADAS Technology Director at Renesas Electronics, at this forum:
Ying Yichen, ADAS Technology Director at Renesas Electronics
Good morning everyone! I am Ying Yichen from Renesas Electronics. I would like to thank Gaishi Automotive for inviting us semiconductor suppliers to participate in this forum on software-defined vehicles. Today, I will introduce how we, as semiconductor suppliers, can better support the trend of software-defined vehicles.
The theme today will focus on ADAS, and I will share some of our products and related information regarding ADAS at Renesas. For those of you in the automotive industry, you may not be unfamiliar with Renesas Electronics. We are a semiconductor manufacturing and design company headquartered in Tokyo, Japan, with factories in Japan, overseas, and mainland China. Many are more familiar with our MCU and SoC products, as Renesas MCUs can be found in various applications within every vehicle manufacturer. Our company currently has over 20,000 employees globally, with two factories in China, one in Beijing and one in Suzhou, and corresponding subsidiaries in Beijing, Shanghai, and Shenzhen.
In the past two years, we acquired two other companies because we felt that simply providing ECUs was insufficient to meet market demands. Therefore, we strive to provide our products in a solution-oriented manner, not just chips but also including peripheral chips, and we collaborate with partners to provide a better ecosystem for the entire supply chain, offering better support to customers in a solution-oriented environment. Following the acquisition of these two companies, we will also have new product lines, including PMIC and BMS, to better support SoCs and MCUs. We will optimize the analog products they previously had to support our SoC and MCU products, while our existing products will continue to iterate and upgrade. Our entire product line is quite comprehensive, covering various applications such as body control, gateways, entertainment, etc., where you can find chips that Renesas can provide. Therefore, the challenges for our chips in various fields are quite high, as the systems in vehicles are becoming increasingly complex, requiring many software applications to run on our chips. We also have some auxiliary devices to provide good support for our MCUs.
Including the high electrification of new energy vehicles, we also have BMS products, which collect voltage or current information through analog front-end chips, and combine them with our MCUs to create a complete BMS solution. In addition, we have expanded our product line for the currently popular ADAS scenarios, including AHL solutions from the companies we acquired, using traditional analog cables to transmit high-definition video. We also collaborate with laser partners to build radar solutions by integrating some radar sensors, reducing the costs of radar and laser.
Furthermore, we also have position sensors to provide good support for motor and gear control. From the current market environment, a single chip is no longer sufficient to meet market demands. Therefore, we are committed to providing customers with controllable solutions, safe ADAS solutions, and environmentally friendly electric vehicle motor solutions, all of which can be found in our product line.
In addition to automotive applications, Renesas Electronics also has product lines related to industrial, IoT, and infrastructure construction, and we welcome everyone to visit our official website for more product support.
The theme this time might be software, as software is an interdependent yet conflicting part of our chips. Cars have developed for many years, and initially, they were primarily mechanical. Many systems might have been under various university mechanical majors. As cars have developed to the present, mechanical components have decreased, and electrification has increased. This is something you may have seen in many ADAS forums; its functions have been continuously evolving. In the early days, ADAS functions were simple, such as LKA and LDW, with relatively simple algorithms and small amounts of code running. Now, the entire automotive system is becoming increasingly large, with code volumes possibly increasing from tens of thousands or millions of lines to over a billion lines of code. The transformation of automobiles by software is significant, and it also poses a considerable challenge for us chip manufacturers. Previously, an MCU with 2K DMIPS could meet most application scenarios, but now, software suppliers may require a chip with 80K DMIPS CPU performance and 60T deep learning computing power, which is a massive challenge for us. We need to balance software performance, power consumption, and product requirements to ensure our chips can be mass-produced in vehicles, rather than just conducting tests. Our goal remains to advance the functions beyond ADAS entertainment towards vehicle mass production, allowing passengers to enjoy a better experience. A simple scenario today may involve many algorithms and functional perceptions, requiring us to have strong computing power to support them.
This is our platform, which covers all aspects of ADAS, including V2X gateway application scenarios, radar and camera sensor perception scenarios, as well as fusion decision-making scenarios between various sensors, and vehicle control scenarios. You can find corresponding coverage from our relevant MCUs, SoCs, and analog devices, so we have also registered a brand to build a better complete ADAS platform.
As mentioned earlier, software cannot run without hardware, and hardware cannot achieve relevant functions without software. Therefore, currently, in addition to providing semiconductor chips, we also provide corresponding software drivers based on the chips and collaborate with our partners to provide operating systems like Android and QNX, creating a complete ecosystem.
Here we can see our R-Car alliance, which includes many partners in the software field, including middleware based on our chips and algorithm OS middleware partners, working together on our platform to create a better ecological environment for customers.
Therefore, our platform provides end-to-end solutions for customers, including cloud services, perception algorithms, perception platforms, and a complete solution for vehicle control. Let me first introduce the gateway part. One of the major features here is OTA, which allows you to upgrade firmware to various ECUs in vehicles through the gateway. The demand for OTA support is increasingly strong as the ECUs in vehicles move towards centralization. A crucial requirement for hardware is the need for hardware encryption and decryption. Our MCU series will have hardware encryption and decryption modules that can verify the transmitted upgraded software images, allowing the required software to be transmitted to various ECUs in the vehicle via CAN or other Ethernet methods to achieve software OTA, enabling rapid upgrades of software functions and bug fixes. Our RH850 hardware has been widely applied, and by combining it with communication modules, we have created a complete Tbox solution.
For perception, we mainly focus on the R-Car SoC product line, which covers many application scenarios within the vehicle, including entertainment with our R-Car D series for instrument cluster scenarios, using our R-Car H3/M3 for central navigation scenarios, and employing R-Car products for large gateways and central domain controllers, as well as our R-Car V series for autonomous driving and front-facing perception scenarios. Therefore, our R-Car product line is extensive and can cover various application scenarios within the vehicle.
This is a more complete product line of R-Car. Currently, the third-generation R-Car products are in mass production, and the most advanced one is the R-Car H3, which is an 8+1 core chip that can cover different scenarios. For ADAS, we mainly promote the R-Car V3 series of chips, which have many internal structures that are quite similar, including internal IP settings, so for software, its upgrade and reusability are relatively high.
Here are our product series. The internal IP pairing of our chips is also a combination of software and hardware. We have many IPs for visual acceleration. Our basic ARM cores can handle tasks with high flexibility or task scheduling. For visual algorithms, we may need dedicated hardware acceleration, so we have placed many acceleration boxes here, including our programmable computing CV Engine for implementing custom algorithms. We also have highly hardware-accelerated IMP cores, as well as CNN IP for deep learning, which is essential for ADAS. Additionally, three special IP cores will be included in V3H, and all these IPs work together to provide excellent acceleration for the entire chip.
Regarding algorithms, although algorithms may seem like a software concept, for our chips, it is not just software. How to use our hardware IP to accelerate algorithms is also a significant challenge for our company. You can design the chip to have high flexibility, but flexibility may come at the cost of power consumption and price. Therefore, balancing performance, power consumption, and price factors is also a design direction for our Renesas chips. Here we can see that we have combined many fixed and flexible IPs to balance various computing powers within the chip.
Here, we can first look at the internal block diagram of our V series. This chip is primarily based on the ARM A53 core. Our chip has many IP cores, including the IMPX5 part and the CNN part, as well as commonly used interfaces, DDR interfaces, and our chip’s built-in ISP, which can save the camera’s ISP. One of the headaches for software development is the poor compatibility between chips, which requires many adaptations for each chip and additional development for the middleware. However, for Renesas Electronics, our compatibility is quite good, allowing for seamless switching. We have successfully ported many algorithms to V3M, and the corresponding algorithms can be easily migrated to V3H, which also has an A53 core and can also handle functional safety-related features. Its internal structure is almost identical, with only relevant upgrades made, including enhancing the CNN functionality and increasing the number of IP cores to improve overall computing power. Therefore, when using our chips, you don’t have to worry about many migrations between different chips; in fact, our chips have excellent compatibility, making software development relatively easier.
Here we also mentioned the design philosophy of our chips, which emphasizes scalability, flexibility, and sustainability. You can see from this that we have various application scenarios and levels, ranging from low-end to mid-range and high-end, targeting different scenarios. Our chips are quite flexible, integrating various application scenarios, and the main IPs and architectures of our chips are the same. Our new products will also reuse some of the IPs from our old products, making upgrades and modifications, and then incorporate them into our new products. Therefore, you don’t have to worry about needing more development resources for our new products in the future, making it more convenient for customers to use our products.
Now, let’s look at the decision-making part. The demand for our chips in the decision-making part is more focused on CPU, bandwidth, and peripherals. This entire system will have many sensors connected, with many cameras; currently, vehicles may have two to six camera connections, along with many millimeter-wave, ultrasonic, and sensors connected, and some vehicles have even begun to deploy LiDAR sensors connected to our chips. This multi-sensor connection places high demands on the CPU and bandwidth of our chips. Therefore, we introduced the H3 product, which has many powerful CPUs inside to provide high support, helping customers implement some path planning-related algorithms on our chips, and its bandwidth is wide, with rich interfaces. However, the H3 chip lacks some deep learning support. Therefore, if you require deep learning support for this platform, we recommend pairing it with a V series product to achieve a more complete ADAS platform.
Of course, we are also planning new products that will combine CPU chips with good deep learning support, launching single-chip solutions to provide better chip support for L3, L4, and L5 platforms in the future. Here is a general internal block diagram of the H3, which is an 8-core ARM chip with four A57 and four A53 cores. It also provides four DDR channels, and the four-channel DDR can provide 50G of bandwidth, offering excellent hardware support for the ADAS platform. You can confidently port many algorithms to the platform without worrying about performance issues, and its peripheral interfaces are also quite rich, including video output and input, etc.
Below the H3, we have the H3N chip, which is nearly identical in internal structure to the H3, except it lacks two DDR channels. Below that, we have the M3, which has two fewer large cores, becoming a chip with two A57 cores, while the rest are almost the same. Therefore, our product line has considerable coverage, ranging from the high-end H3 to the M3 and M3N. The H3N, M3, and M3N chips can all achieve pin-to-pin compatibility, meaning that software has excellent portability, allowing for multi-dimensional support for high-end, mid-range, and low-end application scenarios, making it quite convenient for software development with our chips.
As mentioned earlier, the advantage of ARM cores is their good compatibility since many chip developments on the market are based on ARM cores. Algorithms from other ARM cores can also be well ported to our platform. This is the bandwidth concept I mentioned earlier. Our H3 has the highest bandwidth of 50G, while the M3 only has half the bandwidth, and the subsequent ones have even less, allowing for different chip combinations to support various customer application scenarios.
Thus, a significant feature of our chips is their excellent compatibility. Our design generally starts from the high-end H series, making modifications or adjustments for different application scenarios, resulting in our M, E series, and other application scenarios for the D series, as well as the V series specifically for ADAS. Therefore, our product design considers compatibility from the outset, and we have already begun designing new products before mass production of our old products to ensure product iteration. We are currently on the third generation of products, and the second generation has been around for quite some time, and we are now planning the design of new fourth-generation products.
In terms of software, we also consider complete software compatibility. Generally, we can support the entire series, and the same BSP can generate corresponding BSPs by modifying options. Thus, our entire hardware platform can provide good support for various application scenarios within the vehicle.
Here we can look at the R-Car software and performance situation. Based on our R-Car semiconductor chips, many OS supports include QNX, Android, AGL, etc., as we also support AUTOSAR due to the presence of Cortex R7 in our chips. Therefore, our support for various software platforms is also quite good.
Deep learning is also a crucial part of ADAS, as most machine recognition algorithms are based on deep learning. Currently, the common way to develop deep learning is based on the PC side, where I may first have a designed network trained with a GPU, combined with accumulated data to generate corresponding networks and parameters. However, this aspect presents considerable difficulty in migrating to edge computing chips, as the structure of GPUs and SoC chip architectures are quite different. Renesas provides a comprehensive toolchain and SDK to facilitate the import of networks and parameters trained on the PC side. Our toolchain will analyze the network to see which layers can be accelerated by internal modules, which modules require CPU calculations, or which require support from our internal IPs, ultimately generating code that our chips can recognize and run on our platform.
Generally, a deep learning network will have many supports, and our toolchain can provide good analysis for many layers, including whether the data is prone to issues and whether each layer’s results require some validation, all of which can be well observed through our toolchain.
Finally, I will introduce our related products for vehicle control. Vehicle control is a critical part, as this area requires the highest safety standards. Therefore, we mainly launch products that comply with functional safety ASIL-D series. Previously, our main products were RH850P series, which are already in mass production and have been tested in the market. We are now launching new products that will also have internal encryption and decryption modules, as indicated in purple, providing good support for online upgrades. Our leather chip supports FOTA application scenarios and operates at a frequency of 400 MHz, offering good MCU support for highly integrated domain controllers and control parts.
Our RH850 MCU-related products also have good software compatibility, as many hardware modules and IP designs from the original F and P series will be referenced in this U series. Therefore, if you are already familiar with our MCU products, you will quickly adapt to our new products.
That concludes my presentation for today, and I would like to thank Gaishi Automotive for inviting us to participate in this exhibition. Thank you!
Gaishi Automotive’s 2020 Annual Special Topic “Software Defined Vehicle Era – Accelerating the Start”. For more content, click “Read the Original” to check it out.
Click “Read the Original” to see more