Huawei Zhang Baofeng: NPU Empowers AI Computing on Devices

Highlights

The 2018 New Intelligence Industry Leap AI Technology Summit has successfully concluded. Click the link to review the grand event:

iQIYI http://www.iqiyi.com/l_19rr3aqz3z.html

Tencent News http://v.qq.com/live/p/topic/49737/preview.html

Sina Technology http://video.sina.com.cn/l/p/1722511.html

Yunqi Community https://yq.aliyun.com/webinar/play/419

Douyu Live https://www.douyu.com/432849

Tianchi Live Room http://t.cn/RnQPhuY

IT Experts Speak http://www.itdks.com/eventlist/detail/1992

New Intelligence Original

Speaker: Zhang Baofeng

Editor: Craig

[New Intelligence Guide]The breakthroughs in big data, algorithms, and computational theory have sparked the third wave of AI. However, on personal devices, the complexity of AI technology and various hardware limitations severely restrict AI development. To overcome these constraints and promote faster and better development of the entire industry, a more open ecosystem is needed, Huawei HiAI has emerged at the right time. At the March 29, 2018 New Intelligence Industry Leap AI Technology Summit, Huawei’s VP of Software Engineering, Zhang Baofeng, detailed how HiAI quickly helps developers and empowers the industry chain.

Zhang Baofeng VP of Huawei Software Engineering

Zhang Baofeng is currently responsible for the development and delivery of Huawei’s terminal AI software. He is the founder of Huawei’s Noah’s Ark Lab and has served as the deputy director of the lab, overseeing mid- to long-term technology research management in the field of data science, focusing on data mining, machine learning, and artificial intelligence. He is a member of the Chinese Nuclear High Base Expert Group and the Chinese CCF Big Data Expert Committee. Zhang Baofeng joined Huawei in 1998 and has over 20 years of experience in the information technology field, with rich experience in international/national standards organization activities, having served as the rapporteur for the ITU-T Study Group 13’s Fixed-Mobile Convergence project and deputy chairman of the Network and Switching Technology Working Committee of the China Communications Standards Association.

Below is Zhang Baofeng’s speech:

It is a great honor to have the opportunity to stand on the New Intelligence stage and share some changes Huawei has undergone in the past few months. I remember the last time I shared here, I talked about how to make AI usable in real scenarios on mobile phones. Recently, many companies in the mobile phone industry have also used AI technology to enhance the effects and efficiency of photography, so I would like to share some thoughts after deploying and launching these changes over the past few months.

The pursuit of openness, equality, collaboration, speed, and sharing is what we call the spirit of the Internet, and it is also the theme I want to share with you today.

The industry generally faces pain points such as computing power, fragmented services, and difficulties in talent acquisition. The NPU releases AI computing power on devices, activating AI application scenarios.

Last year, we faced the main issue of insufficient computing power. In the second half of the year, we launched a chip with an independent NPU on the Mate 10 to accelerate deep learning models.

Now we are encountering another problem: when using AI to accelerate all functions and features on mobile phones, how can users truly benefit from AI? The standard model now is that users download apps, whether shopping apps or hotel booking apps, which are provided by third-party applications. Users may face hundreds of thousands of app choices and service capabilities in all aspects of their daily lives. Solving this problem is currently our greatest challenge in allowing end users to benefit directly.

Another issue is how to quickly turn AI into software technology capabilities. A simple advertising business involves so many complex mathematics to understand. In applications, there are issues with computing power and performance. How can ordinary developers quickly use AI capabilities to solve their application software problems and address user pain points? This is another opportunity we see.

It is hard to find a good general, and good talent is even harder to find. In the current hype around AI, it is difficult to find enough resources to solve industry applications. I believe all companies working on AI will face these basic issues, and what are we doing? We hope to open these resources.

Last year, we aimed to empower mobile phones with AI capabilities, wanting to turn mobile phones into personal assistants. We have only taken a small step in this field, but in the future, we hope to open all three layers of capability to partners.

First is the capability of chips, executing deep learning on devices, which we believe can meet the basic needs of many software applications, such as graphics processing, video segmentation, and object recognition. Efficient execution on mobile phones is also a significant challenge that needs to be realized at the foundation level.

Secondly, we need to simplify capabilities, and we want to open up NLU and CV capabilities. There will be some basic examples later, but I can tell you a number: previously, we worked with a partner to debug a system that took only four hours to transform document recognition, document conversion, and model transformation into native capabilities embedded in their application software. This is how the AI Engine turns mature models into tools that third-party partners can use for industrial applications.

The third is services and ecosystems. When pursuing the entire AI capability, we hope to start from the final user’s perspective to understand how users can benefit from which businesses. Therefore, we hope to quickly reach all Huawei users with these capabilities.

These three layers are actually how Huawei quickly empowers the entire industry chain with AI capabilities, accelerating the maturity of the AI industry. We have several advantages: first, the execution efficiency of models. When executing models on devices, especially mobile devices, a significant problem often arises—not just computing power but also the computability issue. NVIDIA emphasizes achieving 118.5/Turing Flops on GPUs, but we need to talk about the indicators of power consumption per watt.

Another advantage is simplicity and efficiency. When mature models are nested in third-party applications, we aim for hour-level assembly so that development can quickly empower the software itself. A complete toolchain is something all developers dream of, and finally, there is a mature service ecosystem.

Typical case: 50x performance ratio, 20x efficiency improvement, and 300% improvement in edge-side operational efficiency.

The following image shows the specifications of our mobile phone from last year, with a 50x performance ratio, 20x efficiency improvement, and one service achieving nearly 300% efficiency improvement, with a single frame rendering time of 2.5 seconds.

Next, we worked with a foreign company called Prisma, where you can click to see the basic execution time.

Prisma is very popular in Europe, emphasizing how to turn a picture into an artistic style, claiming to transform “bad pictures into artifacts,” rendering in the style of Van Gogh’s oil paintings. Typically, rendering one frame takes 10 seconds, which demonstrates the acceleration effect we achieved. In the future, execution efficiency on the edge side for graphics/image-based applications is a very essential indicator, so applications need to execute quickly and smoothly as a fundamental requirement.

In terms of application development, we have accumulated a lot, surveying nearly 300 partners in China, with the main needs still being in CV and ASR.

China also has many companies providing such capabilities, and we emphasize the hope to empower edge-side execution so that users can truly enjoy the convenience brought by AI, even without an internet connection, mainly focusing on live streaming, social platforms, photo processing, and more.

We aim to empower the entire industry chain with these capabilities, hoping that these tools and AI models can be integrated into their application software in the lightest way, such as during shopping, including QR code scanning, service recommendations, identity recognition, etc., which can all be encapsulated into an API. Once we have such capabilities, it allows partners to work faster.

Below is our own photo album, whose basic function is to find the best photo among a series of pictures. During our user research, we often encountered a problem: users take many photos in the same place and scene, but finding the best photo to share can be quite painful. We used an aesthetic scoring model, which means that for our photo album application, developers only need to embed two lines of code into their business logic.

Huawei Zhang Baofeng: NPU Empowers AI Computing on Devices

The same model has also been implemented by Prisma, where the scene recognition model was previously used in cameras, but Prisma has many styles. If this photo is a portrait, how can we find the style change pattern that best matches the portrait style, or how to find the most suitable pattern for landscapes? These are its challenges, and during integration, it typically takes just a few lines of code to embed a scene detection capability based on the image itself. We can turn it into a part of the code within minutes using Java and JSON, and then within hours, it can become a functional part of the application.

Below is our service direct access, including service atomization. A few days ago, we also formed a light application alliance that does not require installation. Here, we feel that with the deployment of AI capabilities, users will face confusion in service selection, as there may be several different choices in appropriate scenarios, such as for rides or shopping.

How can these basic capabilities effectively reach users at the right time and in the right scenario? This needs to be solved based on the user’s own scenarios, preferences, and continuous learning.

Mature service ecosystem for win-win with partners, covering over 350,000 developers.

We will turn these into individual entry points, allowing the highest quality service providers to find the true needs of users, creating a win-win situation. We hope that the AI capabilities inherent in mobile phones can enable these truly high-quality service providers to quickly promote their services to users, which is our effort and attempt in application promotion.

Currently, the Huawei HiAI ecosystem has benefited 340 million end users, and Huawei has an annual shipment of 150 million units, which is the foundation for serving a wide range of users. By sharing this across the entire industry chain with third-party developers, it amounts to about 350,000+ scale.

Through this ecosystem, we aim to quickly benefit Huawei users. Huawei focuses on the actual experience of mobile phone users; we do not want AI to be just a marketing gimmick but hope to enable end users to truly benefit.

We also need to focus on the IDE, integrating the capabilities of Foundation and Engine into the Studio, allowing for model debugging with just a few buttons, and even connecting with their simulation environment, enabling one-click integration of AI capabilities into edge-side software. What we mainly emphasize here is the integration time; we do not want AI to be a myth, but hope that every application can enjoy such a foundation.

We are building a developer alliance, which is the platform we are currently operating. Recently, we have also collaborated with New Intelligence for many deployments and promotions. Our true dream is to enable the entire industry chain to serve Huawei mobile phone users as quickly as possible.

We hope to build such an ecosystem to quickly execute software under considerable pressure. What we often say internally is Make it possible. We are also collaborating with Douyin to see how to accurately engrave a person’s object. If it were previously on CPU and GPU, the phone would quickly overheat and would not be able to sustain, hoping that users can get the best experience through such mechanisms, while also being simple and very easy to promote AI as part of native applications. Finally, we aim to create a win-win effect within the entire ecosystem, allowing users of Huawei phones to enjoy the best services, conveniences, and experiences brought by artificial intelligence.

These are the themes we wanted to share with everyone today, to bring AI down from the pedestal, not to overhype what AI can bring, but to truly focus on what kind of user experience and services AI can provide from the user’s perspective. Only in this way can we drive the healthy development of the entire industry chain.