By now, you should have probably heard about Huawei's latest chipset that will be featured in their October 2017 flagship device – the Huawei Mate 10. Here's a quick refresher on what the new Hisilicon Kirin 970 offers:
Below is a visual representation of the Kirin 970's architecture.
As seen above, the Kirin 970's NPU is a third processing unit (with the traditional two being a CPU and a GPU). It offers 25 times the performance of the standard CPU at half of the CPU's size while consuming just 1/50th of power that's needed to fuel CPU that shares the chipset.
According to Huawei, the challenge in creating such tiny, yet powerful processing units lies in a regular smartphone's current limits. A phone's chassis lacks room, and it has 2,000 to 4,000mAh to work with. These factors complicate the process of creating a processor for AI tasks with performance and efficiency in mind.
Huawei uses all three processors (CPU, GPU, and now an NPU) in their latest Kirin 970 chipset. It does so by handing off graphical rendering to the GPU, memory allocation and tasks scheduling to the CPU, and finally, AI computing to the NPU alone. This results in reduced stress on the CPU since the CPU in a regular smartphone usually takes on the task of AI computing (and we believe they are using the term “AI computing” quite loosely here).
Below is an example of "A.I. computing", where the NPU is up against its rivals' CPUs in the realm of image recognition.
It's worth noting that the NPU does not have a machine-learning nature – Huawei specifically said that the AI is pre-loaded into the processor, and it uses its programming to do what it must. Should the algorithm require fine-tuning or new features, Huawei will provide them as OTA (over-the-air) firmware updates.
The above image explains how the chipset decides to allocate resources. At the bottom lies all the different processors for different purposes, and the workload is delegated by the Resource Management layer above it. The chipset regards the Android operating system runtime separately from their mobile AI's runtime, which reduces the overall impact on the phone's performance while allowing it to multi-task.
According to Huawei, the NPU also works with the image signaling processor (ISP) for smartphone photography, and it offers AI noise reduction for all forms of recorded video (audio clips, calls, and speech recognition).
Huawei's research into smartphone photography found that users are dissatisfied at the disconnect between the user's intention to snap a photo, and the image processing that comes after. These split-second differences can result in missed, or blurred images, and Huawei wants to help via its AI.
The inclusion of the NPU helps to enhance the ISP's response time by:
To combat blurry-looking photos, the NPU helps via:
If all that sounds familiar to you, that's because the latest Sony Xperia phones (XZ Premium, XZ1, XZ1 Compact) has an image-capturing AI called Predictive Capture. Sony's version helps you capture multiple shots in a single shutter press before it lets you choose the best take. Predictive Capture is entirely decided by the phone. It sounds an awful lot like AI, but it's certainly a different implementation from the Kirin 970's NPU by Huawei.
The NPU complements the ISP in other indirect ways. For instance, having a 3x3 face area detection marker (as opposed to your regular 'square' detection) and its AI can result in better-balanced portraits, especially in low-light situations (example above). The ISP is helped along with other traditional smartphone technologies, such as Hybrid autofocus, face tracking, etc.
In essence, while the camera and ISP perform their duty of 'seeing' the subject, it's the AI Vision that provides the context by reading other elements, such as the environment, the face, body, and even the mood and behavior of your subject. We're not entirely sure to what extent can the AI perform these tasks, but that's how they chose to promote their NPU.
The NPU uses its built-in depth neural network to process recorded audio. In the simple diagram shown above, Huawei claims that it works for voice calls, speech recognition commands, and typically recorded audio. It's not exactly an AI for audiophiles*, but the AI helps to improve voice quality by reducing interference.
*= while the AI may not affect music quality, the Kirin 970 still supports full-path 32-bit audio at 384Kbps.
In a nutshell, the Kirin 970's much-hyped AI processor is a component that plays along with other processor cores, enhancing the overall performance of the chipset in various ways. It's not strictly limited to the tasks we've mentioned, but it can help to moderate our expectations for the incoming Huawei Mate 10 flagship smartphone.