Imagining WebNPU API

Warning

This post is more than a year old. Information may be outdated.

WebGPU is a recently developed web interface that provides access to modern graphics processing unit (GPU) capabilities, such as Direct3D 12, Metal, and Vulkan, for rendering and computation tasks. Its aim is comparable to WebGL APIs, but WebGPU offers access to more advanced GPU features. What if we could do this with NPU? I hereby suggest a hypothetical WebNPU API. Think this way:

WebAssembly unlocks the bare-metal power of CPUs.
WebGPU unlocks the bare-metal power of GPUs.
WebNPU unlocks the bare-metal power of NPUs.

We are already observing blooming On-device inferencing ML technology with Apple's Neural Engine. This includes many jobs, from simple Siri commands to full-blown Stable Diffusion models. While they are way weaker in power compared to full-server scale inferencing, they have far better latency and privacy, which are required for some specific jobs. Other companies are catching up, such as Google's Tensor. Unfortunately, so far, these chips are only accessible through native applications. For web-based solutions, we have high-level services running on the CPU, like TensorFlow.js.

At one point in the future, when AI-native (just like us Cloud-Natives) become the norm, there will be more and more need for an On-device API to access NPUs, especially with Web technologies. These are good for all stakeholders:

Backlinks3

230218 230214 30-Day Tweet Test (Harry Stebbings)

Comments0

No comments yet. Be the first to share your thoughts.

Comments