Google announced the LiteRT Tensor API, a lightweight C++ layer for building on‑device AI models. It gives full control over the computation graph while staying portable across CPUs, GPUs (via ML Drift) and NPUs. The API was demonstrated with Meta’s Segment Anything Model 2 running entirely in a browser on WebGPU, achieving over 30 fps at 512×512 resolution. Developers can also load weights directly from checkpoints and keep data on accelerators with zero‑copy buffers.
Why it matters
App developers can ship faster, lower‑latency AI features to phones and browsers without maintaining separate code for each chip.
