LiteRT in Google Play services is Android's on-device inference runtime for custom models, handling more than a trillion inferences a day across billions of devices. Today, it gets its biggest upgrade yet: GPU inference that runs about 1.4x faster than the legacy GPU delegate, powered by ML Drift and available through the newly introduced CompiledModel API, LiteRTโ€™s modern successor to the legacy Interpreter API.

Until now, taking advantage of this faster GPU stack required bundling LiteRT in your app. In our testing, apps that switched to LiteRT in Google Play services are about 8 MB smaller, making them faster to download and lighter on users' devices, because the runtime, including the GPU accelerator, ships once with Play services rather than inside every app. More than 100K Android apps bundle their own copy of LiteRT or TensorFlow Lite today, so the savings add up quickly as apps switch.

If your Android app runs custom ML models on CPU or GPU, we encourage you to switch to LiteRT in Google Play services. You can see it in action in the Google AI Edge Gallery app, where the Scrapbook feature's image segmentation runs on LiteRT in Google Play services.

Developers that adopt LiteRT in Google Play Services enjoy immediate benefits:.

LiteRT is the evolution of TensorFlow Lite. It keeps the same .tflite model format you rely on and adds the CompiledModel API, which makes acceleration across fragmented hardware reliable:.

Whether you're starting fresh or switching from bundled LiteRT, using LiteRT in Google Play services takes a few lines of code:.

โ†’ Read original article