Your users' conversations never leave the phone.
Run LLMs on Android with one Gradle line. Chat completion, embeddings, GGUF models — on-device, no cloud, no API key, no per-token billing. Private by design, works offline, real-time inference.
// llama.cpp · GGUF models · arm64-v8a · API 24+ · Android 15 ready · 16 KB aligned
Cloud LLM APIs see everything your users type.
Every prompt sent to ChatGPT, Claude, or Gemini APIs leaves the phone. For health, legal, financial, or personal use cases — that's a dealbreaker. On-device LLM runs locally: nothing uploaded, nothing stored, nothing billed per token. Private, offline, zero latency to a server.
Chat for free. Go Pro for GPU and streaming.
The free tier gives you full LLM inference on CPU. Pro adds Vulkan GPU acceleration (significantly faster on modern SoCs), streaming tokens via Kotlin Flow, and concurrent multi-session support.
- llama.cpp prebuilt · CPU ARM NEON optimized
- Chat completion (prompt → response)
- Embeddings (text → vector)
- System prompts + chat templates (auto-detect)
- All GGUF models (Llama, Qwen, Phi, Gemma...)
- arm64-v8a · single session
- Everything in Free
- Vulkan GPU acceleration — faster on Snapdragon, Exynos, Tensor
- Streaming tokens via Kotlin Flow (word by word)
- JSON / Grammar mode — force structured output (GBNF)
- Vision / Multimodal — image + text input (LLaVA, Llama Vision)
- Context save/restore — resume conversations after app kill
- LoRA adapters — specialize models without retraining
- Function calling / tool use — structured agent actions
- Concurrent multi-session (shared model, isolated KV caches)
- arm64-v8a + x86_64 (emulators, Chromebooks)
Downloaded separately — pick what fits your app.
Models are not bundled in the AAR (too large, 400 MB to 8 GB). Download a GGUF model from HuggingFace and ship it with your app or download it at first launch.
No NDK. No Python. No build scripts. Just add the dependency.
The prebuilt AAR includes the compiled libllama.so and the Kotlin API. Add one line to your build file, download a model, chat.
Force JSON output. Every time. Guaranteed.
Cloud APIs sometimes return malformed JSON. On-device LLMs are worse — unless you constrain them. Grammar mode forces the model to generate structurally valid output, defined by a GBNF grammar. JSON, XML, custom formats — if you can define it, the model will follow it.
Send an image. Get an answer.
With multimodal GGUF models (LLaVA, Llama 3.2 Vision), the Pro tier accepts image + text input. Describe photos, read receipts, analyze screenshots — all on-device, no cloud upload.
Android killed your app. The conversation survives.
Android aggressively kills background apps to free memory. Without context persistence, every return to the app restarts the conversation from zero. Pro saves the KV cache to disk — the conversation resumes instantly, exactly where it left off.
Tokens arrive word by word. Just like ChatGPT.
The free tier returns the complete response at once. Pro streams tokens via Kotlin Flow — your UI updates in real time as the model generates, exactly like a cloud chat UI but running locally.
Voice assistant. Entirely on device.
Pair llama.cpp with Whisper for a complete voice AI pipeline. Whisper transcribes speech, llama.cpp generates the response, Android TTS reads it aloud. No cloud, no API, no subscription — a full assistant running on the phone.