I’ve been building Buddy, a free, open source (Apache 2.0) Android app. You type or say what you want in a floating bubble, and an AI agent reads the screen, taps and types through your apps to do it.
- Your choice of agent: Claude Code, Codex, OpenCode, Antigravity or Grok, running inside the app (Alpine Linux via proot) on your own plan. OpenCode’s free models need no account.
- No root, adb or PC: it reads the screen and taps through the accessibility service.
- Asks first: before anything is sent, posted, bought or deleted. It pauses when you touch the screen and never types passwords or payment details.
- Builds apps: with Linux tools on, it builds real APKs on the phone. This week I drew a maze on a paper towel, took a photo, and it made it a tilt game. The first build crashed on launch; it found the bug and fixed it.
To be upfront:
- What the agent reads on your screen goes to the AI service you pick.
- It’s early (2.2), tested most on a OnePlus 12 with Android 16.
- It’s not on Play (Google rejected the accessibility use), so it’s an APK from GitHub releases, built by GitHub Actions from the source.
Source and releases: github.com/ghorbelhamdi/buddy-android
Happy to answer questions here.
Done: 2.2.25 makes this easy. Ollama, LM Studio and llama.cpp are in the service list, plain http works on your own network and over Tailscale, and no key is needed. README has a section now: https://github.com/ghorbelhamdi/buddy-android#local-models
Honest results so far, on a PC with an RTX 2060 Super: Qwen3 8B and 4B handle simple one-step requests (battery, phone status), but multi-step tasks on the screen failed. The 4B tapped the wrong buttons in the calculator and then claimed it worked. So for real tasks you’ll want a bigger model; I’m trying gpt-oss 20B next.
It’s is expected, they’re small models. But something like Qwen 3.8 27B or Qwen 3.8 Flash Next should handle it.
How much VRAM do you have? If it’s a 12GB card and you also happens to have 64 GB of RAM, you should be able to run Qwen 3.8 Flash Next with Q3. It’s one of the most “hyped” local models right now (because it’s quite capable and has somewhat realistic HW requirements).
Update: you were right. Qwen3.8 27B (unsloth UD-IQ3_XXS) did the calculator task correctly (1234×5678 = 7,006,652), but took 18 minutes on my 8 GB card, almost all of it reading the prompt and screen on the CPU.
Qwen3.5 9B is the interesting one: one run typed the wrong numbers and claimed success, the next did it right in about 2 minutes. So small models can do it, just not reliably yet.
Yeah, 27B only usable if you can fit it entirely in VRAM. Only MoE models are fine to split, like the mentioned Qwen 3.8 Flash Next. Even though it’s large, only the active 6B parameters needs to be located in VRAM, the rest can be in RAM and SSD.