PhoneUse: An AI Agent That Uses Your Real iPhone

Me too!

Free PDF · 7 pages

The guide as a PDF, for you and for your agent.

Read it at your pace, print it or hand it to your AI agent: the prompt to paste is already inside.

Download the PDFCopy the link for your agent

So far AI agents have used the computer: the browser, the terminal, a few desktop apps. The phone stayed out, because an iPhone doesn't let anything drive it from outside. PhoneUse, an open source project released on 7 November 2026, gets around that with two features every iPhone already has: Screen Mirroring and the accessibility pointer. No jailbreak, no app on the phone, no cable.

Before you start: 5 words to know

• AI agent: an assistant like Claude Code that doesn't just answer, it acts: opens programs, types, clicks.
• AirPlay (Screen Mirroring): the iPhone feature that sends the screen to a TV. Here it sends it to the computer.
• Bluetooth HID: how wireless mice and keyboards talk to a device. The computer pretends to be a mouse and a keyboard.
• AssistiveTouch: the accessibility option that shows a pointer on the iPhone when you connect a mouse.
• MCP and skills: two ways to give an agent new tools. PhoneUse offers both for Claude Code.

What It Is

PhoneUse is a Linux program by Aleksey Smolenchuk (lxe on GitHub), MIT licensed. The phone sends its screen to the computer over AirPlay, as it would to an Apple TV. The computer sends taps and keystrokes to the phone as a Bluetooth mouse and keyboard.

A window with the phone inside appears on the computer. You can use it yourself with the mouse, or hand it to an agent that gives commands like "open Settings" or "tap Wi-Fi".

Why It Matters

Plenty of things only happen on the phone: apps with no web version, banking and courier apps, notifications, device settings. Until now, getting an agent to do them meant emulators, developer accounts or modified phones. PhoneUse works on a normal iPhone, straight out of the box.

The clever part: the agent doesn't need a screenshot at every step. The program reads the text on screen (OCR) and hands it over as a list of lines with positions. According to the project's docs, reading the screen as text costs a few hundred tokens, a screenshot about 1,300.

How It Works, in Three Parts

1. The screen comes in. From the iPhone's Control Center pick Screen Mirroring, then "PhoneUse". The computer receives the video and shows it in a window and in a browser page (local to your computer only).
2. Taps go out. The computer shows up on the iPhone as a Bluetooth mouse and keyboard. With AssistiveTouch on, a click becomes a tap and a drag becomes a swipe. If the computer's Bluetooth isn't enough, there's a small bridge on an ESP32 board you leave next to the phone.
3. The agent drives. Each action is a command: `phoneuse open Settings`, `phoneuse text` (reads the screen), `phoneuse tap-on "Wi-Fi"`, `phoneuse done` (hands the phone back). The same commands exist as an MCP server for Claude Code, plus a ready-made skill.

While the agent works, the browser page glows and shows where it's pointing. "Take over" stops it so you can step in.

How to Try It

You need to be comfortable with Linux. The author tested it on a single setup: iPhone 17 Pro on iOS 26, Arch Linux, Hyprland (a window manager) and an NVIDIA GPU. Other systems will probably need fixes.

1. Check what you need: a Linux computer with Bluetooth, on the same Wi-Fi as the phone, and the packages listed in the README (UxPlay, GStreamer, GTK 4, Python 3.11).
2. Install: clone the project from GitHub and run `phoneuse install`. It asks for your admin password.
3. Start it: `phoneuse up`. On the iPhone open Control Center, Screen Mirroring, "PhoneUse".
4. Connect mouse and keyboard: on the iPhone, Settings, Bluetooth, tap "PhoneUse".
5. Turn on the pointer: `phoneuse cursor on` asks Siri for you, or go to Settings, Accessibility, Touch, AssistiveTouch. Keep Dwell Control off, or the iPhone taps wherever the pointer stops.
6. Connect Claude Code: add the MCP server with the README command, or link the skill into `~/.claude/skills/phoneuse`.

Ready to copy, in Claude Code inside the PhoneUse folder:

Small Test

Ten minutes, with the phone connected. Ask the agent: "Open Settings, read me the name of the Wi-Fi network I'm on, then hand the phone back." It worked if it reads the right name without touching anything else and ends with `phoneuse done`. Watch it all at http://localhost:8777.

Watch Out For

• It's a technology demo. The author says so: under construction and tested on one setup. Don't put it in production.
• The agent uses your real phone, with your apps and accounts. Start with a spare phone, or at least keep banking and email out of reach.
• The passcode: PhoneUse can save it and types it only after you approve a prompt. Think before saving it.
• Your computer's Bluetooth changes: the install turns off the module that accepts Bluetooth mice and keyboards for the computer itself. The uninstall script undoes it.
• Linux only, for now. No Mac or Windows, and on Linux it needs Hyprland.

If you want to see how agents work before giving one your phone, start with AI agents in parallel and how to make Claude Code use less.

Sources: lxe's launch post on X (7 November 2026) and the github.com/lxe/phoneuse repository (README, ARCHITECTURE.md, skill/SKILL.md). We have no ties to the project.

In short: PhoneUse shows the iPhone screen to your computer over AirPlay and taps it as a Bluetooth mouse: an AI agent can use real apps, no jailbreak. It's a demo: try it on a spare phone.