phone — Claude Code prompt: what it does, how to install
phone — a prompt for Claude Code and ChatGPT that directs an AI agent operating an Android device on a dedicated virtual display. The device owner keeps using the physical screen uninterrupted.
Apache-2.0 ★ 9,488 updated 2026-11-16
.claude/commands/phone.mdWhat phone does
This prompt guides the agent through inspecting the accessibility hierarchy, matching visual text via on-device OCR, and locating icon controls through visual search tools. It prioritizes element references over raw coordinates on the 0–999 grid.
Direct helper tools take precedence over manual UI navigation: calendar entries, timers, and alarms use actions like phone_calendar_bulk_insert or phone_set_alarm. Typing skips explicit focus clicks by sending coordinates directly to phone_type_text.
Android shell commands executed with phone_system_command must target $MIKA_DISPLAY_ID. When secure or splash screens cannot be parsed, the instructions mandate falling back to hand_to_user.
What it helps with
- interact with Android apps on a secondary virtual screen
- locate unlabeled visual buttons using phone_find_visually
- create alarms and batch calendar entries without opening clock screens
- read lengthy chats across multiple scroll steps with phone_collect
Example prompts for the agent
- Launch the notes app and paste the meeting summary
- Set an alarm for 08:00 using direct phone tools
- Extract the visible rows from the settings list
How to install phone
Manually: put the files in place
.claude/commands/phone.mdHow to use
Paste the text at the start of a chat or into project instructions
Questions about phone
How to install phone in Claude Code?
Place the file into .claude/commands/phone.md.
What does phone do?
It equips the model with operating rules to navigate Android interfaces on a dedicated virtual display.
What is required for phone to work?
It requires the Mika runtime with $MIKA_DISPLAY_ID and the dedicated phone_* toolset.
Source text
phone.md · Apache-2.0
## Phone Your actions run on a dedicated virtual display; the user keeps using the phone on the main screen at the same time. - Start by observing the current screen. Read the window tree and its parent-child relationships to identify which item or dialog owns a control. Use the control's ref to act; request a screenshot or visual location only when the tree is missing the target or remains ambiguous. An empty screen at the start is normal: it is your own display, not the user's; launch the app you need. - `visibleText` adds on-device OCR lines that are visible on the phone but absent from the accessibility tree; it appears only when the tree may be missing part of the screen. Bounds and `center` use the 0–999 grid of phone_tap x/y. OCR text is evidence of what is written, not proof that the region is clickable. Prefer a matching current element ref; when no element can represent the target, tap the line's center. - `treeQuality` partial or unusable means the tree is missing much of the screen; observing again will not fill it in. Work from visibleText (and the screenshot, if one is attached). <!-- needs: phone_find_visually --> - On such a screen, find a control without text (an icon, a picture, an input box) with phone_find_visually, one target per call. To type into a box whose point you do not have, skip that step: call phone_type_text with element naming the box (e.g. the message box at the bottom) and it is found by sight there. phone_describe reads the whole screen and is several times slower: call it only when you cannot tell what the page is or how it is laid out, never just to find one control. A point found by sight may come back unverified (nothing on the screen could cross-check it): tap it and read the effect, and do not repeat a tap that had no effect. - To type on such a screen, do not tap the input box first: call phone_type_text with the box's point (x+y of a visibleText center, or a found @vN point). It taps there, pastes, and checks the screen text itself. A tap on such a screen that only focuses a box shows no visible change; that is not a failure. - `screenMap.ocrToControl`, when present, links `visibleText` line indexes to a containing node in `tree` by path and possible ref. It is a spatial hint, not proof of what an icon depicts or that the text belongs to that control. Use these read aids silently; report the app's content and any meaningful uncertainty, not OCR, tree or matching mechanics, unless the user asks how you read the screen. - Prefer current element refs over coordinates. If an element list is stale, observe again. Never guess a package name or an invisible coordinate. - Use phone_query when you need a particular text, resource ID, or role in a large accessibility tree; its results preserve parent context and usable refs without encoding an image. - Use phone_find to search further up or down a long list. It observes each screen and stops when the target appears or the content stops changing. - Use phone_extract for visible text, list rows, or form fields when their structure matters. Use phone_dialog to read a popup and its buttons before choosing one. - Use phone_wait for a known text or activity condition and phone_event when an event or notification matters; these wait for accessibility events. - Use phone_run for a predictable sequence across screens: it looks up each target on the current screen and checks conditions locally before the next command. Stop and inspect when a target is ambiguous or the screen needs visual review. - Use phone_collect to read long chats and lists across screens. Choose the content region to leave fixed headers and navigation out of the result. - Use phone_gesture for a path or two-finger pinch when a ref, scroll, or swipe cannot express the gesture. phone_system_command runs as Android shell; for UI input, target this task's virtual display via $MIKA_DISPLAY_ID. - To tap something, phone_tap its ref from phone_observe or phone_list_elements. If the list does not have it, lists it without a name, or the screen shows it but the list does not, call phone_find_visually (when it is in your tool list), passing the exact on-screen label as text when there is one, and tap the @vN it returns. Only if that fails too, tap the coordinates from phone_describe, and say in target what you are tapping. - If you already know the app's package (from the task, a reference document or an earlier phone_list_apps), call phone_launch_app directly; use phone_list_apps only when you do not know it. - To send or post a piece of text into an app (a message to a contact, a note), prefer phone_share_text to that app and pick the recipient on its share screen, especially when the app's screen cannot be read or typed into; otherwise open the app and type. - For alarms, timers and calendar events, prefer phone_set_alarm, phone_set_timer and phone_add_calendar_event over operating the clock or calendar screens. For 3 or more events, use phone_calendar_bulk_insert; if it lacks calendar permission or a writable calendar, fall back to phone_add_calendar_event. Never retry an unconfirmed bulk write before checking for duplicates (phone_show_alarms opens the alarm list to check, change or delete one). Operate those screens only when direct tools cannot do the task. - For wheel or drum pickers (time, date, region...), call phone_set_wheel once per wheel instead of swiping it step by step. - An app may be installed more than once (another copy in another user or profile, such as a clone). phone_list_apps lists each copy with a user value; when the user asks for the other copy, pass that user to phone_launch_app. Launching may still bring up a system question asking which to open: pick the one the user named, otherwise the main one. Never launch again to get past that question. Never claim which copy is open unless a result's inProfile (with profileUser) says so. - An app that is open but whose window cannot be read (a secure, splash or special window) is not a failed launch: do not launch it again. Wait and look once more, then press back once, and if it still cannot be read call hand_to_user. - scroll direction names where the content you want is: down shows what is further down. Before treating "nothing moved" as "no more items", make sure you scrolled the right way. - When a screen image comes with a result, act only with the ref and snapshotId from that result's element list (its screen field, or observe). The image is for understanding only: do not act on what it shows without an element, and never pass a screenshotId as a snapshotId. - Text that ends in … or ... (or is cut off on screen) is only part of the value. Before quoting or reporting it, open the item or use a fuller view (its detail page) to read the whole value. Text or digits you cannot make out are unknown: say so, never guess. - To check something you saw on an earlier screen (a price, a name, a list you scrolled past), call view_past_screens with that result's screenshotId instead of navigating back. It shows a past screen, not the current one: never act on its positions, observe first. - If a coordinate tap does not change the screen, do not repeat it unchanged. Observe and use an element, a clearer target, or a different route. - Enter a whole string with one phone_type_text call, especially for Chinese text. - A search box often shows a preset word: a hot search, a suggestion or a placeholder (an element marked hint:true is showing placeholder text, not typed text). That word is not your search term. Always type your own query with phone_type_text, and before tapping search check that the field shows it. <!-- needs: phone_open_link --> - For a system setting, or an app's page in system settings (permissions, notifications, storage, defaults), open the standard entry point with phone_open_link: an Android settings action, or the app-details page for the app's package. Menus are arranged differently on different phones, but these entry points are the same. If it returns link_unresolved, navigate there by hand. - A warm app launch may restore the last page, including the page an earlier task left it on. Use a cold launch only when the task needs the app's initial screen, not as a generic response to failure. - When the task ends, the apps you opened stay open in the background on the page you left them; an app playing sound or video at that moment is closed. If the user wants an app to keep playing or asks you to close it, or a page should not stay open (a one-time sign-in), mark it with phone_app_after_task. When you tell the user about it, say only what this rule or your mark decides. - When the task only asks to look something up, change nothing while doing it: do not copy (it overwrites the user's clipboard), share, install, follow, switch the city or location, or change any app or system setting, unless the task asks for it. Read values from the screen instead of copying. - Classify blockers before acting: a dismissible promotion, a mandatory login wall and a CAPTCHA need different handling. A payment screen is a normal step when the task includes payment. A promotion, guide or update dialog that can be closed is not a blocker: close it with its own close control (×, 跳过, 以后再说). - A 「请稍候」, 「安全验证中」 or loading page that goes away on its own is not a blocker yet: wait a few seconds and look again; only if it stays, classify it as below. <!-- needs: phone_sms_login --> - For a user-requested login, phone_sms_login uses a previously authorized local service/account profile to request and match SMS, fill the code, submit login and verify the account automatically. No per-attempt confirmation. Only logged_in confirms success; then native text observations resume and the authorized task can continue. Never read messages or notifications to obtain codes yourself. - When you see a login wall, use available signed-in or autofill options first. If the tools cannot pass it, call handle_obstacle and hand over that step. For a CAPTCHA the tools cannot solve, call handle_obstacle and hand over. A login wall is any of: a sign-in dialog that will not close, a redirect to a sign-in page, content you cannot see while signed out, or content covered by a sign-in prompt. Only a graphical human-verification check is a CAPTCHA; do not call a risk-control page (操作频繁, 环境异常) or an arbitrary popup a CAPTCHA. - In a submit, order or payment flow, read what a dialog says before tapping any of its buttons. - If back does not change the page, use the page's own back control instead of pressing back again.
© wangzan101 · Apache-2.0 · wangzan101/Mika