Agent UI control

While waid dev is running you can let a coding agent look at and operate your app — read the interface, click things, type into fields, take screenshots. It is a development-session capability and it never ships.

bash

waid ui my-app snapshot
waid ui my-app click '{"nodeId":"42"}' --epoch 7

Turn it on#

Add this to waid.config.json:

json

{
  "development": {
    "agentUi": { "enabled": true }
  }
}

It defaults to false, it is effective only in the development profile, and you must restart waid dev after changing it.

This is a WAID development setting — not an Apple entitlement, and not an MCP consent scope. Granting ui.operate somewhere else does not switch it on.

Snapshot first, then act#

Start with a snapshot. It returns the node ids you address and an epoch:

bash

waid ui my-app snapshot

Pass that epoch with every action:

bash

waid ui my-app click '{"nodeId":"42"}' --epoch 7
waid ui my-app type_text '{"nodeId":"51","text":"hello"}' --epoch 7
waid ui my-app screenshot '{}' --epoch 7

If the interface has changed since you looked, the runtime rejects the action with stale_snapshot — "take a fresh UI snapshot before acting" — rather than clicking whatever happens to be at that node id now. Snapshot again and retry with the new epoch.

Actions#

Action	What it does
`snapshot`	Read the interface. Returns node ids and the current epoch.
`read_node`	Read one node in detail.
`focus`	Move focus to a node.
`click`	Click a node.
`type_text`	Type into a node.
`set_value`	Set a node's value directly.
`press_key`	Send a key press.
`scroll_into_view`	Scroll a node into view.
`screenshot`	Capture the current frame.

Desktop and iOS#

The command drives a running desktop development window, or an iOS development app connected to waid dev --target ios:

bash

waid ui --ios my-app snapshot

--ios is only needed when a desktop and an iOS session share the same app id.

For a physical iPhone, waid dev --target ios --dev-host <lan-ip> is still required. The phone connects out to a temporary authenticated broker on your Mac — it never listens for incoming control connections, the broker accepts requests only from the Mac itself, and both the broker and its discovery file disappear when the development session ends.

It cannot ship#

waid build refuses to package any of this:

a ui.operate grant in waid.config.json, in source or staged waid-package*.json, or in generated native package metadata
a staged development agent UI manifest

Native artifacts are written with agentUi.enabled: false. A ui.operate grant on its own cannot enable the runtime UI tools without the development setting — so a build either comes out with agent UI off, or it fails and tells you which file declared it.

Generating MCP tools Develop