Start
How it works
looktap ships its own small helper for each platform. Nothing is installed on simulators, emulators or Android devices. A real iPhone or iPad is the exception: it gets a small test runner (Using a real iPhone).
| Item | iOS simulator | Android emulator / device |
|---|---|---|
| Read the screen | looktap-sim, a macOS helper that reads the simulator's accessibility tree through Apple's simulator frameworks (CoreSimulator, AccessibilityPlatformTranslation). Web content is found by probing the screen | looktap-server.jar, an 8 KB helper pushed to /data/local/tmp and run with app_process. It keeps one UiAutomation connection open |
| Tap, type, swipe, keys | The same helper, as touch and keyboard events through SimulatorKit | The same helper, as input events through UiAutomation |
Screenshots, burst | The simulator display's framebuffer, read in-process | Emulators: the emulator's gRPC API, which also gets past FLAG_SECURE. Devices: the helper |
Video (record) | simctl io recordVideo | screenrecord on the device |
Orientation (rotate) | The orientation event Simulator.app sends to the guest. Coordinates and frames are converted, so everything looktap prints and takes is as the user sees the screen | settings (user rotation, auto-rotate off). The helper reports the turned display size |
| Logs, crashes | log show inside the simulator, and its .ips crash reports | logcat (by app uid), the crash buffer, and dumpsys activity exit-info |
| Deep links, launch, biometrics | xcrun simctl | adb and the emulator console |
Each helper starts on first use, keeps running in the background, and is replaced automatically when looktap is
upgraded. looktap setup starts it ahead of time, and looktap teardown stops it. The CLI talks to it over a
local socket, using JSON lines.
iOS: private frameworks and Xcode versions
The iOS helper uses Apple's private simulator frameworks, the same ones idb uses. They can change between Xcode
releases. looktap doctor checks they're present and shows which input path applies:
- Xcode 26 and earlier (CoreSimulator below 1155.4): touches and keys go through SimulatorKit's HID client ("indigo"). Tested.
- Xcode 27 (CoreSimulator 1155.4 and later): touches and keys go as XPC messages to the simulator's
dtuhiddservice ("dtuhid"), the same way idb does it. Implemented, not yet tested against Xcode 27. Reading the screen and screenshots don't depend on it.
LOOKTAP_IOS_HID=indigo|dtuhid overrides the choice. It applies when the helper starts, so run
looktap teardown first.
Using a real iPhone
A simulator can be driven from the Mac with nothing installed in it; a real iPhone or iPad can't. So the first
command aimed at one builds a small XCUITest runner (native/ios-device), signs it with your Apple ID, and installs
it. It reads the screen and taps, types and swipes through XCUITest, and looktap reaches it through usbmuxd, over
USB or Wi-Fi. Every looktap user signs the runner with their own Apple ID. A free one is enough; the paid
Developer Program isn't needed.
Once per Mac and device:
- In Xcode → Settings → Accounts, sign in with your Apple ID. That gives you a "Personal Team".
- Pair the device with the Mac (connect it with a cable once and tap Trust; Wi-Fi works after that), and turn on Developer Mode on it: Settings → Privacy & Security → Developer Mode.
- Unlock the device and run any command against it, for example
looktap -d <hardware UDID> ls(looktap deviceslists the UDID). The first run builds the runner, which takes about a minute. - With a free Apple ID, the device asks you to trust the developer the first time: Settings → General → VPN & Device Management → your Apple ID → Trust. Then run the command again.
If Xcode is signed in to more than one team, choose the one to sign with:
LOOKTAP_TEAM=<team id>, or {"ios": {"team": "<team id>"}} in looktap.config.json. The error message
lists your teams and their ids.
Picking the device: -d <hardware UDID> always picks it. Without -d, a paired device is used only when no
simulator or emulator is running, so nothing changes for simulator workflows.
Signing expiry: a runner signed with a free Apple ID stops launching after 7 days, and one signed with a paid team after a year. looktap reads the expiry date and builds and signs the runner again before it lapses. If iOS refuses to launch it anyway (for example, after a certificate was revoked), looktap rebuilds it once and retries. Free Apple IDs have Apple's own limits too: about 3 devices and 10 app ids per 7 days, and at most 3 apps signed by the free account installed on one device at a time. The runner counts as one, so if the device is full, looktap names the apps it found and you delete one.
What works on a device: ls, tap, longpress, type, fill, key, hidekb, swipe, scrollto,
zoom (around the screen's centre), back, wait, shot, burst (screenshots in a loop), launch, open,
rotate, appearance, install (a device .app or .ipa), uninstall and app. location, push,
statusbar, textsize, bio, record, permission, reset, logs and crashes aren't available, and say so.
A command takes about 1–1.5 s, slower than on a simulator, because XCUITest takes longer to read the screen.
The device must be unlocked while the runner starts. It keeps running until looktap teardown, or until the
device is restarted.
Android: the fallback path
If the helper can't start (for example, another tool holds UiAutomation), looktap falls back to plain adb +
uiautomator dump. That works everywhere but takes about 2.5 s per screen read. LOOKTAP_ANDROID_HELPER=0
forces that path.
Generated from README.md. Edit it there.