Agents

Agent guide (skill)

looktap drives iPhone and Android devices: simulators, emulators and real phones. Every command is one short shell call that prints a few lines of plain text. Nothing is installed on simulators, emulators or Android devices; a real iPhone gets a small test runner. The helper starts on first use.

Quick start

# check the toolchain and see what's booted
looktap doctor
# start the app (id from --app or looktap.config.json)
looktap launch com.example.myapp --ls
# act by label or by @ref from the last screen read, and read the new screen in the same call
looktap tap "Sign in" --ls
looktap fill "Email" sam@example.com --ls
looktap tap @8 --ls
# wait for what the next screen should show, instead of sleeping
looktap wait "Welcome"
# screenshot (only open it when you need to see layout or colours)
looktap shot home --scale 0.5

The loop

Look, act, read the result. ls prints one row per element:

@1   Button           16, 285 358x54  General
@2   Button           16, 339 358x54  Accessibility
@3   TextField        33, 773 324x38  Search  = Search

The columns are: ref, type, x,y, width×height (iOS points / Android dp), label, and = value for fields.

Add --ls to an action and it prints the screen once it has settled, so you don't need a separate ls. It works on tap, longpress, zoom, tapxy, fill, type, key, hidekb, swipe, back, open, launch, bio, rotate and scrollto. It waits for the screen to change (up to 1.5 s, 6 s for launch/open), then for it to stop changing.

--ls stops at a loading skeleton too. If it shows a heading but none of the content, run looktap wait "<text you expect>" rather than acting on the placeholder.

Commands

Look

looktap ls
# include elements without a label
looktap ls --all
# structured: {screen, elements: [{ref, label, type, x, y, w, h, value}]}
looktap ls --json
# until it's on screen (10 s by default; --wait 30 for longer), or until it's gone
looktap wait "Welcome"
looktap wait "Loading" --gone
# swipe until it's fully on screen; fails at the end of the list
looktap scrollto "Privacy & Security"
looktap scrollto "General" --up

Tap and press

# a label: exact first, else the one label that starts with / contains it (ambiguous is an error)
looktap tap "General"
looktap tap "Messages" --exact
# a row from the last ls on this device
looktap tap @7
# a point, in the units ls prints; for things ls doesn't list
looktap tapxy 205 874
# hold (default 800 ms): context menus, drag handles, edit modes
looktap longpress "Photo"
looktap longpress @4 1500
looktap longpress 120 470
# two-finger pinch around the centre or a point; --scale is how far the fingers travel (default 2.5)
looktap zoom in
looktap zoom out 200 400 --scale 4
# Android: system Back · iOS: the navigation bar's back button
looktap back

Type

# focus the field, clear it, type; empty text just clears
looktap fill "Email" sam@example.com
looktap fill @4 ""
# into the focused field
looktap type "@example.com"
# return escape delete tab space; Android also back home KEYCODE_*; iOS also selectall paste
looktap key return
looktap key selectall delete
# dismiss the keyboard; exits 2 if it's still up
looktap hidekb

Swipe and navigate

# scroll down: the finger moves up (optional 5th argument: duration in ms)
looktap swipe 200 700 200 250
# deep link
looktap open "myapp://orders/42"
# (re)start the app
looktap launch
looktap launch com.example.myapp
# Face ID / fingerprint
looktap bio match
looktap bio nomatch

Screenshots and video

# saves <out>/<name>.png and prints the path; --scale 0.5 is a quarter of the image tokens
looktap shot checkout
looktap shot checkout --scale 0.5
# Android emulators capture FLAG_SECURE screens by default; --plain shows them black, as on a device
looktap shot payment --plain
# a frame each time the screen changes, for 3 s; duplicates dropped (start it in the background)
looktap burst ./frames 3 &
# a video of everything between start and stop (Android: at most 3 min)
looktap record start checkout
looktap record stop

Logs and crashes

# the app's log since your last `logs` (first time: the last minute); at most 80 lines, newest kept
looktap logs
looktap logs --errors
looktap logs --grep checkout
looktap logs --since 5m --lines 200
# iOS: keep Apple-framework messages too
looktap logs --all
# crashes, ANRs and low-memory kills since your last `crashes` (first time: the last hour)
looktap crashes
looktap crashes com.example.myapp --since 1h
# which app is in front, and whether the given one is running
looktap app
looktap app com.example.myapp

App state

# a simulator .app or an .apk (keeps the app's data; --grant: every runtime permission); prints the app id
looktap install build/MyApp.app
looktap install app-debug.apk --grant
looktap uninstall com.example.myapp
# as if freshly installed: data and permissions cleared (iOS keeps the keychain)
looktap reset com.example.myapp
# camera microphone location location-always photos contacts calendar notifications motion all
# (Android also android.permission.*)
looktap permission grant location com.example.myapp
looktap permission revoke photos com.example.myapp
looktap permission reset camera com.example.myapp

Conditions

# simulated GPS (Android: emulators); `clear` on iOS
looktap location 51.5072 -0.1276
looktap location clear
# no argument prints the current value
looktap appearance dark
looktap appearance light
# xs s m l xl xxl xxxl ax1…ax5 (default l)
looktap textsize ax2
# 9:41, full battery and signal, for screenshots; `live` to undo
looktap statusbar clean
looktap statusbar live
# portrait landscape left right upsidedown; ls, taps and shots follow the turned screen
looktap rotate landscape --ls
# Android only: airplane mode
looktap network offline
looktap network online
# iOS only: a remote notification; plain text, '{json}', or a .json file
looktap push com.example.myapp "Your order has shipped"
looktap push com.example.myapp '{"aps":{"alert":"Hi"},"orderId":7}'

Batch

# several steps in one call; stops at the first failure and names it
looktap run 'tap General; tap About; wait "iOS Version"; ls'
# one step per line from stdin, # for comments
looktap run - < steps.txt

Devices and setup

looktap devices
looktap doctor
# start the helper ahead of time (Android: also adb reverse tcp:8081 for Metro); teardown undoes it
looktap setup
looktap teardown
# serve the same commands over MCP, for clients without a shell; --config prints a client config
looktap mcp --config
# print this guide, or install it to ~/.claude/skills/looktap/SKILL.md
looktap skill --install

Options and config

# with more than one booted device, name the platform (-p, --platform) and/or the device (-d, --device)
looktap -p android ls
looktap -p ios -d 42E41FA7-EA40-45C1-8C6E-13A70BAF1FD6 ls
# where screenshots and videos go (-o, --out; default ./shots)
looktap -o out/run1 shot home
# the app for launch / open / logs / crashes / reset …
looktap --app com.example.myapp launch
# how long tap / fill / back wait for their target (default 3 s)
looktap --wait 10 tap "Continue"
# every command and option: -h or --help · the version: -v or --version
looktap --help
looktap --version

Put the app id in looktap.config.json at the repo root, so you don't have to pass it every time. looktap uses the nearest one, walking up from the current folder:

{ "out": "shots", "wait": 3, "ios": { "app": "com.example.myapp" }, "android": { "app": "com.example.myapp" } }

Command-line options beat LOOKTAP_* environment variables, which beat the config file. Passing both -p and -d makes every command about 0.2 s faster.

Exit codes: 0 OK · 2 usage or not found (bad arguments, no such element, no device) · 1 anything else. On exit 2, read the message, run ls, adjust. Don't retry the same command blindly.

Targeting elements

@N refs come from the most recent ls/--ls on that device. Use them for unlabelled buttons and duplicate labels. A ref whose element moved or vanished is an error, never a wrong tap: run ls again.
Labels match exactly first, then a unique prefix or substring. An ambiguous label is an error that lists the matches: use more of the label, --exact, or a ref.
tapxy x y is the last resort, for things ls doesn't list.
After rotate, read the screen again. Old refs and coordinates are stale.

looktap launch --ls
looktap fill "Email" sam@example.com
looktap fill "Password" "$TEST_PASSWORD"
looktap tap "Sign in"
looktap wait "Home"
looktap shot home --scale 0.5

Only with a test account the user gave you. Never type real credentials (see Rules).

Example: reproduce a bug and collect the evidence

looktap open "myapp://cart" --ls
looktap tap "Pay" --ls
# right after the step that misbehaved: only what's new is shown
looktap logs --errors
# "warning: … is no longer running" after an action means it crashed or was killed
looktap crashes
looktap shot pay__error

On iOS the full crash report is written up to a minute later; run crashes again. On Android, LOW_MEMORY means the system killed the app, which isn't a bug in it.

Example: screenshots in dark mode with large text

looktap statusbar clean
looktap appearance dark
looktap textsize ax2
looktap launch --ls
looktap shot home__dark_ax2
# set back what you changed
looktap textsize l
looktap appearance light
looktap statusbar live

Example: start from a clean install

looktap install build/MyApp.app
looktap reset com.example.myapp
looktap permission grant location com.example.myapp
looktap launch --ls

Example: catch a toast or a spinner

looktap burst ./frames 5 &
looktap swipe 200 250 200 700
wait

Then look at the few frames that matter, not all of them.

Rules

Never type real credentials, OTPs or card numbers. A person signs in, then you take over.
Never burst or record a sign-in or passcode screen. The frames capture what's typed.
Against a live backend every tap is real. Use the smallest amounts, and undo what you create.
reset signs the user out of most apps: ask before resetting an app someone else signed in to.
Set back what you change. Appearance, text size, the status bar, the orientation and the network stay as you leave them, for the next person and for other apps.
App-state and condition commands need the app id. They never act on whatever happens to be on screen.
If tap warns that another element covers the point (a toast, a banner, a floating bar), the tap probably hit that: dismiss it, then tap again.
If ls says only the status bar is readable, a system prompt (passcode, permission, alert) is on top. shot to see it. Never enter a passcode unless the user gave it to you for this device.
If hidekb exits 2 (iOS search keyboards have no Done key), tap the screen's close or cancel control.
Log lines can contain tokens or personal data. Quote only the lines that matter.

Installation

If looktap isn't on the PATH, use npx @looktap/cli in every command (or npx -y @looktap/cli to skip the prompt). Installing a global package may need the user's approval:

npm install -g @looktap/cli
looktap doctor

Needs Node 18+. iOS: macOS with Xcode and a booted simulator. Android: adb, with an emulator or a device.

Generated from skills/looktap/SKILL.md. Edit it there.

Agent guide (skill)

#Quick start

#The loop

#Commands

#Look

#Tap and press

#Type

#Swipe and navigate

#Screenshots and video

#Logs and crashes

#App state

#Conditions

#Batch

#Devices and setup

#Options and config

#Targeting elements

#Example: sign in and check the home screen

#Example: reproduce a bug and collect the evidence

#Example: screenshots in dark mode with large text

#Example: start from a clean install

#Example: catch a toast or a spinner

#Rules

#Installation