microsoft/amplifier-bundle-android-tester

microsoft/amplifier-bundle-android-tester

Description: Android tester bundle for the Amplifier project

Language: Python

License: MIT

Stars: 0

Forks: 0

Open issues: 1

Created: 2026-08-06T15:41:50Z

Pushed: 2026-08-07T05:36:25Z

Default branch: main

Fork: no

Archived: no

README:

amplifier-bundle-android-tester

Let an agent actually see and drive the Android screen — so UI fixes stop shipping "verified" and arriving broken.

Unit tests pass. Server-side curl passes. The user opens the app and every one of the last four fixes is broken. The failure is always at the render/interaction layer, and nothing in the loop has ever looked at the screen. This bundle closes that gap.

The Load-Bearing Rule

uiautomator is the sensor. The screenshot is for judgment, never for targeting.

Measured live on an aarch64 host — same screenshot, same "Refresh" button:

| Source | Bounds | Center | |---|---|---| | uiautomator dump | [877,142][1006,195] | (941, 168) | | VLM reading the PNG | [810,50][950,100] | (880, 75) |

Delta dx −61px, dy −93px — the VLM-derived center lands outside the real button.

And the miss is silent. Android has no concept of "you tapped nothing": no error, no exception, and often a screenshot that still looks right. The dangerous case is landing on a *neighbouring* element — in a recorded run, a tap meant for the Base-URL field hit the API-key field, and the run typed a server URL into the API key and reported success.

So: vision answers *"does this look right / what state am I in / is anything clipped"*. Coordinates always come from `ui_dump`.

How It Works

A single android_inspector tool wraps adb and uiautomator with a selector-first contract — the safe path is the default path:

• tap cannot fire without resolving a selector against a fresh accessibility dump
• type_text asserts focused="true" before typing and asserts readback after
• wait_for polls the tree; there are no bare sleeps anywhere
• tap_xy (raw coordinates) is named to be conspicuous and warns in its own result

These four rules were each learned by breaking something real. An agent *told* to follow them skips them at turn 40 of a long run. A tool that structurally *cannot* skip them does not.

Quick Start

Installation

Add as an app bundle (recommended):

amplifier bundle add git+https://github.com/microsoft/amplifier-bundle-android-tester@main#subdirectory=behaviors/android-tester.yaml --app

Compose into another bundle:

includes:
- bundle: git+https://github.com/microsoft/amplifier-bundle-android-tester@main#subdirectory=behaviors/android-tester.yaml
as: android-tester

Prerequisites

Start by asking the bundle. doctor takes no parameters, never errors, and tells you everything that is wrong in one call:

report = android_inspector(operation="doctor")
# report["ready"] — false if any check failed
# report["checks"] — [{name, status: ok|warn|fail, detail, remediation}, ...]
# report["summary"] — what to fix first

Ten checks — host arch/OS, ANDROID_HOME, adb binary, adb server and attached device states, emulator binary, KVM, ptrace_scope, gdb, AVDs available, cmdline-tools. It deliberately does not stop at the first failure: you get the whole picture and fix the host in one pass, instead of discovering its problems one 60-second timeout at a time. Measured 0.26s on a healthy host. A broken machine is a *successful diagnosis*, not a tool error — read ready, not success.

No AVD? create_avd provisions one. ABI is auto-detected from the host arch, it will not clobber an existing AVD without force, it will not accept SDK licences on your behalf without accept_licenses, and it verifies with emulator -list-avds afterwards rather than trusting an exit code:

android_inspector(operation="create_avd", name="my-harness") # 1.44s with the image already local

What the bundle will not do for you, and why:

| Not automated | Why | |---|---| | Installing the Android SDK | Out of scope | | Downloading the community linux-aarch64 emulator build | It is an unsigned third-party binary. Whether it goes on a machine is a human's trust decision, not a tool's. doctor detects the gap and points you at [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md), which carries the URL and sha256 |

So the underlying requirements remain: an Android SDK at ANDROID_HOME (default ~/android-sdk), a working adb, an emulator binary, and /dev/kvm readable and writable.

On aarch64 Linux hosts, two things Google does not ship — doctor reports both by name:

| Need | Problem | Fix | |---|---|---| | adb | platform-tools/adb is x86_64-only → Exec format error | Native-arch adb (e.g. extracted from Ubuntu's arm64 debs) at platform-tools-arm64/ | | emulator | No linux-aarch64 emulator exists in Google's SDK repo | A community linux-aarch64 build + hand-written package.xml |

Full detail, including the libpcre2 boot crash and the ptrace_scope workaround, in [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md).

Basic Usage

Delegate to an agent — do not drive adb from the root session.

# Boot (applies host workarounds + the two-stage readiness gate)
r = android_inspector(operation="start_emulator", avd="my-harness", port=5556)
serial = r["serial"]

android_inspector(operation="install", serial=serial, apk_path="/tmp/app.apk")
android_inspector(operation="launch", serial=serial, package="com.example.app")

# Gate on state, never on time
android_inspector(operation="wait_for", serial=serial,
selector={"res_id": "com.example:id/root"}, timeout_s=30)

# Coordinates come from the tree — this resolves, taps, re-dumps, reports what changed
android_inspector(operation="tap", serial=serial, selector={"text": "Settings"})

# The verified field write: focus assertion + readback
r = android_inspector(operation="type_text", serial=serial,
selector={"res_id": "com.example:id/url"},
text="http://localhost:9000")
assert r["verified"], f"field write failed, readback: {r['readback']!r}"

# Screenshot for judgment — never for coordinates
snap = android_inspector(operation="screenshot", serial=serial)

android_inspector(operation="stop_emulator", serial=serial)

Agents

android-operator (primary) — [coding, general]

The driver: boot → install → launch → interact → verify → report. Owns the emulator lifecycle, the verified...