The linter for AI
Know your AI product works for real humans.
A population of synthetic humans sees and operates your real GUI inside a sandbox, then returns an evidence-backed verdict.
Black-box · no source access · advisory, never gating
- Chat sliceEndpoint targets, rung 1--endpoint
- Vision GUI loopChromium + Claude vision--ui <url>
- E2B deny-all isolationProfile-frozen egress floor--e2b
- Composed microVMThe target served inside--e2b <template>
Your evals test prompts. Your users test the product.
Unit tests and prompt evals never touch the thing a person actually reaches: a real screen, clicked by real hands, read with real eyes. That gap is where AI products quietly break, and it is the one test you cannot write by hand.
same input, two verdicts
Passes on every input you remembered to write.
beyond the score
Whatever breaks under a persona's hands is recorded with the evidence that produced it — never folded into a single number.
The operator loop
One loop, run by a whole population.
Every persona runs the same five moves, then hands the next one a fresh browser. The loop is the unit of work — coverage is how many times it closes.
- 01boot
Discover the UI
A fresh browser instance boots and observes the entry screen with its own eyes.
- 02compile
Compile a persona population
Synthetic humans are grounded across locale, device, fluency and assistive-tech axes.
- 03operate
Operate the GUI
Each persona sees screenshots as image blocks and acts through semantic-first UI actions.
- 04record
Capture evidence
Every observation, action and oracle check is recorded as a replayable trajectory.
- 05reduce
Return an advisory verdict
A deterministic reducer folds the run into one evidence-backed, non-gating verdict.
05 → 01 · the next persona restarts the loop on a clean sandbox.
The report is the product
Every verdict is backed by what the synthetic humans saw and did.
No score without the trajectory behind it. The canonical report carries the screenshots, the semantic actions, and each oracle rung that produced the verdict. The schematic below shows the shape of that report — illustrative, not run output.
- target
- gui/<app>@sha256:<digest>
- mode
- --e2b
Oracle ladder
gating → advisory- 1entry_reachablereachability
- 2ui_element_presentui-state
- 3ui_outcome_postconditioncausal
- 4task_completioncoverage
- 5advisory_judgenon-gating
Evidence viewer
illustrative- 01observe<entry screen>
- 02click<semantic target>
- 03type<text> → <field>
- 04observe<resulting screen>
The four layers.
The chat slice, the vision GUI loop, the deny-all isolation floor and the composed verdict.
Persona-driven synthetic humans
A population, not a single script. Each persona carries its own locale, device, fluency and assistive-tech profile, so coverage reflects the humans who actually use your product.
Vision-grounded GUI loop
A chromium instance and Claude vision: the operator reads the screen from pixels and acts through semantic-first UI actions behind a safety gate.
E2B deny-by-default isolation
Every run lands in an E2B microVM with egress frozen from the sandbox profile. A 7-rung canary ladder probes the isolation from inside before any target is touched.
Non-gating, evidence-backed verdict
Mirage advises, it never blocks. The verdict ships beside the screenshots, actions and oracle rungs that produced it, projected into a canonical mirage-report.json.
Isolation floor
Nothing leaves the sandbox.
Every run lands in an isolated microVM with the network closed by default. The synthetic humans operate your product with no path to the outside, and a canary ladder probes the isolation from inside before anything else happens.
Deny-by-default egress
The network is closed before the target boots. Nothing the synthetic humans do can reach the outside.
Profile-frozen
The egress policy is frozen from the SandboxProfile, not configured at runtime.
Probed per run
A canary ladder probes the isolation from inside the E2B microVM before any target is exercised.
- 01sandbox_infosandbox identifies itself by image digest
- 02internal_net_bootproduct boots on the internal-only network
- 03no_public_portno port is bound on a public interface
- 04egress_deny_probeegress to a non-allowlisted host is denied
- 05app_healthapplication answers an app-level health probe
- 06browser_renderbrowser renders the first frame
- 07capturean observation channel produces a capture
Mirage advises. It never blocks your pipeline.
A verdict is a signal, not a gate. You keep the ship decision. Every report carries the evidence, so the call is yours to make with your eyes open.
a gate stands on the line and stops the merge when a check goes red.
mirage sits beside the run, emits a signal, and lets the flow keep moving.
- commit
- build
- test
- deploy
- ship
Reference
Questions engineers ask.
The short answers to what Mirage is, how a run is isolated, and why the verdict never touches your pipeline.
5· advisory by design
No. Every verdict is advisory and non-gating by design. Mirage tells you what synthetic humans saw and did on your product, then leaves the ship decision to you. It never blocks a pipeline.
No. Mirage is black-box. It operates your product through its real interface, the same way a human would. Point it at a chat endpoint, a live GUI, or a template served inside the sandbox.
Each run executes inside an E2B microVM with deny-by-default egress frozen from the sandbox profile. A 7-rung canary readiness ladder probes the isolation from inside the VM before any target is exercised.
A vision-capable Claude model both operates the GUI and judges the evidence. When no live model is configured, a run honestly returns INCONCLUSIVE. Mirage never fabricates a PASS.
Yes. Targets are pinned by sha256 digest, trajectories are recorded action-by-action, and any run can be replayed. The canonical mirage-report.json is the authority for every projection.
Lint your AI before it ships.
Request access and put a population of synthetic humans on your product, inside a sandbox, before your users find the edges.
the report a run hands back
- target
- gui/<app>@sha256:<digest>
- mode
- --e2b