Skip to main content
skill CX Codex

strobe — Claude Code skill: performance benchmarks

strobe — a skill for Claude Code that measures app performance and explains the results. It runs for version comparisons, slowdown investigations, and benchmark writing.

MIT ★ 173 updated 2026-11-16

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add anomalyco/strobe --skill strobe -a claude-code
Open on GitHub

What strobe does

strobe creates and runs benchmarks for web pages, Electron and native apps, terminal programs, and Bun or Node processes. It compares a baseline with a new version, reports a verdict for each measurement series, and lists suspected causes with their sources and follow-up commands.

To start, strobe init creates a config and a draft benchmark from project files. Run strobe bench <file> --head . --check to check it. A clean comparison needs at least eight pairs. When the measurements cannot support a conclusion, strobe returns a separate verdict instead of treating the run as proof that nothing changed.

What it helps with

  • compare performance between a baseline and a new version
  • trace a slowdown or increase in memory use
  • create a draft benchmark from project files
  • check that a benchmark runs and produces metrics

Example prompts for the agent

  • Compare the changes with origin/dev using perf/smoke.bench.ts.
  • Find out why this page got slower and where memory is going.
  • Create a benchmark for the app in this project.

How to install strobe

1Skills CLI (Vercel)

npx skills add anomalyco/strobe --skill strobe -a claude-code

2Manually: put the files in place

~/.claude/skills/skill/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/skill/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about strobe

How do I install strobe in Claude Code?

Run npx skills add anomalyco/strobe --skill strobe -a claude-code. The skill is installed at ~/.claude/skills/skill/SKILL.md.

What does strobe do?

It measures software performance and compares runs, such as a baseline against a new version. It reports verdicts and helps investigate possible causes of slowdowns or increased memory use.

What kinds of apps can strobe measure?

The listed targets include web pages, Electron and native apps, terminal programs, Bun and Node processes, and other processes.

Source text

SKILL.md · MIT
---
name: strobe
description: Measure and explain performance of web pages, Electron apps, native apps, terminal programs, Bun or Node processes and any process. Use when the user asks whether a change is faster, why something is slow, janky or heavy, where memory goes, what the CPU or GPU is doing, or to write a benchmark.
---

# Strobe

strobe records a target on one timeline and answers with verdicts and handles (`rec_…`, `exp_…`) to
drill into. It refuses measurements it cannot defend: a refused run is an answer, never "no value".
The design and every rule behind this file: SPEC.md.

## Playbooks

### No bench yet: start one

1. `bun add -d @hona/strobe` pins strobe for the project (`bunx strobe …` then runs that version).
2. `strobe init` writes `strobe.json` (its `$schema` linked) and a runnable `perf/smoke.bench.ts` from
   the folder's package.json (`build`, `preview`, `start`, `dev`, `exports`, `module`, `types`, `main`,
   `bin`, an electron dependency), Cargo.toml binaries or index.html.
   `--target web|lib|server|native|electron|tty|cli` overrides the guess: web serves the build with
   `perf/serve.ts` (a cold load, an idle second, and a commented reload: a small page's reload is
   mostly strobe's own driver work, `OverheadBudget`), lib imports the package's entry from
   `perf/lib.ts` (a cold import per run; a commented `work` case calls an export with `ctx.call`),
   server hosts it in `perf/server.ts` (`PORT`),
   native builds and launches the Cargo app, cli runs the command to its exit. It never replaces a file
   without `--force`.
3. Run what it prints: `strobe config`, then `strobe bench perf/smoke.bench.ts --head . --check`
   (`--headless` with no display or a locked session). Then edit the cases to what users do.

### Did my change make it faster?

- Allow at least eight pairs for the clean-pair check. `strobe run --plan` previews a command comparison.
- On a busy desk use `cpu.work` or, for headless pages, `main.work`. Read any `Background` warning.

1. Write or find a bench file (below; `strobe init` writes a first one). Check it first:
   `strobe bench <file> --head . --check` (each case once, pass or fail with a screenshot).
2. `strobe plan <file> --base origin/dev --head .` says pairs and minutes per series (`time unknown`,
   never a guess, for a series no `--check`, `--pilot` or earlier run measured here).
3. `strobe bench <file> --base origin/dev --head .`: a verdict per series (case × target × params).
   - Read the ★ primary's verdict: `faster`/`slower` (`less`/`more`, `fewer`), `same (±2%)`,
     `unclear`, `underpowered` (more pairs or a quieter machine, not "no change"), `refused`.
   - Secondaries get words only after the primary decided (Holm-corrected); `·` is descriptive.
   - `ledger  3rd experiment on these variants; earlier: …`: reruns count. Paste it as it is.
   - `strobe query <exp> insights`: per series the verdict (or a check's status), ranked suspects
     with their source, the warnings that bear on it, and commands to run next as printed (`--pairs
     N`, `record --bench <file>#<case>` for a side nothing profiled). Any kind: check, `--blocks`, leak
     run. Head's hits read `(over its threshold on head only)`, `(… on base too)` or `(base not read
     for it)`; beside `same`, `unclear` or an A/A no mover is a suspect (two launches differ anyway).
4. Put the evidence in the PR (below).

- Refactor or dependency bump? `--noworse 2%` (or `noworse: "2%"` on the case;
  `--noworse 2%,cpu.wakes=5%` per metric): `no worse (≤2%)`, `worse (>2%)` or `unclear`. A bare exact
  counter is `0%` (`--noworse count.items`); a bare other metric exits 2 (`main.work needs a margin`).
- `strobe run --base "<old command>" --head "<new command>"`: the same A/B for two command lines.
  `--check --head "<cmd>"` (and `--base "<cmd>"`) runs each once: `cpu.work · run.duration ·
  memory.peak`. With `--check` every `--noworse` exits 2 (a command gives no exact counter): drop
  `--check` and use `--noworse cpu.work=2%`.
- `ShellLine`: `&&`, `||`, pipes and redirects run through the platform shell. Its startup counts.
  Plain external commands use PATH; use `./tool` or `.\tool.exe` to measure each variant's build.
- `--aa --head .` checks the method on this machine (same build both sides).
- Long benches keep running in the daemon: `strobe query exp_… progress`. Ctrl-C, SIGTERM or
  `strobe bench --stop <exp>` stop after the block (exit 130, what finished is kept);
  `strobe bench --resume <exp>` goes on.

### How a run ends (the exit code says whether strobe measured, never who won)

- A timed comparison can exit 0 on `slower` or `worse`: read `series[].primary.verdict_kind`.
  Every non-gap primary refused: 4. `--require-verdict`: 5 if undecided, 4 if any primary is refused
  or any journey is a parity gap. `--require-parity` alone makes gaps exit 4, not undecided results.
- A check is no verdict: each case once (each side once with `--base`), proving the bench runs and each
  declared metric gives a value or a stated refusal.
  - `--check`: 0 when every case passes; 1 when one `FAIL`s (its `fix:` line says what to change); a
    preflight refusal is listed beside passing cases; all non-gap cases refused up front: 4.
    A run defect fails its case (1); a primary the target, engine or display cannot measure is
    listed as refused. A refused secondary alone does not fail a case.
  - `--check --base <ref>`: base's line under head's; the summary ends `base and head each checked;
    nothing compared: the same command without --check times head against base` (exit 0 if both ran).
    A base build error stops the check before the case runs (1). A base run error fails the case (1):
    `1 failed on base`; head's values if it passed, head's error if it failed too. Leave head-only cases
    out with `--filter` until the change merges.
  - `--check --base <ref> --noworse count.x`: each counter row ends `no worse` (passes), `worse`,
    `unstable`, `missing` or `not_compared` (each fails the case, exit 1). If cases ran and none failed
    but no counter compared, exit **4**, with `passed: false` in JSON. If both sides refused every
    counter, read the refused lines. For `Uncounted`, check the spelling against `ctx.count`; if the run
    counts it only on some paths, add `metrics: ["count.x"]` on that case so 0 → 0 compares.
    If every case is a parity gap, no counter compared: exit 4. `--require-parity` requires every journey.
  - `--check --base <target> --head <target>`: shared journeys get a parity matrix, not a counter
    comparison. Gaps alone exit 0 (even if no case ran), or 4 with `--require-parity`; a mismatch or
    case failure exits 1. With two bare targets, `--noworse` exits 2: it needs two builds.
- `--blocks N`: one build's values, no verdict. `--probes memory`: a leak run exits 0 even for `leaks`
  (`worse`); 5 if undecided with `--require-verdict`. Both modes exit 4 if every series is refused.
- `--pixels-only`: 1 if snapshots differ or one side lacks one, otherwise 0 on completion. `--bisect`:
  `found`, `no change`, `unclear` exit 0; `--require-verdict` makes an `unclear` search exit 5.
- Stops and caps: 130 stopped; 124 for an exhausted `--unlock-wait` or a non-comparison timeout.
  A timed comparison hitting `--max-time` keeps its finished pairs and normal exit rules (usually
  0, or 5 if undecided with `--require-verdict`). The slot wait for display fit before the first
  series can still exit 124. Resume stopped work with `bench --resume <exp>` if state was saved.

### Why is it slow?

- `bench … --why always` (an attribution block per series), `bench … --check --why always` (one build),
  or `strobe record <target> --probes attribution` / `strobe record --bench <file>#<case>`. Then:
  - `query <rec> hotspots` (CPU by function, full stacks, source-mapped file:line:column;
    `--by file|package`; `--sort total` for callers; native: `--thread <name>`, `--pid <pid|exe>`,
    `--phase <name>`).
  - `query <rec> layout` (forced layouts by their call site), `style` (what invalidated style:
    attribution blocks trace it for a series declaring `main.style` or `style.count`, or with
    `--style`), `alloc` (who allocates), `gc`, `summary` (per run: style, layout, paint ms), `insights`.
  - A page served from your own folder (`record https://localhost:… --served <dir>`): its source-mapped
    frames resolve to files on disk.
  - What changed: `query <exp> hotspots` (each series' base..head) or `query <recA>..<recB> hotspots`;
    `~` marks a change inside its sampling interval (one launch a side: not a verdict).
- `strobe open <rec>` (or `<exp>/head`): the whole recording in a local Perfetto UI. For seeing;
  numbers come from strobe's views. The "strobe itself" track is strobe's own cost: check it first.
- A thread waiting (a lock, IO, a timer) and who wakes it: `record pid:<n> --probes waits --for 10s`,
  then `query <rec> waits --thread <name>` (Linux needs CAP_PERFMON, Windows the deep helper;
  macOS refuses `NoWaits`).
- A slow page frame in the renderer or the GPU process: `frames.renderer.p95` against `frames.viz.p95`,
  and `query <rec> frames` (each worst frame's stages).
- Inputs with no latency sample (`busy`), or one that looks wrong: `query <rec> inputs [--case <c>]`
  (the first 100; `--top N`, `--json` every one):
  each input's run and time, what proved its change, the frames around it, its latency or why it
  was busy (`mid_drawing`, `behind`, `ambiguous_press`, `multi_frame_pixel`) and how much later it
  had to come. Give the case that wait before such inputs.
- A profiled launch is slower than a timed one: compare totals only from the timed pairs.

### Where is the memory? Does it leak?

- `query <rec> memory` (per process; `--by layer|allocator` for Chromium's accounting). A native
  app: what holds it (host heap, GPU memory mapped in, unmapped commit, shared, files) beside
  `gpu.vram`. On Windows `memory.app.private` counts VRAM a driver maps in: quote `memory.app.host`.
  A phone's page: processes from Chromium's dump (no OS memory); listeners are the page's own.
- `query <rec> heap`: classes by retained size, detached DOM, retainer paths; `query <a>..<b> heap`.
- A case's leak: `bench <file> --filter <case> --head . --probes memory --repeat 10`: growth per run
  with an interval, and `query <rec> leaks` for what from the first half is still alive.
- An app over time: `record <target> --probes memory --for 60s` while it does the suspect thing.
- A script target: `js.heap.retained` (what a run keeps after a collection, buffers counted once). On
  a Node host it adds ArrayBuffer contents in releases after 0.31.0 (which left them out): compare
  across that upgrade as two histories (`track` starts a new series, `metric revision 2`; Bun series
  keep theirs). Every Node-hosted script series, any metric, started a new history once in 0.32.0:
  its identity records `runtime node` (`strobe history` shows it). It does not see add-ons' native
  memory, mapped files, or on Node WebAssembly memory and strings outside the heap: read
  `memory.app.private` for those. On macOS a Bun host's
  `memory.app.private` is refused (`EngineUnsupported`): Bun's allocator hides memory from the OS.
- Sizes are binary: KB, MB and GB are 2^10, 2^20 and 2^30 bytes.

### A native app, or any process from outside

- `record app:<id> --for 10s` (a strobe.json profile) or `record exe:<path> --native`: CPU, thread
  wakes, IO, GPU, memory, the UI thread and what reached the screen. `--windowless` for a tray app
  (`look … --windowless` first: does it stay up, what does it print?).
  `record pid:<n>` or `proc:<name>` for a process strobe did not start.
- `query <rec> threads`: wakes/s and CPU per thread, the module each started in, whose (app, driver,
  os, `unknown (?)`); `--timeline` per second. `query <rec> insights` rules include dominant-thread,
  off-cpu-bound, style-layout-dominant, wakes-while-idle (idle blocks too), hot-function (a named
  function of the target's over 25% of busy samples, one hit each; a native symbol without line
  info counts), http-stalls (measured runs only).
- Native hotspots of a tree pool every process: read `by process`, then `--pid <pid|exe>`.
  A native attribution recording (`record --bench`, `--why`) keeps frames too: `query <rec> inputs`.
  `! Coverage`: samples cover under 80% of the CPU (read shares, not totals); it names what the
  sampler counted of the rest (`late_ms` before a thread was opened, `forgone_ms`, unopened threads,
  lost events). A launched app's children are sampled from their start. Two resolved shares
  (leaf, all frames): a low all-frame share means the app's own binary has no symbols.
  macOS samples are untimed snapshots: counts, never ms.
- `record pid:… --probes deep|waits` prints `cost`; `! CaptureCost`: the capture slowed a
  switch-heavy process, so its stacks describe that slower process.
- Benching it: `native("my-app")` in a bench file (below).

### Two implementations of one app (a port, a rewrite)

- One bench file, one target per implementation, the same cases (journeys):
  `bench <file> --base original --head rewrite` (`rewrite=git:main`, `original=exe:/path`;
  `target:<name>` when a folder has that name). Each target's `name` is what reports call it.
- Pick a primary both sides measure the same way: `cpu.work`, `run.duration`, `launch.ready`,
  `memory.app.private`. `main.work` or `ui.cpu` across engines exits 2 (`NotComparable`).
- Give each target a `ui` adapter for what differs (`ui: { row: ".row", open: ({ page }) => … }`);
  journeys say `@row`, `await ctx.ui.open()`. A journey one side cannot run is a parity gap
  (`unsupported: { rewrite: "why" }`, or an `@name` its `ui` lacks): listed, never run.
- `expect: { same: ["@topRow"] }`: both sides must read it alike after setup, each run and reset, or
  `NotEquivalent`. Both talk to one `fixture("<server> --port {port}")` (`{fixture.url}` in args/env).
- A page against a native app (or another engine) gets OS input: run it headed, window in front;
  `page.act` on one side and `click` on the other is `InputMismatch`.
- `--check --base a --head b` prints the parity matrix; `--why always` profiles each side once.
- Over time: `track <file> --base original=exe:/path --head rewrite=git:main --schedule commits`.

### Which commit moved it? Did anything regress?

- `bench <file> --filter <case> --bisect <good>..<bad>`: one series; each step is a full experiment;
  the culprit is benched against its own parent.
- `track <file> --range <old>..<branch> --every 3` (or `--schedule` on a branch), then
  `history <file>` or `query changes`. Only `confirmed` changes are findings.

### In CI: every pull request

- GitHub Actions: `uses: anomalyco/[email protected]` with `bench: perf/app.bench.ts` (inputs: action.yml).
  The default `mode: check` runs each case once, headless, and fails the job on a failed case.
  `noworse: net.requests=0,count.items=0` also runs the base commit and fails a case whose exact counter
  is over base's or varies between runs. A case base cannot run fails too, and a check whose cases
  compared none of the named counters exits 4 and fails the job: declare each counter in the metrics of
  the cases that count it. The step summary is `report experiment <exp> --format summary`.
- A shared runner (GitHub-hosted, `CI_SHARED_ENVIRONMENT=true`, or a CI job in a VM) refuses timing:
  `SharedRunner`, exit 4, for a timed bench, `--blocks`, `track`, `--bisect`, `plan --pilot` and a timed
  `record`. `status` says so. A dedicated CI VM says `RUNNER_ENVIRONMENT=self-hosted` or
  `CI_SHARED_ENVIRONMENT=false` (neither overrides GitHub-hosted). There: `--check` (`--base` with
  counter `--noworse`), leak runs (`--probes memory`), `--pixels-only`, `record --probes attribution`.
- Timing verdicts in CI: `mode: compare` on a self-hosted runner that runs one job at a time; the block
  goes in a same-repository PR's body (never a fork's). Never `pull_request_target`.
- The action installs nothing into the runner user's profile (its own strobe, no PATH entry or skill):
  it runs `up --skip self,skill,daemon,mcp,deep,calibrate`, which only fetches the engine.
- The step summary, the job's log and the artifacts never name the machine's other processes, other
  apps' windows or its user: the uploaded result.json is a scrubbed copy and its HTML report is drawn
  from that copy alone (no recordings, so no hotspots; the runner keeps them, `report-path`). The PR
  block, summary, card and badge name no other process, window title or person's folder either:
  every string of the result is scrubbed before they read it (a covering window by its process id, a
  plain folder variant as `<base folder>`/`<head folder>`; SPEC §11.2).

### No display, a locked session, a server

- `--headless` runs page cases (check or timed) with no display: frames, latency, scroll, canvas and
  GPU use refuse `Headless`; CPU, main-thread work, counters, phases and memory stay. Headless,
  what frames cost is `main.paint` beside `main.style` and `main.layout`.
- A headless page's lean block (and a `window: false` app's) takes no trace unless a case declares
  `phase.*`, `span.*`, `layout.shift`, `main.gc`, `gpu.compile` or `main.paint`: its `meta.json` says
  `"traced": false`. `summary` works. `frames` refuses `Headless` for the page or `NotForTarget` for
  the windowless app (4). `layout`, `style` and `gc` need attribution; `threads` lacks thread counters;
  `query a..b threads` says `this block took no trace` (2). With a trace, `frames` reads Chromium's
  pipeline, but the bench still refuses off-screen frame metrics. For trace data use a timed
  `bench --trace full` or a headed run. For a headless check, query the `--why always` attribution
  recording: `--trace full` alone does not change the check block's lean probes.
- A locked session runs what needs no presenting window: scripts, `cli()`, `tty()`, leak runs,
  headless pages, `window: false` native cases measuring `cpu.*`, `memory.*`, `gpu.vram`, `io.*`,
  `count.*`, `phase.*`, and any native case under `--check`. Compare them only with locked runs.
- The rest waits for an unlock (a line a minute). `--unlock-wait 2h` caps it: then those series are
  not run (exit 124, resumable) and the windowless ones still run.
- Locked, a native app's tree, `page.act` and keys work; clicks, moves and wheels refuse
  `SessionLocked`. On Windows its pixels still read (image anchors, `--pixels`).
- Linux without `DISPLAY`: native apps run on strobe's headless Wayland compositor (install sway,
  dbus-daemon and at-spi2-core); frames and latency refuse `Headless`.

### A busy server: which thread held it, and could a client get an answer?

- Host the server in a `script()` target. In `setup`, `await ctx.httpProbe({ url, authorization })`:
  strobe probes it every `cadenceMs` (100) from its own threads on fixed deadlines, `timeoutMs` 1000.
  The whole runnable example (`server.ts` too): `node_modules/@hona/strobe/README.md`.

```ts
import { bench, script } from "@hona/strobe"

export default bench({
  targets: { server: script("./server.ts") }, // exports start(tokenHash) and stall(ms)
  async setup(ctx) {
    const token = crypto.randomUUID() // the server gets its hash only
    const hash = new Bun.CryptoHasher("sha256").update(`Bearer ${token}`).digest("hex")
    const url = await ctx.call<string>("start", hash)
    const expect = { jsonPath: { path: "$.status", equals: "ok" } }
    await ctx.httpProbe({ url, authorization: `Bearer ${token}`, expect }) // every 100 ms, new connections
  },
  cases: {
    "slow handler": {
      primary: "http.latency.p95", // 4 stalls 100 ms apart: ~26 probes a run, enough for a p95
      metrics: ["cpu.main", "cpu.main.busy", "http.timeouts"],
      run: async (ctx) => {
        for (let i = 0; i < 4; i++) {
          await ctx.call("stall", 550) // back to back, a probe can wait out two stalls
          await Bun.sleep(100)
        }
      },
    },
  },
})
```

- `record --bench <file>#<case>` (or `--check`, `--why always`): stacks, the threads' timeline and the
  probe on one recording. `query <rec> http`: idle against the runs, timeouts, errors and the stalled
  cohorts: one stall (`answered none for X`) or stalls back to back (`spanned N stalls …`), timeouts
  wherever they fell; a quantile the block refuses reads `—` with why. `strobe open <rec>` draws the
  probes beside the runs and threads, and `report` an HTTP probe card.
- To compare it, declare `http.latency.p95` (or `http.timeouts`, `http.errors`): the probe then runs
  in probe blocks of its own. A clean timing block never probes.
- A timeout ranks above every answer: a quantile on one refuses `RunIncomplete` (raise `timeoutMs`).
  A 500 or a failed `expect` refuses the latencies. `http.missed_sends` is strobe's lag, not the server's.
- `authorization` stays in strobe's memory: never in a file, report, trace or answer. Loopback only.
- `cpu.main` is the main thread alone beside `cpu.work` (the tree): `cpu.main.busy` near 100% is a
  saturated event loop; a worker or a child process shows in `cpu.work` only. A script's (or a
  windowless native app's) check line shows both even when the case does not declare them.
- A script's `query <rec> hotspots` is its measured runs; `--source host` the whole host process
  (start, setup, warmup, resets), its `by lifecycle` lines each part's wall time against the JavaScript
  sampled in it; `--source startup` the host's start and the target's static imports, `--source setup`
  the bench file's setup (a server's dynamic imports); `--phase <name>` one phase.
  - Slow startup with few rows: `N modules evaluated` (each module is a stack of its own: `--by file`
    sums them). Lifecycle shares are of wall time: unsampled wall is mostly waiting, and on Bun
    native work (transpiling, module loading) too, which `--cpu-prof` never samples.
  - `! Missed`: the main thread used CPU (`cpu.main`) the profile has no sample of, so rows
    undercount (Node labels short timer callbacks `(idle)`; Bun never samples its event loop's own
    work). `! Waited`: rows include time the thread waited for a CPU (samples weigh wall time).
  - `query <rec> threads --timeline`: which thread was busy when, runs marked.

### Phones

- Emulated: `web({ emulate: "mobile" })` is a phone (390×844 @3x, touch, headless): `click` is a tap,
  `drag` a finger, `scroll` swipes (Chromium only); `hover`, `move`, `wheel` and mouse buttons refuse
  `DeviceMismatch`. Headless, so `frames.*` and `interaction.*` refuse `Headless`: measure
  `main.work`, `cpu.*` or `run.duration` there, and frames and latency on a real phone (`--mobile`).
  `"touch"` is a desktop with a touch screen: the mouse stays its primary pointer.
- A real Android phone: `--mobile` (or `--mobile=<serial>`) runs the file's `emulate: "mobile"`
  targets in Chrome on the phone over adb; `look --mobile`, `record --mobile` too.
  - Give strobe a channel once: install Chrome Canary (or Dev, Beta), open it once, then
    `strobe up --mobile=<serial> --channel canary`. strobe never drives the phone's own Chrome.
  - An emulator runs `--check`, `look`, attribution and leak runs (`--probes memory`) only (`Emulator`);
    timed runs refuse Wi-Fi adb, Battery Saver, under 20% battery, and wait up to 3 min for a hot phone.
  - Leak runs and `record --mobile --probes memory` work on a phone (the phone's heap and snapshots;
    no `memory.app.private`: read `js.heap.used`, its growth and the leak suspects, or a desktop
    browser's memory recording). Loopback addresses `prepare` returns
    (an API, a WebSocket) are reversed to the phone, like `serve`.
  - `cpu.*`, `memory.*`, `gpu.*` refuse `NotForTarget` (this computer's processes): measure
    `main.work`, `frames.*`, `interaction.*`, `run.duration`. A lost phone ends the bench (exit 255).
- An iPhone on a cable: `look wir:` and `record wir:` read an open Safari tab or WKWebView over Web
  Inspector, read-only (`hotspots`, `heap`, `memory`, `log --source console|device|crash`,
  `network`); `strobe status` lists the phones and their pages.

### Writing the PR: every performance claim comes from strobe

1. **Discovery**: the strobe command and recording id that pointed at the problem, and what it showed.
2. **The change**: what the code does differently.
3. **Results**: `strobe report experiment <exp> --format pr --update-pr <n>` puts strobe's block in
   PR n's body, led by its result card (or paste the `--format pr` output).
   - The block is delimited (`<!-- strobe:report <exp> v1 begin sha256:… -->` … "end of generated
     report"). Never edit inside it: no retyped, rounded or dropped numbers, no rewritten verdicts;
     strobe refuses to update a hand-edited block. Write your reading above or below it, and say
     there when a metric was refused or `unclear`. One block per experiment.
   - Never put a card or badge in a PR without the block. `--format card|badge` is for elsewhere.
   - The sides read "Before" and "This PR" only when head is the working tree or `HEAD`, and with
     `--update-pr` at the PR's head commit; an A/B of other builds reads by commit, an A/A as `Base (A/A)`
     and `Head (A/A)`. Name them with `--labels <base>,<head>` (`--labels v1.4,v1.5`).
4. **The bench**: the case's code, or a link to it. A new case ships in the PR of the first change
   it measures.

Never put hand-timed numbers, single recordings or `--check` values in a results table: label them
noisy discovery evidence, or leave them out.

### When strobe refuses

- Exit 4 names the validator and the fix: fix the setup, never work around it. Common ones:
  - `WindowHidden`, `SessionLocked`: the window must show; wait or unlock.
  - `Background`: other work on the target's CPUs (named); a timed bench waits it out before refusing.
  - `Throttled`: the speed check stayed slow on quiet CPUs, a Linux CPU quota throttled the runs,
    or a headless page ran with the display off.
  - A `--check` never waits on the speed check: a slow one is a `! Background` or `! Throttled` line and
    the case runs at once. Its values are then a loaded machine's.
  - `RunIncomplete` on `cpu.renderer`/`cpu.gpu`/`cpu.browser` names why: a process of that role ended
    in the run, or ended processes (their CPU pooled) used over 1% of it. `cpu.work` keeps
    their CPU: compare that.
  - `RunIncomplete`: a run threw, timed out, its `until` never came, or the app exited. "The host
    wrote none": a script host's profile never came; it says how the host ended and ends `script.log`.
  - `ResetIncomplete`: a run started from another state than run 1 (`the reset did not restore …`).
  - `TailTooShort`: a percentile lacks its minimum n (`interaction.p95` needs 20 samples a block).
  - `AmbiguousInteraction`: over half the inputs came while the target was drawing.
  - `NotOnThisOs`, `NotForTarget`, `EngineUnsupported`, `Headless`, `NoCounter`: this OS, target,
    engine or mode cannot give that metric; pick another primary (the message names one). A page
    (or an app over DevTools) has no `cpu.wakes`, `io.*` or `ui.*` (on macOS: `cpu.idle_wakes`).
  - A declared metric no reading gave is refused with why, never dropped from the table: `<m>: a script
    target never gives it; instead, measure cpu.main or cpu.work` (`NotForTarget`), `… gives it on other
    OSes, not on Windows` (`NotOnThisOs`), `no measured run sent the input it times` (`NoInput`), else
    `NoMetric`. As the primary: the case FAILs in a check (exit 1), the series is refused at block 1 in
    a comparison. A `cli()` reading the case did not declare is left out, never listed refused.
  - An app with `window: false` (on the target or its profile; `window: true` on a native target wins):
    frames, latency, scrolling, canvas, display and GPU use are refused `NotForTarget` (a primary of
    them before it launches: pick `cpu.work`, `run.duration`, a memory metric or `phase.<name>`). Its
    run cases need no window or unlocked session; a window it shows anyway refuses the block
    (`VariantMismatch: … declares window: false, yet it showed a window on screen`).
  - Two sides on one engine where one has nothing on screen (a headless page, a `window: false` app) and
    the other shows its window: `VariantMismatch` before anything builds, in `bench` and `plan`, also for
    two profiles of one target (`--base app:desk --head app:tray`). Run both off screen or both on it.
  - `js.heap.retained` on a script host with no collector to call: that metric alone,
    `EngineUnsupported`. macOS Bun `memory.app.private`: `EngineUnsupported` (a primary before the
    file's other targets run).
  - `WindowHidden` on a page: it stayed hidden 200 ms at a frame wait (a `window.open` popup hides
    its opener on macOS): refused at once, never waited out. Close a popup the case opened before
    such a wait, or measure the case without frame metrics.
  - Exit 3 naming the step: macOS held the launch (Gatekeeper after a download or update): open the
    app once from the Finder.
  - `OverheadBudget`: strobe's own work disturbed timed runs (a `--check` only warns).
  - `VariantMismatch`, `DisplayTooSmall`, `DeviceMismatch`, `InputMismatch`: the sides differ.
  - `SharedRunner`: a CI runner other jobs share; no timing there. `--check`, counters against base, leak
    runs and attribution work (In CI, above); timing needs a machine of its own.
- In comparisons, single-build blocks and leak runs, a series refused while it runs (`NoPhase`, for
  example) ends alone: `refused` in the listing. In a check a run defect fails the case (1), but a
  primary the target, engine or display cannot provide is refused. Only
  the machine's refusals (`SessionLocked`, `Background`, `Thermal`, …) stop the bench. Every series
  refused: exit 4.
- `!` lines warn next to a kept number; `i` lines inform. Read them before trusting a number:
  `NotSteady`, `HighVariance`, `ScreenCh...

© anomalyco · MIT · anomalyco/strobe

Similar skills

All skills for Testing and QA →

Project sponsors

Companies that keep this analytics open