strobe — скилл для Claude Code: замеры производительности
Скилл strobe для Claude Code измеряет производительность приложений и объясняет результаты. Запускается, когда нужно сравнить версии, найти причину тормозов или написать бенчмарк.
MIT ★ 173 обновлён 2026-11-16
npx skills add anomalyco/strobe --skill strobe -a claude-codeЧто делает strobe
strobe помогает создавать и запускать бенчмарки для веб-страниц, Electron- и нативных приложений, терминальных программ и процессов Bun или Node. Он сравнивает базовую и новую версии, выдаёт вердикт по сериям измерений и показывает подозрительные места с источниками и следующими командами для проверки.
Для старта strobe init создаёт конфигурацию и черновой бенчмарк на основе файлов проекта; затем его можно проверить через strobe bench <file> --head . --check. Для чистого сравнения нужно как минимум восемь пар измерений. Если результаты недостаточны для обоснованного вывода, strobe сообщает об этом отдельным вердиктом, а не трактует запуск как отсутствие разницы.
Какие задачи решает
- сравнить производительность базовой и новой версии
- найти участок, связанный с замедлением или ростом памяти
- создать черновой бенчмарк из файлов проекта
- проверить, что бенчмарк запускается и выдаёт метрики
Примеры запросов агенту
- Сравни производительность изменений с
origin/devс помощьюperf/smoke.bench.ts. - Разберись, почему эта страница стала медленнее и где растёт память.
- Создай бенчмарк для приложения из этого проекта.
Как установить strobe
1Через Skills CLI (Vercel)
npx skills add anomalyco/strobe --skill strobe -a claude-code2Вручную: положить файлы
~/.claude/skills/skill/SKILL.mdПапка скилла целиком; для одного проекта — .claude/skills/skill/
1Через Skills CLI (Vercel)
npx skills add anomalyco/strobe --skill strobe -a codex2Вручную: положить файлы
.agents/skills/skill/SKILL.mdДля всех проектов — ~/.agents/skills/
1Через Skills CLI (Vercel)
npx skills add anomalyco/strobe --skill strobe -a cursor2Вручную: положить файлы
.cursor/skills/skill/SKILL.mdCursor читает и .agents/skills/, и .claude/skills/
1GitHub CLI
gh skill install anomalyco/strobe skill2Вручную: положить файлы
.github/skills/skill/SKILL.md1Через Skills CLI (Vercel)
npx skills add anomalyco/strobe --skill strobe -a gemini-cli2Командой платформы
gemini skills install https://github.com/anomalyco/strobe --path skillВручную: положить файлы
.devin/skills/skill/SKILL.mdСтарый путь .windsurf/skills/ тоже читается
Claude.ai
Упакуйте папку в zip и загрузите в Settings → Capabilities → Skills
По шагам, с командами для каждой платформы: Как установить скилл в Claude Code: пять способов
Вопросы о strobe
Как установить strobe в Claude Code?
Выполни npx skills add anomalyco/strobe --skill strobe -a claude-code. Скилл устанавливается в ~/.claude/skills/skill/SKILL.md.
Что делает strobe?
Скилл измеряет производительность программ и сравнивает запуски, например базовую и новую версии. Он выдаёт вердикты и помогает исследовать возможные причины замедления или роста памяти.
Какие приложения strobe умеет измерять?
В описании перечислены веб-страницы, Electron-приложения, нативные приложения, терминальные программы, процессы Bun и Node, а также другие процессы.
Исходный текст
SKILL.md · MIT
---
name: strobe
description: Measure and explain performance of web pages, Electron apps, native apps, terminal programs, Bun or Node processes and any process. Use when the user asks whether a change is faster, why something is slow, janky or heavy, where memory goes, what the CPU or GPU is doing, or to write a benchmark.
---
# Strobe
strobe records a target on one timeline and answers with verdicts and handles (`rec_…`, `exp_…`) to
drill into. It refuses measurements it cannot defend: a refused run is an answer, never "no value".
The design and every rule behind this file: SPEC.md.
## Playbooks
### No bench yet: start one
1. `bun add -d @hona/strobe` pins strobe for the project (`bunx strobe …` then runs that version).
2. `strobe init` writes `strobe.json` (its `$schema` linked) and a runnable `perf/smoke.bench.ts` from
the folder's package.json (`build`, `preview`, `start`, `dev`, `exports`, `module`, `types`, `main`,
`bin`, an electron dependency), Cargo.toml binaries or index.html.
`--target web|lib|server|native|electron|tty|cli` overrides the guess: web serves the build with
`perf/serve.ts` (a cold load, an idle second, and a commented reload: a small page's reload is
mostly strobe's own driver work, `OverheadBudget`), lib imports the package's entry from
`perf/lib.ts` (a cold import per run; a commented `work` case calls an export with `ctx.call`),
server hosts it in `perf/server.ts` (`PORT`),
native builds and launches the Cargo app, cli runs the command to its exit. It never replaces a file
without `--force`.
3. Run what it prints: `strobe config`, then `strobe bench perf/smoke.bench.ts --head . --check`
(`--headless` with no display or a locked session). Then edit the cases to what users do.
### Did my change make it faster?
- Allow at least eight pairs for the clean-pair check. `strobe run --plan` previews a command comparison.
- On a busy desk use `cpu.work` or, for headless pages, `main.work`. Read any `Background` warning.
1. Write or find a bench file (below; `strobe init` writes a first one). Check it first:
`strobe bench <file> --head . --check` (each case once, pass or fail with a screenshot).
2. `strobe plan <file> --base origin/dev --head .` says pairs and minutes per series (`time unknown`,
never a guess, for a series no `--check`, `--pilot` or earlier run measured here).
3. `strobe bench <file> --base origin/dev --head .`: a verdict per series (case × target × params).
- Read the ★ primary's verdict: `faster`/`slower` (`less`/`more`, `fewer`), `same (±2%)`,
`unclear`, `underpowered` (more pairs or a quieter machine, not "no change"), `refused`.
- Secondaries get words only after the primary decided (Holm-corrected); `·` is descriptive.
- `ledger 3rd experiment on these variants; earlier: …`: reruns count. Paste it as it is.
- `strobe query <exp> insights`: per series the verdict (or a check's status), ranked suspects
with their source, the warnings that bear on it, and commands to run next as printed (`--pairs
N`, `record --bench <file>#<case>` for a side nothing profiled). Any kind: check, `--blocks`, leak
run. Head's hits read `(over its threshold on head only)`, `(… on base too)` or `(base not read
for it)`; beside `same`, `unclear` or an A/A no mover is a suspect (two launches differ anyway).
4. Put the evidence in the PR (below).
- Refactor or dependency bump? `--noworse 2%` (or `noworse: "2%"` on the case;
`--noworse 2%,cpu.wakes=5%` per metric): `no worse (≤2%)`, `worse (>2%)` or `unclear`. A bare exact
counter is `0%` (`--noworse count.items`); a bare other metric exits 2 (`main.work needs a margin`).
- `strobe run --base "<old command>" --head "<new command>"`: the same A/B for two command lines.
`--check --head "<cmd>"` (and `--base "<cmd>"`) runs each once: `cpu.work · run.duration ·
memory.peak`. With `--check` every `--noworse` exits 2 (a command gives no exact counter): drop
`--check` and use `--noworse cpu.work=2%`.
- `ShellLine`: `&&`, `||`, pipes and redirects run through the platform shell. Its startup counts.
Plain external commands use PATH; use `./tool` or `.\tool.exe` to measure each variant's build.
- `--aa --head .` checks the method on this machine (same build both sides).
- Long benches keep running in the daemon: `strobe query exp_… progress`. Ctrl-C, SIGTERM or
`strobe bench --stop <exp>` stop after the block (exit 130, what finished is kept);
`strobe bench --resume <exp>` goes on.
### How a run ends (the exit code says whether strobe measured, never who won)
- A timed comparison can exit 0 on `slower` or `worse`: read `series[].primary.verdict_kind`.
Every non-gap primary refused: 4. `--require-verdict`: 5 if undecided, 4 if any primary is refused
or any journey is a parity gap. `--require-parity` alone makes gaps exit 4, not undecided results.
- A check is no verdict: each case once (each side once with `--base`), proving the bench runs and each
declared metric gives a value or a stated refusal.
- `--check`: 0 when every case passes; 1 when one `FAIL`s (its `fix:` line says what to change); a
preflight refusal is listed beside passing cases; all non-gap cases refused up front: 4.
A run defect fails its case (1); a primary the target, engine or display cannot measure is
listed as refused. A refused secondary alone does not fail a case.
- `--check --base <ref>`: base's line under head's; the summary ends `base and head each checked;
nothing compared: the same command without --check times head against base` (exit 0 if both ran).
A base build error stops the check before the case runs (1). A base run error fails the case (1):
`1 failed on base`; head's values if it passed, head's error if it failed too. Leave head-only cases
out with `--filter` until the change merges.
- `--check --base <ref> --noworse count.x`: each counter row ends `no worse` (passes), `worse`,
`unstable`, `missing` or `not_compared` (each fails the case, exit 1). If cases ran and none failed
but no counter compared, exit **4**, with `passed: false` in JSON. If both sides refused every
counter, read the refused lines. For `Uncounted`, check the spelling against `ctx.count`; if the run
counts it only on some paths, add `metrics: ["count.x"]` on that case so 0 → 0 compares.
If every case is a parity gap, no counter compared: exit 4. `--require-parity` requires every journey.
- `--check --base <target> --head <target>`: shared journeys get a parity matrix, not a counter
comparison. Gaps alone exit 0 (even if no case ran), or 4 with `--require-parity`; a mismatch or
case failure exits 1. With two bare targets, `--noworse` exits 2: it needs two builds.
- `--blocks N`: one build's values, no verdict. `--probes memory`: a leak run exits 0 even for `leaks`
(`worse`); 5 if undecided with `--require-verdict`. Both modes exit 4 if every series is refused.
- `--pixels-only`: 1 if snapshots differ or one side lacks one, otherwise 0 on completion. `--bisect`:
`found`, `no change`, `unclear` exit 0; `--require-verdict` makes an `unclear` search exit 5.
- Stops and caps: 130 stopped; 124 for an exhausted `--unlock-wait` or a non-comparison timeout.
A timed comparison hitting `--max-time` keeps its finished pairs and normal exit rules (usually
0, or 5 if undecided with `--require-verdict`). The slot wait for display fit before the first
series can still exit 124. Resume stopped work with `bench --resume <exp>` if state was saved.
### Why is it slow?
- `bench … --why always` (an attribution block per series), `bench … --check --why always` (one build),
or `strobe record <target> --probes attribution` / `strobe record --bench <file>#<case>`. Then:
- `query <rec> hotspots` (CPU by function, full stacks, source-mapped file:line:column;
`--by file|package`; `--sort total` for callers; native: `--thread <name>`, `--pid <pid|exe>`,
`--phase <name>`).
- `query <rec> layout` (forced layouts by their call site), `style` (what invalidated style:
attribution blocks trace it for a series declaring `main.style` or `style.count`, or with
`--style`), `alloc` (who allocates), `gc`, `summary` (per run: style, layout, paint ms), `insights`.
- A page served from your own folder (`record https://localhost:… --served <dir>`): its source-mapped
frames resolve to files on disk.
- What changed: `query <exp> hotspots` (each series' base..head) or `query <recA>..<recB> hotspots`;
`~` marks a change inside its sampling interval (one launch a side: not a verdict).
- `strobe open <rec>` (or `<exp>/head`): the whole recording in a local Perfetto UI. For seeing;
numbers come from strobe's views. The "strobe itself" track is strobe's own cost: check it first.
- A thread waiting (a lock, IO, a timer) and who wakes it: `record pid:<n> --probes waits --for 10s`,
then `query <rec> waits --thread <name>` (Linux needs CAP_PERFMON, Windows the deep helper;
macOS refuses `NoWaits`).
- A slow page frame in the renderer or the GPU process: `frames.renderer.p95` against `frames.viz.p95`,
and `query <rec> frames` (each worst frame's stages).
- Inputs with no latency sample (`busy`), or one that looks wrong: `query <rec> inputs [--case <c>]`
(the first 100; `--top N`, `--json` every one):
each input's run and time, what proved its change, the frames around it, its latency or why it
was busy (`mid_drawing`, `behind`, `ambiguous_press`, `multi_frame_pixel`) and how much later it
had to come. Give the case that wait before such inputs.
- A profiled launch is slower than a timed one: compare totals only from the timed pairs.
### Where is the memory? Does it leak?
- `query <rec> memory` (per process; `--by layer|allocator` for Chromium's accounting). A native
app: what holds it (host heap, GPU memory mapped in, unmapped commit, shared, files) beside
`gpu.vram`. On Windows `memory.app.private` counts VRAM a driver maps in: quote `memory.app.host`.
A phone's page: processes from Chromium's dump (no OS memory); listeners are the page's own.
- `query <rec> heap`: classes by retained size, detached DOM, retainer paths; `query <a>..<b> heap`.
- A case's leak: `bench <file> --filter <case> --head . --probes memory --repeat 10`: growth per run
with an interval, and `query <rec> leaks` for what from the first half is still alive.
- An app over time: `record <target> --probes memory --for 60s` while it does the suspect thing.
- A script target: `js.heap.retained` (what a run keeps after a collection, buffers counted once). On
a Node host it adds ArrayBuffer contents in releases after 0.31.0 (which left them out): compare
across that upgrade as two histories (`track` starts a new series, `metric revision 2`; Bun series
keep theirs). Every Node-hosted script series, any metric, started a new history once in 0.32.0:
its identity records `runtime node` (`strobe history` shows it). It does not see add-ons' native
memory, mapped files, or on Node WebAssembly memory and strings outside the heap: read
`memory.app.private` for those. On macOS a Bun host's
`memory.app.private` is refused (`EngineUnsupported`): Bun's allocator hides memory from the OS.
- Sizes are binary: KB, MB and GB are 2^10, 2^20 and 2^30 bytes.
### A native app, or any process from outside
- `record app:<id> --for 10s` (a strobe.json profile) or `record exe:<path> --native`: CPU, thread
wakes, IO, GPU, memory, the UI thread and what reached the screen. `--windowless` for a tray app
(`look … --windowless` first: does it stay up, what does it print?).
`record pid:<n>` or `proc:<name>` for a process strobe did not start.
- `query <rec> threads`: wakes/s and CPU per thread, the module each started in, whose (app, driver,
os, `unknown (?)`); `--timeline` per second. `query <rec> insights` rules include dominant-thread,
off-cpu-bound, style-layout-dominant, wakes-while-idle (idle blocks too), hot-function (a named
function of the target's over 25% of busy samples, one hit each; a native symbol without line
info counts), http-stalls (measured runs only).
- Native hotspots of a tree pool every process: read `by process`, then `--pid <pid|exe>`.
A native attribution recording (`record --bench`, `--why`) keeps frames too: `query <rec> inputs`.
`! Coverage`: samples cover under 80% of the CPU (read shares, not totals); it names what the
sampler counted of the rest (`late_ms` before a thread was opened, `forgone_ms`, unopened threads,
lost events). A launched app's children are sampled from their start. Two resolved shares
(leaf, all frames): a low all-frame share means the app's own binary has no symbols.
macOS samples are untimed snapshots: counts, never ms.
- `record pid:… --probes deep|waits` prints `cost`; `! CaptureCost`: the capture slowed a
switch-heavy process, so its stacks describe that slower process.
- Benching it: `native("my-app")` in a bench file (below).
### Two implementations of one app (a port, a rewrite)
- One bench file, one target per implementation, the same cases (journeys):
`bench <file> --base original --head rewrite` (`rewrite=git:main`, `original=exe:/path`;
`target:<name>` when a folder has that name). Each target's `name` is what reports call it.
- Pick a primary both sides measure the same way: `cpu.work`, `run.duration`, `launch.ready`,
`memory.app.private`. `main.work` or `ui.cpu` across engines exits 2 (`NotComparable`).
- Give each target a `ui` adapter for what differs (`ui: { row: ".row", open: ({ page }) => … }`);
journeys say `@row`, `await ctx.ui.open()`. A journey one side cannot run is a parity gap
(`unsupported: { rewrite: "why" }`, or an `@name` its `ui` lacks): listed, never run.
- `expect: { same: ["@topRow"] }`: both sides must read it alike after setup, each run and reset, or
`NotEquivalent`. Both talk to one `fixture("<server> --port {port}")` (`{fixture.url}` in args/env).
- A page against a native app (or another engine) gets OS input: run it headed, window in front;
`page.act` on one side and `click` on the other is `InputMismatch`.
- `--check --base a --head b` prints the parity matrix; `--why always` profiles each side once.
- Over time: `track <file> --base original=exe:/path --head rewrite=git:main --schedule commits`.
### Which commit moved it? Did anything regress?
- `bench <file> --filter <case> --bisect <good>..<bad>`: one series; each step is a full experiment;
the culprit is benched against its own parent.
- `track <file> --range <old>..<branch> --every 3` (or `--schedule` on a branch), then
`history <file>` or `query changes`. Only `confirmed` changes are findings.
### In CI: every pull request
- GitHub Actions: `uses: anomalyco/[email protected]` with `bench: perf/app.bench.ts` (inputs: action.yml).
The default `mode: check` runs each case once, headless, and fails the job on a failed case.
`noworse: net.requests=0,count.items=0` also runs the base commit and fails a case whose exact counter
is over base's or varies between runs. A case base cannot run fails too, and a check whose cases
compared none of the named counters exits 4 and fails the job: declare each counter in the metrics of
the cases that count it. The step summary is `report experiment <exp> --format summary`.
- A shared runner (GitHub-hosted, `CI_SHARED_ENVIRONMENT=true`, or a CI job in a VM) refuses timing:
`SharedRunner`, exit 4, for a timed bench, `--blocks`, `track`, `--bisect`, `plan --pilot` and a timed
`record`. `status` says so. A dedicated CI VM says `RUNNER_ENVIRONMENT=self-hosted` or
`CI_SHARED_ENVIRONMENT=false` (neither overrides GitHub-hosted). There: `--check` (`--base` with
counter `--noworse`), leak runs (`--probes memory`), `--pixels-only`, `record --probes attribution`.
- Timing verdicts in CI: `mode: compare` on a self-hosted runner that runs one job at a time; the block
goes in a same-repository PR's body (never a fork's). Never `pull_request_target`.
- The action installs nothing into the runner user's profile (its own strobe, no PATH entry or skill):
it runs `up --skip self,skill,daemon,mcp,deep,calibrate`, which only fetches the engine.
- The step summary, the job's log and the artifacts never name the machine's other processes, other
apps' windows or its user: the uploaded result.json is a scrubbed copy and its HTML report is drawn
from that copy alone (no recordings, so no hotspots; the runner keeps them, `report-path`). The PR
block, summary, card and badge name no other process, window title or person's folder either:
every string of the result is scrubbed before they read it (a covering window by its process id, a
plain folder variant as `<base folder>`/`<head folder>`; SPEC §11.2).
### No display, a locked session, a server
- `--headless` runs page cases (check or timed) with no display: frames, latency, scroll, canvas and
GPU use refuse `Headless`; CPU, main-thread work, counters, phases and memory stay. Headless,
what frames cost is `main.paint` beside `main.style` and `main.layout`.
- A headless page's lean block (and a `window: false` app's) takes no trace unless a case declares
`phase.*`, `span.*`, `layout.shift`, `main.gc`, `gpu.compile` or `main.paint`: its `meta.json` says
`"traced": false`. `summary` works. `frames` refuses `Headless` for the page or `NotForTarget` for
the windowless app (4). `layout`, `style` and `gc` need attribution; `threads` lacks thread counters;
`query a..b threads` says `this block took no trace` (2). With a trace, `frames` reads Chromium's
pipeline, but the bench still refuses off-screen frame metrics. For trace data use a timed
`bench --trace full` or a headed run. For a headless check, query the `--why always` attribution
recording: `--trace full` alone does not change the check block's lean probes.
- A locked session runs what needs no presenting window: scripts, `cli()`, `tty()`, leak runs,
headless pages, `window: false` native cases measuring `cpu.*`, `memory.*`, `gpu.vram`, `io.*`,
`count.*`, `phase.*`, and any native case under `--check`. Compare them only with locked runs.
- The rest waits for an unlock (a line a minute). `--unlock-wait 2h` caps it: then those series are
not run (exit 124, resumable) and the windowless ones still run.
- Locked, a native app's tree, `page.act` and keys work; clicks, moves and wheels refuse
`SessionLocked`. On Windows its pixels still read (image anchors, `--pixels`).
- Linux without `DISPLAY`: native apps run on strobe's headless Wayland compositor (install sway,
dbus-daemon and at-spi2-core); frames and latency refuse `Headless`.
### A busy server: which thread held it, and could a client get an answer?
- Host the server in a `script()` target. In `setup`, `await ctx.httpProbe({ url, authorization })`:
strobe probes it every `cadenceMs` (100) from its own threads on fixed deadlines, `timeoutMs` 1000.
The whole runnable example (`server.ts` too): `node_modules/@hona/strobe/README.md`.
```ts
import { bench, script } from "@hona/strobe"
export default bench({
targets: { server: script("./server.ts") }, // exports start(tokenHash) and stall(ms)
async setup(ctx) {
const token = crypto.randomUUID() // the server gets its hash only
const hash = new Bun.CryptoHasher("sha256").update(`Bearer ${token}`).digest("hex")
const url = await ctx.call<string>("start", hash)
const expect = { jsonPath: { path: "$.status", equals: "ok" } }
await ctx.httpProbe({ url, authorization: `Bearer ${token}`, expect }) // every 100 ms, new connections
},
cases: {
"slow handler": {
primary: "http.latency.p95", // 4 stalls 100 ms apart: ~26 probes a run, enough for a p95
metrics: ["cpu.main", "cpu.main.busy", "http.timeouts"],
run: async (ctx) => {
for (let i = 0; i < 4; i++) {
await ctx.call("stall", 550) // back to back, a probe can wait out two stalls
await Bun.sleep(100)
}
},
},
},
})
```
- `record --bench <file>#<case>` (or `--check`, `--why always`): stacks, the threads' timeline and the
probe on one recording. `query <rec> http`: idle against the runs, timeouts, errors and the stalled
cohorts: one stall (`answered none for X`) or stalls back to back (`spanned N stalls …`), timeouts
wherever they fell; a quantile the block refuses reads `—` with why. `strobe open <rec>` draws the
probes beside the runs and threads, and `report` an HTTP probe card.
- To compare it, declare `http.latency.p95` (or `http.timeouts`, `http.errors`): the probe then runs
in probe blocks of its own. A clean timing block never probes.
- A timeout ranks above every answer: a quantile on one refuses `RunIncomplete` (raise `timeoutMs`).
A 500 or a failed `expect` refuses the latencies. `http.missed_sends` is strobe's lag, not the server's.
- `authorization` stays in strobe's memory: never in a file, report, trace or answer. Loopback only.
- `cpu.main` is the main thread alone beside `cpu.work` (the tree): `cpu.main.busy` near 100% is a
saturated event loop; a worker or a child process shows in `cpu.work` only. A script's (or a
windowless native app's) check line shows both even when the case does not declare them.
- A script's `query <rec> hotspots` is its measured runs; `--source host` the whole host process
(start, setup, warmup, resets), its `by lifecycle` lines each part's wall time against the JavaScript
sampled in it; `--source startup` the host's start and the target's static imports, `--source setup`
the bench file's setup (a server's dynamic imports); `--phase <name>` one phase.
- Slow startup with few rows: `N modules evaluated` (each module is a stack of its own: `--by file`
sums them). Lifecycle shares are of wall time: unsampled wall is mostly waiting, and on Bun
native work (transpiling, module loading) too, which `--cpu-prof` never samples.
- `! Missed`: the main thread used CPU (`cpu.main`) the profile has no sample of, so rows
undercount (Node labels short timer callbacks `(idle)`; Bun never samples its event loop's own
work). `! Waited`: rows include time the thread waited for a CPU (samples weigh wall time).
- `query <rec> threads --timeline`: which thread was busy when, runs marked.
### Phones
- Emulated: `web({ emulate: "mobile" })` is a phone (390×844 @3x, touch, headless): `click` is a tap,
`drag` a finger, `scroll` swipes (Chromium only); `hover`, `move`, `wheel` and mouse buttons refuse
`DeviceMismatch`. Headless, so `frames.*` and `interaction.*` refuse `Headless`: measure
`main.work`, `cpu.*` or `run.duration` there, and frames and latency on a real phone (`--mobile`).
`"touch"` is a desktop with a touch screen: the mouse stays its primary pointer.
- A real Android phone: `--mobile` (or `--mobile=<serial>`) runs the file's `emulate: "mobile"`
targets in Chrome on the phone over adb; `look --mobile`, `record --mobile` too.
- Give strobe a channel once: install Chrome Canary (or Dev, Beta), open it once, then
`strobe up --mobile=<serial> --channel canary`. strobe never drives the phone's own Chrome.
- An emulator runs `--check`, `look`, attribution and leak runs (`--probes memory`) only (`Emulator`);
timed runs refuse Wi-Fi adb, Battery Saver, under 20% battery, and wait up to 3 min for a hot phone.
- Leak runs and `record --mobile --probes memory` work on a phone (the phone's heap and snapshots;
no `memory.app.private`: read `js.heap.used`, its growth and the leak suspects, or a desktop
browser's memory recording). Loopback addresses `prepare` returns
(an API, a WebSocket) are reversed to the phone, like `serve`.
- `cpu.*`, `memory.*`, `gpu.*` refuse `NotForTarget` (this computer's processes): measure
`main.work`, `frames.*`, `interaction.*`, `run.duration`. A lost phone ends the bench (exit 255).
- An iPhone on a cable: `look wir:` and `record wir:` read an open Safari tab or WKWebView over Web
Inspector, read-only (`hotspots`, `heap`, `memory`, `log --source console|device|crash`,
`network`); `strobe status` lists the phones and their pages.
### Writing the PR: every performance claim comes from strobe
1. **Discovery**: the strobe command and recording id that pointed at the problem, and what it showed.
2. **The change**: what the code does differently.
3. **Results**: `strobe report experiment <exp> --format pr --update-pr <n>` puts strobe's block in
PR n's body, led by its result card (or paste the `--format pr` output).
- The block is delimited (`<!-- strobe:report <exp> v1 begin sha256:… -->` … "end of generated
report"). Never edit inside it: no retyped, rounded or dropped numbers, no rewritten verdicts;
strobe refuses to update a hand-edited block. Write your reading above or below it, and say
there when a metric was refused or `unclear`. One block per experiment.
- Never put a card or badge in a PR without the block. `--format card|badge` is for elsewhere.
- The sides read "Before" and "This PR" only when head is the working tree or `HEAD`, and with
`--update-pr` at the PR's head commit; an A/B of other builds reads by commit, an A/A as `Base (A/A)`
and `Head (A/A)`. Name them with `--labels <base>,<head>` (`--labels v1.4,v1.5`).
4. **The bench**: the case's code, or a link to it. A new case ships in the PR of the first change
it measures.
Never put hand-timed numbers, single recordings or `--check` values in a results table: label them
noisy discovery evidence, or leave them out.
### When strobe refuses
- Exit 4 names the validator and the fix: fix the setup, never work around it. Common ones:
- `WindowHidden`, `SessionLocked`: the window must show; wait or unlock.
- `Background`: other work on the target's CPUs (named); a timed bench waits it out before refusing.
- `Throttled`: the speed check stayed slow on quiet CPUs, a Linux CPU quota throttled the runs,
or a headless page ran with the display off.
- A `--check` never waits on the speed check: a slow one is a `! Background` or `! Throttled` line and
the case runs at once. Its values are then a loaded machine's.
- `RunIncomplete` on `cpu.renderer`/`cpu.gpu`/`cpu.browser` names why: a process of that role ended
in the run, or ended processes (their CPU pooled) used over 1% of it. `cpu.work` keeps
their CPU: compare that.
- `RunIncomplete`: a run threw, timed out, its `until` never came, or the app exited. "The host
wrote none": a script host's profile never came; it says how the host ended and ends `script.log`.
- `ResetIncomplete`: a run started from another state than run 1 (`the reset did not restore …`).
- `TailTooShort`: a percentile lacks its minimum n (`interaction.p95` needs 20 samples a block).
- `AmbiguousInteraction`: over half the inputs came while the target was drawing.
- `NotOnThisOs`, `NotForTarget`, `EngineUnsupported`, `Headless`, `NoCounter`: this OS, target,
engine or mode cannot give that metric; pick another primary (the message names one). A page
(or an app over DevTools) has no `cpu.wakes`, `io.*` or `ui.*` (on macOS: `cpu.idle_wakes`).
- A declared metric no reading gave is refused with why, never dropped from the table: `<m>: a script
target never gives it; instead, measure cpu.main or cpu.work` (`NotForTarget`), `… gives it on other
OSes, not on Windows` (`NotOnThisOs`), `no measured run sent the input it times` (`NoInput`), else
`NoMetric`. As the primary: the case FAILs in a check (exit 1), the series is refused at block 1 in
a comparison. A `cli()` reading the case did not declare is left out, never listed refused.
- An app with `window: false` (on the target or its profile; `window: true` on a native target wins):
frames, latency, scrolling, canvas, display and GPU use are refused `NotForTarget` (a primary of
them before it launches: pick `cpu.work`, `run.duration`, a memory metric or `phase.<name>`). Its
run cases need no window or unlocked session; a window it shows anyway refuses the block
(`VariantMismatch: … declares window: false, yet it showed a window on screen`).
- Two sides on one engine where one has nothing on screen (a headless page, a `window: false` app) and
the other shows its window: `VariantMismatch` before anything builds, in `bench` and `plan`, also for
two profiles of one target (`--base app:desk --head app:tray`). Run both off screen or both on it.
- `js.heap.retained` on a script host with no collector to call: that metric alone,
`EngineUnsupported`. macOS Bun `memory.app.private`: `EngineUnsupported` (a primary before the
file's other targets run).
- `WindowHidden` on a page: it stayed hidden 200 ms at a frame wait (a `window.open` popup hides
its opener on macOS): refused at once, never waited out. Close a popup the case opened before
such a wait, or measure the case without frame metrics.
- Exit 3 naming the step: macOS held the launch (Gatekeeper after a download or update): open the
app once from the Finder.
- `OverheadBudget`: strobe's own work disturbed timed runs (a `--check` only warns).
- `VariantMismatch`, `DisplayTooSmall`, `DeviceMismatch`, `InputMismatch`: the sides differ.
- `SharedRunner`: a CI runner other jobs share; no timing there. `--check`, counters against base, leak
runs and attribution work (In CI, above); timing needs a machine of its own.
- In comparisons, single-build blocks and leak runs, a series refused while it runs (`NoPhase`, for
example) ends alone: `refused` in the listing. In a check a run defect fails the case (1), but a
primary the target, engine or display cannot provide is refused. Only
the machine's refusals (`SessionLocked`, `Background`, `Thermal`, …) stop the bench. Every series
refused: exit 4.
- `!` lines warn next to a kept number; `i` lines inform. Read them before trusting a number:
`NotSteady`, `HighVariance`, `ScreenCh...
© anomalyco · MIT · anomalyco/strobe