К основному содержанию
скилл CX Codex

strobe — скилл для Claude Code: замеры производительности

Скилл strobe для Claude Code измеряет производительность приложений и объясняет результаты. Запускается, когда нужно сравнить версии, найти причину тормозов или написать бенчмарк.

MIT ★ 173 обновлён 2026-11-16

Установить в Claude Code · Через Skills CLI (Vercel) Все способы и платформы ↓
npx skills add anomalyco/strobe --skill strobe -a claude-code
Открыть на GitHub

Что делает strobe

strobe помогает создавать и запускать бенчмарки для веб-страниц, Electron- и нативных приложений, терминальных программ и процессов Bun или Node. Он сравнивает базовую и новую версии, выдаёт вердикт по сериям измерений и показывает подозрительные места с источниками и следующими командами для проверки.

Для старта strobe init создаёт конфигурацию и черновой бенчмарк на основе файлов проекта; затем его можно проверить через strobe bench <file> --head . --check. Для чистого сравнения нужно как минимум восемь пар измерений. Если результаты недостаточны для обоснованного вывода, strobe сообщает об этом отдельным вердиктом, а не трактует запуск как отсутствие разницы.

Какие задачи решает

  • сравнить производительность базовой и новой версии
  • найти участок, связанный с замедлением или ростом памяти
  • создать черновой бенчмарк из файлов проекта
  • проверить, что бенчмарк запускается и выдаёт метрики

Примеры запросов агенту

  • Сравни производительность изменений с origin/dev с помощью perf/smoke.bench.ts.
  • Разберись, почему эта страница стала медленнее и где растёт память.
  • Создай бенчмарк для приложения из этого проекта.

Как установить strobe

1Через Skills CLI (Vercel)

npx skills add anomalyco/strobe --skill strobe -a claude-code

2Вручную: положить файлы

~/.claude/skills/skill/SKILL.md

Папка скилла целиком; для одного проекта — .claude/skills/skill/

По шагам, с командами для каждой платформы: Как установить скилл в Claude Code: пять способов

Вопросы о strobe

Как установить strobe в Claude Code?

Выполни npx skills add anomalyco/strobe --skill strobe -a claude-code. Скилл устанавливается в ~/.claude/skills/skill/SKILL.md.

Что делает strobe?

Скилл измеряет производительность программ и сравнивает запуски, например базовую и новую версии. Он выдаёт вердикты и помогает исследовать возможные причины замедления или роста памяти.

Какие приложения strobe умеет измерять?

В описании перечислены веб-страницы, Electron-приложения, нативные приложения, терминальные программы, процессы Bun и Node, а также другие процессы.

Исходный текст

SKILL.md · MIT
---
name: strobe
description: Measure and explain performance of web pages, Electron apps, native apps, terminal programs, Bun or Node processes and any process. Use when the user asks whether a change is faster, why something is slow, janky or heavy, where memory goes, what the CPU or GPU is doing, or to write a benchmark.
---

# Strobe

strobe records a target on one timeline and answers with verdicts and handles (`rec_…`, `exp_…`) to
drill into. It refuses measurements it cannot defend: a refused run is an answer, never "no value".
The design and every rule behind this file: SPEC.md.

## Playbooks

### No bench yet: start one

1. `bun add -d @hona/strobe` pins strobe for the project (`bunx strobe …` then runs that version).
2. `strobe init` writes `strobe.json` (its `$schema` linked) and a runnable `perf/smoke.bench.ts` from
   the folder's package.json (`build`, `preview`, `start`, `dev`, `exports`, `module`, `types`, `main`,
   `bin`, an electron dependency), Cargo.toml binaries or index.html.
   `--target web|lib|server|native|electron|tty|cli` overrides the guess: web serves the build with
   `perf/serve.ts` (a cold load, an idle second, and a commented reload: a small page's reload is
   mostly strobe's own driver work, `OverheadBudget`), lib imports the package's entry from
   `perf/lib.ts` (a cold import per run; a commented `work` case calls an export with `ctx.call`),
   server hosts it in `perf/server.ts` (`PORT`),
   native builds and launches the Cargo app, cli runs the command to its exit. It never replaces a file
   without `--force`.
3. Run what it prints: `strobe config`, then `strobe bench perf/smoke.bench.ts --head . --check`
   (`--headless` with no display or a locked session). Then edit the cases to what users do.

### Did my change make it faster?

- Allow at least eight pairs for the clean-pair check. `strobe run --plan` previews a command comparison.
- On a busy desk use `cpu.work` or, for headless pages, `main.work`. Read any `Background` warning.

1. Write or find a bench file (below; `strobe init` writes a first one). Check it first:
   `strobe bench <file> --head . --check` (each case once, pass or fail with a screenshot).
2. `strobe plan <file> --base origin/dev --head .` says pairs and minutes per series (`time unknown`,
   never a guess, for a series no `--check`, `--pilot` or earlier run measured here).
3. `strobe bench <file> --base origin/dev --head .`: a verdict per series (case × target × params).
   - Read the ★ primary's verdict: `faster`/`slower` (`less`/`more`, `fewer`), `same (±2%)`,
     `unclear`, `underpowered` (more pairs or a quieter machine, not "no change"), `refused`.
   - Secondaries get words only after the primary decided (Holm-corrected); `·` is descriptive.
   - `ledger  3rd experiment on these variants; earlier: …`: reruns count. Paste it as it is.
   - `strobe query <exp> insights`: per series the verdict (or a check's status), ranked suspects
     with their source, the warnings that bear on it, and commands to run next as printed (`--pairs
     N`, `record --bench <file>#<case>` for a side nothing profiled). Any kind: check, `--blocks`, leak
     run. Head's hits read `(over its threshold on head only)`, `(… on base too)` or `(base not read
     for it)`; beside `same`, `unclear` or an A/A no mover is a suspect (two launches differ anyway).
4. Put the evidence in the PR (below).

- Refactor or dependency bump? `--noworse 2%` (or `noworse: "2%"` on the case;
  `--noworse 2%,cpu.wakes=5%` per metric): `no worse (≤2%)`, `worse (>2%)` or `unclear`. A bare exact
  counter is `0%` (`--noworse count.items`); a bare other metric exits 2 (`main.work needs a margin`).
- `strobe run --base "<old command>" --head "<new command>"`: the same A/B for two command lines.
  `--check --head "<cmd>"` (and `--base "<cmd>"`) runs each once: `cpu.work · run.duration ·
  memory.peak`. With `--check` every `--noworse` exits 2 (a command gives no exact counter): drop
  `--check` and use `--noworse cpu.work=2%`.
- `ShellLine`: `&&`, `||`, pipes and redirects run through the platform shell. Its startup counts.
  Plain external commands use PATH; use `./tool` or `.\tool.exe` to measure each variant's build.
- `--aa --head .` checks the method on this machine (same build both sides).
- Long benches keep running in the daemon: `strobe query exp_… progress`. Ctrl-C, SIGTERM or
  `strobe bench --stop <exp>` stop after the block (exit 130, what finished is kept);
  `strobe bench --resume <exp>` goes on.

### How a run ends (the exit code says whether strobe measured, never who won)

- A timed comparison can exit 0 on `slower` or `worse`: read `series[].primary.verdict_kind`.
  Every non-gap primary refused: 4. `--require-verdict`: 5 if undecided, 4 if any primary is refused
  or any journey is a parity gap. `--require-parity` alone makes gaps exit 4, not undecided results.
- A check is no verdict: each case once (each side once with `--base`), proving the bench runs and each
  declared metric gives a value or a stated refusal.
  - `--check`: 0 when every case passes; 1 when one `FAIL`s (its `fix:` line says what to change); a
    preflight refusal is listed beside passing cases; all non-gap cases refused up front: 4.
    A run defect fails its case (1); a primary the target, engine or display cannot measure is
    listed as refused. A refused secondary alone does not fail a case.
  - `--check --base <ref>`: base's line under head's; the summary ends `base and head each checked;
    nothing compared: the same command without --check times head against base` (exit 0 if both ran).
    A base build error stops the check before the case runs (1). A base run error fails the case (1):
    `1 failed on base`; head's values if it passed, head's error if it failed too. Leave head-only cases
    out with `--filter` until the change merges.
  - `--check --base <ref> --noworse count.x`: each counter row ends `no worse` (passes), `worse`,
    `unstable`, `missing` or `not_compared` (each fails the case, exit 1). If cases ran and none failed
    but no counter compared, exit **4**, with `passed: false` in JSON. If both sides refused every
    counter, read the refused lines. For `Uncounted`, check the spelling against `ctx.count`; if the run
    counts it only on some paths, add `metrics: ["count.x"]` on that case so 0 → 0 compares.
    If every case is a parity gap, no counter compared: exit 4. `--require-parity` requires every journey.
  - `--check --base <target> --head <target>`: shared journeys get a parity matrix, not a counter
    comparison. Gaps alone exit 0 (even if no case ran), or 4 with `--require-parity`; a mismatch or
    case failure exits 1. With two bare targets, `--noworse` exits 2: it needs two builds.
- `--blocks N`: one build's values, no verdict. `--probes memory`: a leak run exits 0 even for `leaks`
  (`worse`); 5 if undecided with `--require-verdict`. Both modes exit 4 if every series is refused.
- `--pixels-only`: 1 if snapshots differ or one side lacks one, otherwise 0 on completion. `--bisect`:
  `found`, `no change`, `unclear` exit 0; `--require-verdict` makes an `unclear` search exit 5.
- Stops and caps: 130 stopped; 124 for an exhausted `--unlock-wait` or a non-comparison timeout.
  A timed comparison hitting `--max-time` keeps its finished pairs and normal exit rules (usually
  0, or 5 if undecided with `--require-verdict`). The slot wait for display fit before the first
  series can still exit 124. Resume stopped work with `bench --resume <exp>` if state was saved.

### Why is it slow?

- `bench … --why always` (an attribution block per series), `bench … --check --why always` (one build),
  or `strobe record <target> --probes attribution` / `strobe record --bench <file>#<case>`. Then:
  - `query <rec> hotspots` (CPU by function, full stacks, source-mapped file:line:column;
    `--by file|package`; `--sort total` for callers; native: `--thread <name>`, `--pid <pid|exe>`,
    `--phase <name>`).
  - `query <rec> layout` (forced layouts by their call site), `style` (what invalidated style:
    attribution blocks trace it for a series declaring `main.style` or `style.count`, or with
    `--style`), `alloc` (who allocates), `gc`, `summary` (per run: style, layout, paint ms), `insights`.
  - A page served from your own folder (`record https://localhost:… --served <dir>`): its source-mapped
    frames resolve to files on disk.
  - What changed: `query <exp> hotspots` (each series' base..head) or `query <recA>..<recB> hotspots`;
    `~` marks a change inside its sampling interval (one launch a side: not a verdict).
- `strobe open <rec>` (or `<exp>/head`): the whole recording in a local Perfetto UI. For seeing;
  numbers come from strobe's views. The "strobe itself" track is strobe's own cost: check it first.
- A thread waiting (a lock, IO, a timer) and who wakes it: `record pid:<n> --probes waits --for 10s`,
  then `query <rec> waits --thread <name>` (Linux needs CAP_PERFMON, Windows the deep helper;
  macOS refuses `NoWaits`).
- A slow page frame in the renderer or the GPU process: `frames.renderer.p95` against `frames.viz.p95`,
  and `query <rec> frames` (each worst frame's stages).
- Inputs with no latency sample (`busy`), or one that looks wrong: `query <rec> inputs [--case <c>]`
  (the first 100; `--top N`, `--json` every one):
  each input's run and time, what proved its change, the frames around it, its latency or why it
  was busy (`mid_drawing`, `behind`, `ambiguous_press`, `multi_frame_pixel`) and how much later it
  had to come. Give the case that wait before such inputs.
- A profiled launch is slower than a timed one: compare totals only from the timed pairs.

### Where is the memory? Does it leak?

- `query <rec> memory` (per process; `--by layer|allocator` for Chromium's accounting). A native
  app: what holds it (host heap, GPU memory mapped in, unmapped commit, shared, files) beside
  `gpu.vram`. On Windows `memory.app.private` counts VRAM a driver maps in: quote `memory.app.host`.
  A phone's page: processes from Chromium's dump (no OS memory); listeners are the page's own.
- `query <rec> heap`: classes by retained size, detached DOM, retainer paths; `query <a>..<b> heap`.
- A case's leak: `bench <file> --filter <case> --head . --probes memory --repeat 10`: growth per run
  with an interval, and `query <rec> leaks` for what from the first half is still alive.
- An app over time: `record <target> --probes memory --for 60s` while it does the suspect thing.
- A script target: `js.heap.retained` (what a run keeps after a collection, buffers counted once). On
  a Node host it adds ArrayBuffer contents in releases after 0.31.0 (which left them out): compare
  across that upgrade as two histories (`track` starts a new series, `metric revision 2`; Bun series
  keep theirs). Every Node-hosted script series, any metric, started a new history once in 0.32.0:
  its identity records `runtime node` (`strobe history` shows it). It does not see add-ons' native
  memory, mapped files, or on Node WebAssembly memory and strings outside the heap: read
  `memory.app.private` for those. On macOS a Bun host's
  `memory.app.private` is refused (`EngineUnsupported`): Bun's allocator hides memory from the OS.
- Sizes are binary: KB, MB and GB are 2^10, 2^20 and 2^30 bytes.

### A native app, or any process from outside

- `record app:<id> --for 10s` (a strobe.json profile) or `record exe:<path> --native`: CPU, thread
  wakes, IO, GPU, memory, the UI thread and what reached the screen. `--windowless` for a tray app
  (`look … --windowless` first: does it stay up, what does it print?).
  `record pid:<n>` or `proc:<name>` for a process strobe did not start.
- `query <rec> threads`: wakes/s and CPU per thread, the module each started in, whose (app, driver,
  os, `unknown (?)`); `--timeline` per second. `query <rec> insights` rules include dominant-thread,
  off-cpu-bound, style-layout-dominant, wakes-while-idle (idle blocks too), hot-function (a named
  function of the target's over 25% of busy samples, one hit each; a native symbol without line
  info counts), http-stalls (measured runs only).
- Native hotspots of a tree pool every process: read `by process`, then `--pid <pid|exe>`.
  A native attribution recording (`record --bench`, `--why`) keeps frames too: `query <rec> inputs`.
  `! Coverage`: samples cover under 80% of the CPU (read shares, not totals); it names what the
  sampler counted of the rest (`late_ms` before a thread was opened, `forgone_ms`, unopened threads,
  lost events). A launched app's children are sampled from their start. Two resolved shares
  (leaf, all frames): a low all-frame share means the app's own binary has no symbols.
  macOS samples are untimed snapshots: counts, never ms.
- `record pid:… --probes deep|waits` prints `cost`; `! CaptureCost`: the capture slowed a
  switch-heavy process, so its stacks describe that slower process.
- Benching it: `native("my-app")` in a bench file (below).

### Two implementations of one app (a port, a rewrite)

- One bench file, one target per implementation, the same cases (journeys):
  `bench <file> --base original --head rewrite` (`rewrite=git:main`, `original=exe:/path`;
  `target:<name>` when a folder has that name). Each target's `name` is what reports call it.
- Pick a primary both sides measure the same way: `cpu.work`, `run.duration`, `launch.ready`,
  `memory.app.private`. `main.work` or `ui.cpu` across engines exits 2 (`NotComparable`).
- Give each target a `ui` adapter for what differs (`ui: { row: ".row", open: ({ page }) => … }`);
  journeys say `@row`, `await ctx.ui.open()`. A journey one side cannot run is a parity gap
  (`unsupported: { rewrite: "why" }`, or an `@name` its `ui` lacks): listed, never run.
- `expect: { same: ["@topRow"] }`: both sides must read it alike after setup, each run and reset, or
  `NotEquivalent`. Both talk to one `fixture("<server> --port {port}")` (`{fixture.url}` in args/env).
- A page against a native app (or another engine) gets OS input: run it headed, window in front;
  `page.act` on one side and `click` on the other is `InputMismatch`.
- `--check --base a --head b` prints the parity matrix; `--why always` profiles each side once.
- Over time: `track <file> --base original=exe:/path --head rewrite=git:main --schedule commits`.

### Which commit moved it? Did anything regress?

- `bench <file> --filter <case> --bisect <good>..<bad>`: one series; each step is a full experiment;
  the culprit is benched against its own parent.
- `track <file> --range <old>..<branch> --every 3` (or `--schedule` on a branch), then
  `history <file>` or `query changes`. Only `confirmed` changes are findings.

### In CI: every pull request

- GitHub Actions: `uses: anomalyco/[email protected]` with `bench: perf/app.bench.ts` (inputs: action.yml).
  The default `mode: check` runs each case once, headless, and fails the job on a failed case.
  `noworse: net.requests=0,count.items=0` also runs the base commit and fails a case whose exact counter
  is over base's or varies between runs. A case base cannot run fails too, and a check whose cases
  compared none of the named counters exits 4 and fails the job: declare each counter in the metrics of
  the cases that count it. The step summary is `report experiment <exp> --format summary`.
- A shared runner (GitHub-hosted, `CI_SHARED_ENVIRONMENT=true`, or a CI job in a VM) refuses timing:
  `SharedRunner`, exit 4, for a timed bench, `--blocks`, `track`, `--bisect`, `plan --pilot` and a timed
  `record`. `status` says so. A dedicated CI VM says `RUNNER_ENVIRONMENT=self-hosted` or
  `CI_SHARED_ENVIRONMENT=false` (neither overrides GitHub-hosted). There: `--check` (`--base` with
  counter `--noworse`), leak runs (`--probes memory`), `--pixels-only`, `record --probes attribution`.
- Timing verdicts in CI: `mode: compare` on a self-hosted runner that runs one job at a time; the block
  goes in a same-repository PR's body (never a fork's). Never `pull_request_target`.
- The action installs nothing into the runner user's profile (its own strobe, no PATH entry or skill):
  it runs `up --skip self,skill,daemon,mcp,deep,calibrate`, which only fetches the engine.
- The step summary, the job's log and the artifacts never name the machine's other processes, other
  apps' windows or its user: the uploaded result.json is a scrubbed copy and its HTML report is drawn
  from that copy alone (no recordings, so no hotspots; the runner keeps them, `report-path`). The PR
  block, summary, card and badge name no other process, window title or person's folder either:
  every string of the result is scrubbed before they read it (a covering window by its process id, a
  plain folder variant as `<base folder>`/`<head folder>`; SPEC §11.2).

### No display, a locked session, a server

- `--headless` runs page cases (check or timed) with no display: frames, latency, scroll, canvas and
  GPU use refuse `Headless`; CPU, main-thread work, counters, phases and memory stay. Headless,
  what frames cost is `main.paint` beside `main.style` and `main.layout`.
- A headless page's lean block (and a `window: false` app's) takes no trace unless a case declares
  `phase.*`, `span.*`, `layout.shift`, `main.gc`, `gpu.compile` or `main.paint`: its `meta.json` says
  `"traced": false`. `summary` works. `frames` refuses `Headless` for the page or `NotForTarget` for
  the windowless app (4). `layout`, `style` and `gc` need attribution; `threads` lacks thread counters;
  `query a..b threads` says `this block took no trace` (2). With a trace, `frames` reads Chromium's
  pipeline, but the bench still refuses off-screen frame metrics. For trace data use a timed
  `bench --trace full` or a headed run. For a headless check, query the `--why always` attribution
  recording: `--trace full` alone does not change the check block's lean probes.
- A locked session runs what needs no presenting window: scripts, `cli()`, `tty()`, leak runs,
  headless pages, `window: false` native cases measuring `cpu.*`, `memory.*`, `gpu.vram`, `io.*`,
  `count.*`, `phase.*`, and any native case under `--check`. Compare them only with locked runs.
- The rest waits for an unlock (a line a minute). `--unlock-wait 2h` caps it: then those series are
  not run (exit 124, resumable) and the windowless ones still run.
- Locked, a native app's tree, `page.act` and keys work; clicks, moves and wheels refuse
  `SessionLocked`. On Windows its pixels still read (image anchors, `--pixels`).
- Linux without `DISPLAY`: native apps run on strobe's headless Wayland compositor (install sway,
  dbus-daemon and at-spi2-core); frames and latency refuse `Headless`.

### A busy server: which thread held it, and could a client get an answer?

- Host the server in a `script()` target. In `setup`, `await ctx.httpProbe({ url, authorization })`:
  strobe probes it every `cadenceMs` (100) from its own threads on fixed deadlines, `timeoutMs` 1000.
  The whole runnable example (`server.ts` too): `node_modules/@hona/strobe/README.md`.

```ts
import { bench, script } from "@hona/strobe"

export default bench({
  targets: { server: script("./server.ts") }, // exports start(tokenHash) and stall(ms)
  async setup(ctx) {
    const token = crypto.randomUUID() // the server gets its hash only
    const hash = new Bun.CryptoHasher("sha256").update(`Bearer ${token}`).digest("hex")
    const url = await ctx.call<string>("start", hash)
    const expect = { jsonPath: { path: "$.status", equals: "ok" } }
    await ctx.httpProbe({ url, authorization: `Bearer ${token}`, expect }) // every 100 ms, new connections
  },
  cases: {
    "slow handler": {
      primary: "http.latency.p95", // 4 stalls 100 ms apart: ~26 probes a run, enough for a p95
      metrics: ["cpu.main", "cpu.main.busy", "http.timeouts"],
      run: async (ctx) => {
        for (let i = 0; i < 4; i++) {
          await ctx.call("stall", 550) // back to back, a probe can wait out two stalls
          await Bun.sleep(100)
        }
      },
    },
  },
})
```

- `record --bench <file>#<case>` (or `--check`, `--why always`): stacks, the threads' timeline and the
  probe on one recording. `query <rec> http`: idle against the runs, timeouts, errors and the stalled
  cohorts: one stall (`answered none for X`) or stalls back to back (`spanned N stalls …`), timeouts
  wherever they fell; a quantile the block refuses reads `—` with why. `strobe open <rec>` draws the
  probes beside the runs and threads, and `report` an HTTP probe card.
- To compare it, declare `http.latency.p95` (or `http.timeouts`, `http.errors`): the probe then runs
  in probe blocks of its own. A clean timing block never probes.
- A timeout ranks above every answer: a quantile on one refuses `RunIncomplete` (raise `timeoutMs`).
  A 500 or a failed `expect` refuses the latencies. `http.missed_sends` is strobe's lag, not the server's.
- `authorization` stays in strobe's memory: never in a file, report, trace or answer. Loopback only.
- `cpu.main` is the main thread alone beside `cpu.work` (the tree): `cpu.main.busy` near 100% is a
  saturated event loop; a worker or a child process shows in `cpu.work` only. A script's (or a
  windowless native app's) check line shows both even when the case does not declare them.
- A script's `query <rec> hotspots` is its measured runs; `--source host` the whole host process
  (start, setup, warmup, resets), its `by lifecycle` lines each part's wall time against the JavaScript
  sampled in it; `--source startup` the host's start and the target's static imports, `--source setup`
  the bench file's setup (a server's dynamic imports); `--phase <name>` one phase.
  - Slow startup with few rows: `N modules evaluated` (each module is a stack of its own: `--by file`
    sums them). Lifecycle shares are of wall time: unsampled wall is mostly waiting, and on Bun
    native work (transpiling, module loading) too, which `--cpu-prof` never samples.
  - `! Missed`: the main thread used CPU (`cpu.main`) the profile has no sample of, so rows
    undercount (Node labels short timer callbacks `(idle)`; Bun never samples its event loop's own
    work). `! Waited`: rows include time the thread waited for a CPU (samples weigh wall time).
  - `query <rec> threads --timeline`: which thread was busy when, runs marked.

### Phones

- Emulated: `web({ emulate: "mobile" })` is a phone (390×844 @3x, touch, headless): `click` is a tap,
  `drag` a finger, `scroll` swipes (Chromium only); `hover`, `move`, `wheel` and mouse buttons refuse
  `DeviceMismatch`. Headless, so `frames.*` and `interaction.*` refuse `Headless`: measure
  `main.work`, `cpu.*` or `run.duration` there, and frames and latency on a real phone (`--mobile`).
  `"touch"` is a desktop with a touch screen: the mouse stays its primary pointer.
- A real Android phone: `--mobile` (or `--mobile=<serial>`) runs the file's `emulate: "mobile"`
  targets in Chrome on the phone over adb; `look --mobile`, `record --mobile` too.
  - Give strobe a channel once: install Chrome Canary (or Dev, Beta), open it once, then
    `strobe up --mobile=<serial> --channel canary`. strobe never drives the phone's own Chrome.
  - An emulator runs `--check`, `look`, attribution and leak runs (`--probes memory`) only (`Emulator`);
    timed runs refuse Wi-Fi adb, Battery Saver, under 20% battery, and wait up to 3 min for a hot phone.
  - Leak runs and `record --mobile --probes memory` work on a phone (the phone's heap and snapshots;
    no `memory.app.private`: read `js.heap.used`, its growth and the leak suspects, or a desktop
    browser's memory recording). Loopback addresses `prepare` returns
    (an API, a WebSocket) are reversed to the phone, like `serve`.
  - `cpu.*`, `memory.*`, `gpu.*` refuse `NotForTarget` (this computer's processes): measure
    `main.work`, `frames.*`, `interaction.*`, `run.duration`. A lost phone ends the bench (exit 255).
- An iPhone on a cable: `look wir:` and `record wir:` read an open Safari tab or WKWebView over Web
  Inspector, read-only (`hotspots`, `heap`, `memory`, `log --source console|device|crash`,
  `network`); `strobe status` lists the phones and their pages.

### Writing the PR: every performance claim comes from strobe

1. **Discovery**: the strobe command and recording id that pointed at the problem, and what it showed.
2. **The change**: what the code does differently.
3. **Results**: `strobe report experiment <exp> --format pr --update-pr <n>` puts strobe's block in
   PR n's body, led by its result card (or paste the `--format pr` output).
   - The block is delimited (`<!-- strobe:report <exp> v1 begin sha256:… -->` … "end of generated
     report"). Never edit inside it: no retyped, rounded or dropped numbers, no rewritten verdicts;
     strobe refuses to update a hand-edited block. Write your reading above or below it, and say
     there when a metric was refused or `unclear`. One block per experiment.
   - Never put a card or badge in a PR without the block. `--format card|badge` is for elsewhere.
   - The sides read "Before" and "This PR" only when head is the working tree or `HEAD`, and with
     `--update-pr` at the PR's head commit; an A/B of other builds reads by commit, an A/A as `Base (A/A)`
     and `Head (A/A)`. Name them with `--labels <base>,<head>` (`--labels v1.4,v1.5`).
4. **The bench**: the case's code, or a link to it. A new case ships in the PR of the first change
   it measures.

Never put hand-timed numbers, single recordings or `--check` values in a results table: label them
noisy discovery evidence, or leave them out.

### When strobe refuses

- Exit 4 names the validator and the fix: fix the setup, never work around it. Common ones:
  - `WindowHidden`, `SessionLocked`: the window must show; wait or unlock.
  - `Background`: other work on the target's CPUs (named); a timed bench waits it out before refusing.
  - `Throttled`: the speed check stayed slow on quiet CPUs, a Linux CPU quota throttled the runs,
    or a headless page ran with the display off.
  - A `--check` never waits on the speed check: a slow one is a `! Background` or `! Throttled` line and
    the case runs at once. Its values are then a loaded machine's.
  - `RunIncomplete` on `cpu.renderer`/`cpu.gpu`/`cpu.browser` names why: a process of that role ended
    in the run, or ended processes (their CPU pooled) used over 1% of it. `cpu.work` keeps
    their CPU: compare that.
  - `RunIncomplete`: a run threw, timed out, its `until` never came, or the app exited. "The host
    wrote none": a script host's profile never came; it says how the host ended and ends `script.log`.
  - `ResetIncomplete`: a run started from another state than run 1 (`the reset did not restore …`).
  - `TailTooShort`: a percentile lacks its minimum n (`interaction.p95` needs 20 samples a block).
  - `AmbiguousInteraction`: over half the inputs came while the target was drawing.
  - `NotOnThisOs`, `NotForTarget`, `EngineUnsupported`, `Headless`, `NoCounter`: this OS, target,
    engine or mode cannot give that metric; pick another primary (the message names one). A page
    (or an app over DevTools) has no `cpu.wakes`, `io.*` or `ui.*` (on macOS: `cpu.idle_wakes`).
  - A declared metric no reading gave is refused with why, never dropped from the table: `<m>: a script
    target never gives it; instead, measure cpu.main or cpu.work` (`NotForTarget`), `… gives it on other
    OSes, not on Windows` (`NotOnThisOs`), `no measured run sent the input it times` (`NoInput`), else
    `NoMetric`. As the primary: the case FAILs in a check (exit 1), the series is refused at block 1 in
    a comparison. A `cli()` reading the case did not declare is left out, never listed refused.
  - An app with `window: false` (on the target or its profile; `window: true` on a native target wins):
    frames, latency, scrolling, canvas, display and GPU use are refused `NotForTarget` (a primary of
    them before it launches: pick `cpu.work`, `run.duration`, a memory metric or `phase.<name>`). Its
    run cases need no window or unlocked session; a window it shows anyway refuses the block
    (`VariantMismatch: … declares window: false, yet it showed a window on screen`).
  - Two sides on one engine where one has nothing on screen (a headless page, a `window: false` app) and
    the other shows its window: `VariantMismatch` before anything builds, in `bench` and `plan`, also for
    two profiles of one target (`--base app:desk --head app:tray`). Run both off screen or both on it.
  - `js.heap.retained` on a script host with no collector to call: that metric alone,
    `EngineUnsupported`. macOS Bun `memory.app.private`: `EngineUnsupported` (a primary before the
    file's other targets run).
  - `WindowHidden` on a page: it stayed hidden 200 ms at a frame wait (a `window.open` popup hides
    its opener on macOS): refused at once, never waited out. Close a popup the case opened before
    such a wait, or measure the case without frame metrics.
  - Exit 3 naming the step: macOS held the launch (Gatekeeper after a download or update): open the
    app once from the Finder.
  - `OverheadBudget`: strobe's own work disturbed timed runs (a `--check` only warns).
  - `VariantMismatch`, `DisplayTooSmall`, `DeviceMismatch`, `InputMismatch`: the sides differ.
  - `SharedRunner`: a CI runner other jobs share; no timing there. `--check`, counters against base, leak
    runs and attribution work (In CI, above); timing needs a machine of its own.
- In comparisons, single-build blocks and leak runs, a series refused while it runs (`NoPhase`, for
  example) ends alone: `refused` in the listing. In a check a run defect fails the case (1), but a
  primary the target, engine or display cannot provide is refused. Only
  the machine's refusals (`SessionLocked`, `Background`, `Thermal`, …) stop the bench. Every series
  refused: exit 4.
- `!` lines warn next to a kept number; `i` lines inform. Read them before trusting a number:
  `NotSteady`, `HighVariance`, `ScreenCh...

© anomalyco · MIT · anomalyco/strobe

Похожие скиллы

Все скиллы по теме «Тесты и QA» →
bun-runtime affaan-m Скилл bun-runtime для Claude Code помогает выбрать Bun или Node, перейти с Node и настроить Bun для проекта или Vercel. Он подключается к задачам с Bun-скриптами, тестами и конфигурацией. CX Codex +1 6 500установок bun-development sickn33 Скилл bun-development помогает вести JavaScript и TypeScript-разработку на Bun: создавать проекты, переносить код с Node.js и разбирать проблемы рантайма. CX Codex +1 2 900установок electron-desktop travisjneuman Скилл electron-desktop для создания кросс-платформенных десктопных приложений на Electron под Windows, macOS и Linux. CX Codex +1 236установок principle-explain-the-number backnotprop Скилл principle-explain-the-number для Claude Code разбирает, что именно измеряет число и чем оно ограничено. Применяется к ускорению, регрессии, пропускной способности, задержке и eval. CX Codex +1 9 200установок benchmark-checklist backnotprop Скилл benchmark-checklist задаёт агенту список вопросов, которые нужно закрыть доказательствами из прогона, прежде чем докладывать об ускорении или регрессии. CX Codex +1 9 100установок benchmark affaan-m Скилл benchmark измеряет скорость страниц, API и сборки, а затем сравнивает показатели до и после изменений. Пригодится перед релизом или когда страницу считают медленной. CX Codex +1 6 200установок

Спонсоры проекта

Компании, которые помогают держать аналитику открытой