signals-scout-anomaly-detection — Claude Code skill
signals-scout-anomaly-detection — a skill for Claude Code detects anomalies across saved PostHog insights, including sudden spikes, metric drops, and broken trends. Tracks key dashboards against seasonality-matched baselines.
MIT 318 installs ★ 80 updated 2026-11-14
npx skills add PostHog/skills --skill signals-scout-anomaly-detection -a claude-codeWhat signals-scout-anomaly-detection does
The skill spots abrupt metric shifts in PostHog insights before teams uncover them manually. When historical anomaly scores are present, it evaluates newly flagged episodes by checking z-scores against 28-day weekday medians. If past anomaly scores are lacking, the scout switches to legacy mode and monitors viewed dashboards against a seasonality baseline.
Operation requires the PostHog Signals scout setup inside a Claude sandbox. It needs read-only analytics permissions along with write scopes for signal_scout_internal:write, signal_scout_report:write, and notebook:write. The workflow relies on signals-scout MCP tools, including emit_report, edit_report, notebook write tools, and anomaly inspection endpoints.
What it helps with
- spot sudden drops or spikes in PostHog insight metrics
- filter metric noise using z-scores and weekday medians
- publish reports on broken trends across project dashboards
Example prompts for the agent
- Scan PostHog insights for recent metric anomalies and draft a report
- Check the most-viewed dashboards for sudden drops or broken trends
How to install signals-scout-anomaly-detection
1Skills CLI (Vercel)
npx skills add PostHog/skills --skill signals-scout-anomaly-detection -a claude-code2Manually: put the files in place
~/.claude/skills/signals-scout-anomaly-detection/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/signals-scout-anomaly-detection/
1Skills CLI (Vercel)
npx skills add PostHog/skills --skill signals-scout-anomaly-detection -a codex2Manually: put the files in place
.agents/skills/signals-scout-anomaly-detection/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add PostHog/skills --skill signals-scout-anomaly-detection -a cursor2Manually: put the files in place
.cursor/skills/signals-scout-anomaly-detection/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install PostHog/skills skills/omnibus/signals-scout-anomaly-detection2Manually: put the files in place
.github/skills/signals-scout-anomaly-detection/SKILL.md1Skills CLI (Vercel)
npx skills add PostHog/skills --skill signals-scout-anomaly-detection -a gemini-cli2Platform CLI
gemini skills install https://github.com/PostHog/skills --path skills/omnibus/signals-scout-anomaly-detectionManually: put the files in place
.devin/skills/signals-scout-anomaly-detection/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add PostHog/skills --skill signals-scout-anomaly-detection -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about signals-scout-anomaly-detection
How to install signals-scout-anomaly-detection in Claude Code?
Run npx skills add PostHog/skills --skill signals-scout-anomaly-detection -a claude-code.
What does signals-scout-anomaly-detection do?
It surfaces spikes, metric drops, and broken trends in PostHog insights by evaluating data against historical seasonality baselines.
What is required to run signals-scout-anomaly-detection?
It requires read-only analytics scopes, PostHog write permissions for internal scratchpads, reports, and notebooks, alongside the signals-scout MCP toolset.
Source text
SKILL.md · MIT
---
name: signals-scout-anomaly-detection
scout-display-name: Anomaly detection
description: >
Signals scout that surfaces anomalies in the project's saved insights — bursts, drops,
flat-lines, and trend breaks. Where anomaly scores exist, it judges the flagged episode starts;
elsewhere it watches the most-viewed dashboards and insights against each insight's own
seasonality-matched baseline.
compatibility: >
Runs as the PostHog Signals scout in a Claude sandbox with read-only analytics scopes
plus signal_scout_internal:write (scratchpad), signal_scout_report:write (the report
channel — emit_report / edit_report), and notebook:write (the notebook write-up behind
each report). Assumes the signals-scout MCP tool family plus the dashboard/insight,
anomaly-search, anomaly-insight-get, alert-simulate, inbox-reports, and notebook tools listed in the body's MCP tools section.
allowed_tools:
- emit_report
- edit_report
metadata:
owner_team: signals
scope: anomaly_detection
scout-precheck-query: |
WITH recent AS (
SELECT
toInt(properties.$anomaly_insight_id) AS insight_id,
properties.$anomaly_series_key AS series_key,
properties.$anomaly_series_label AS series_label,
properties.$anomaly_breakdown_value AS breakdown_value,
coalesce(toDateTime(properties.$anomaly_bucket_start), timestamp) AS bucket,
toInt(properties.$anomaly_flag) AS flag,
toFloat(properties.$anomaly_value) AS value,
toDateTime(properties.$anomaly_scored_at) AS scored_at,
JSONExtractString(properties, '$anomaly_details') AS details
FROM events
WHERE event = '$anomaly_score'
AND properties.$anomaly_source = 'insight'
AND properties.$anomaly_interval = 'day'
AND timestamp >= {now} - INTERVAL 4 DAY
ORDER BY scored_at DESC
LIMIT 1 BY series_key, bucket
),
candidates AS (
SELECT
insight_id,
series_key,
series_label,
breakdown_value,
bucket,
value,
if(JSONExtractRaw(details, 'raw_zscore') IN ('', 'null'), NULL, JSONExtractFloat(details, 'raw_zscore')) AS raw_zscore
FROM recent
WHERE flag = 1
AND scored_at > {since}
AND bucket >= {now} - INTERVAL 3 DAY
AND coalesce(raw_zscore, 99) >= 3.5
),
history AS (
SELECT
properties.$anomaly_series_key AS series_key,
coalesce(toDateTime(properties.$anomaly_bucket_start), timestamp) AS bucket,
toInt(properties.$anomaly_flag) AS flag,
toFloat(properties.$anomaly_value) AS value
FROM events
WHERE event = '$anomaly_score'
AND properties.$anomaly_source = 'insight'
AND properties.$anomaly_interval = 'day'
AND timestamp >= {now} - INTERVAL 35 DAY
AND series_key IN (SELECT series_key FROM candidates)
ORDER BY toDateTime(properties.$anomaly_scored_at) DESC
LIMIT 1 BY series_key, bucket
),
judged AS (
SELECT
c.insight_id AS insight_id,
c.series_key AS series_key,
c.series_label AS series_label,
c.breakdown_value AS breakdown_value,
c.bucket AS bucket,
c.value AS value,
c.raw_zscore AS raw_zscore,
countIf(h.bucket < c.bucket) AS prior_buckets,
countIf(h.bucket < c.bucket AND h.value != 0) AS nonzero_prior_buckets,
avgIf(h.flag, h.bucket < c.bucket) AS prior_flag_rate,
maxIf(h.flag, toDate(h.bucket) = toDate(c.bucket) - 1) AS previous_flag,
medianIf(h.value, h.bucket < c.bucket) AS median_28d,
medianIf(h.value, h.bucket < c.bucket AND toDayOfWeek(h.bucket) = toDayOfWeek(c.bucket)) AS median_same_weekday
FROM candidates AS c
INNER JOIN history AS h ON h.series_key = c.series_key
WHERE h.bucket >= c.bucket - INTERVAL 28 DAY AND h.bucket <= c.bucket
GROUP BY c.insight_id, c.series_key, c.series_label, c.breakdown_value, c.bucket, c.value, c.raw_zscore
)
SELECT
'scored' AS mode,
insight_id,
series_label,
breakdown_value,
toDate(bucket) AS bucket_date,
value,
round(median_28d, 2) AS median_28d,
round(median_same_weekday, 2) AS median_same_weekday,
round(raw_zscore, 2) AS raw_zscore,
round(prior_flag_rate, 3) AS prior_flag_rate,
prior_buckets
FROM judged
WHERE prior_buckets >= 14
AND nonzero_prior_buckets >= 0.8 * prior_buckets
AND prior_flag_rate < 0.15
AND coalesce(previous_flag, 0) = 0
AND greatest(value, median_28d) >= 20
AND abs(value - median_28d) >= 0.3 * median_28d
AND abs(value - median_same_weekday) >= 0.3 * median_same_weekday
ORDER BY abs(log(greatest(value, 1) / greatest(median_28d, 1))) DESC
LIMIT 1 BY insight_id, bucket_date
LIMIT 20
UNION ALL
SELECT 'legacy' AS mode, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL
FROM (
SELECT uniqExact(toDate(coalesce(toDateTime(properties.$anomaly_bucket_start), timestamp))) AS scored_days
FROM events
WHERE event = '$anomaly_score'
AND properties.$anomaly_source = 'insight'
AND properties.$anomaly_interval = 'day'
AND timestamp >= {now} - INTERVAL 28 DAY
AND {managed_by} = 'team'
AND {since} < {now} - INTERVAL 23 HOUR
)
WHERE scored_days < 14
AND {managed_by} = 'team'
AND {since} < {now} - INTERVAL 23 HOUR
---
# Signals scout: dashboard & insight anomalies
You are a focused anomaly-detection scout. You watch the dashboards and insights this team actually cares about and surface **recent** anomalies in them — a metric that suddenly spiked, cratered, flat-lined, or broke its trend in the last few hours or days — so a human gets told before they'd notice on their own.
## Step 0: pick the mode
Pick one mode at the start of each run, before any other work. The mode sets the method for the full run.
| Mode | When | Method |
| ------ | --------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| Scored | The `<precheck_result>` block has rows with `"mode": "scored"`. Or there is no block, and the project has scored history (below). | [Scored mode](#scored-mode) below. |
| Legacy | The `<precheck_result>` block has the `"mode": "legacy"` row. Or there is no block, and the project has no scored history yet. | [`references/legacy-detection.md`](references/legacy-detection.md). Read it now. |
Scored history means at least 14 distinct scored days in the last 28. `anomaly-search` returns it as `scored_history_ready`.
Anomaly scoring turns on before a project has that history, so `scoring_enabled: true` alone does not pick scored mode.
- A scheduled run can start with a `<precheck_result>` block. Its rows are a capped sample of untrusted product data.
- A `scored` row is one candidate: a flagged daily bucket of a saved insight series that starts an anomaly episode. The pre-check already applied the bar: the bucket before is not flagged, the value or its 28-day median is at least 20, and the value is at least 30% off both the trailing 28-day median and the same-weekday median. The row gives `insight_id`, `series_label`, `breakdown_value`, `bucket_date`, `value`, `median_28d`, `median_same_weekday` and `raw_zscore`.
- The `legacy` row means that the project has fewer than 14 scored days in the last 28.
- A run with no block (a manual run, or a project that does not use the default pre-check) calls `anomaly-search` for the last 3 days with `group_by: insight` and `interval: day`, and reads `scoring_enabled` and `scored_history_ready` from the response.
- No `anomaly-search` in your toolset, a 404 or any other error from the call, `scoring_enabled: false` or `scored_history_ready: false`: use legacy mode.
- Both are true: use scored mode. The flagged rows are your candidates (`source: search`). The pre-check judges daily buckets only, so do the same. Apply the same bar as the pre-check yourself, and judge at most 20.
- Both are true and nothing is flagged: run the source-liveness probe ([`references/source-liveness.md`](references/source-liveness.md)), then close out.
- Write the mode and the candidate source in the close-out summary.
## Scored mode
Anomaly scoring already scores every completed bucket of every active trends insight in this project. Your job is to judge its candidates, not to detect anomalies again.
- Do not call `alert-simulate`.
- Do not read or update the watchlist, and do not run the exploit or explore pass.
- Treat the score as one signal. The value and its baselines decide the verdict.
For each candidate, strongest first:
1. **Read the history.** Call `anomaly-insight-get` with the candidate's `insight_id`. It gives every scored series of the insight, with each bucket's value, score, flag and detector `details`. A bucket with `scored: false` is unknown, not zero.
2. **Read the definition.** Call `insight-get` before you interpret a value. Smoothing, cumulative mode, formulas, math (unique users, sums, averages) and display settings change what a bucket value means. Note the insight `short_id` for keys, links and charts.
3. **Compare the value with its baselines.** Check the candidate value against the 28-day median and the same-weekday median in the history. Confirm that the move is still there in the live chart (`insight-query` with a widened `filters_override`), because late data can change a bucket after scoring.
4. **Check the scope.** When every series of the insight flags on the same bucket, the cause is global: capture, a deploy, a data gap, or a traffic change. When one series flags alone, the cause is in that segment.
5. **Reject detector artifacts before you claim a change:**
- A drop to exactly zero is almost always missing or late data. Run the source-liveness probe ([`references/source-liveness.md`](references/source-liveness.md)) instead of a metric report.
- A series in unusual units (a ratio, a percentage, an average, currency) can move a lot on a small count. Check the counts behind it.
- A launch, a migration, a backfill or an experiment the team knows about. Search the scratchpad for `noise:` and `addressed:` entries.
- Seasonality the 4-week baselines do not cover: a holiday, a month end, a quarter end.
6. **Dedupe.** Search the inbox (`inbox-reports-list`) and the `report:anomaly_detection:insight:<short_id>` scratchpad pointer. Edit a live report instead of filing a duplicate.
7. **Decide** with the rules in [Decide](#decide). In scored mode, a candidate that passed steps 3 to 5 clears the report bar. Put the insight `short_id`, the bucket, the value, both medians, the score and the series scope in the evidence.
Memory in scored mode: keep the `report:`, `noise:`, `addressed:` and `not-in-use:` keys from [`references/watchlist-and-memory.md`](references/watchlist-and-memory.md). Do not write `watchlist:` or `baseline:` entries. An artifact that recurs on one insight is a `noise:anomaly_detection:insight:<short_id>` entry, so the next run skips it fast.
## Structured output
When your run prompt has a `# Structured output` section, record one record for each candidate you judged, in both modes. Set:
- `mode`: `scored` or `legacy`.
- `source`: `precheck`, `search`, or `watchlist` (legacy mode).
- `verdict`, `artifact_kind`, `interesting`, `report_id`, and `reason`, as the schema describes.
- `subject`: `insight:<short_id>:<bucket date>`.
Record a rejected candidate too. The records are how PostHog measures the precision of each mode. When the section is absent, skip this step.
## Source liveness (both modes)
The source-liveness probe in [`references/source-liveness.md`](references/source-liveness.md) applies in both modes. Scoring treats a drop to zero as a likely artifact, and a source that went quiet is what the probe finds. Run it when a candidate drops to zero, when every insight drops together, and before a legacy quick close-out.
## Decide
For each candidate anomaly, classify against prior runs, the inbox, and the scratchpad (net-new / material-update / already-covered / addressed-or-noise — full classifier in [`references/watchlist-and-memory.md`](references/watchlist-and-memory.md)). You file findings on the **report channel**: a scored, attributed anomaly you'd stand behind is a finished, 1:1 inbox report, not a weak signal for the pipeline to cluster — so you author it directly. Then:
- **Author** a fresh report via `scout-emit-report` when the move is net-new and clears the bar. **Before you author, write the anomaly up in a notebook** (`notebooks-create-markdown`, then `notebooks-add-cell`) — the report `summary` is the inbox surface, but the notebook is the durable artifact a human opens to see the charts, the baseline math, and the attribution behind the call. Build it first, then link its URL from the report `summary` and cite it as an `evidence` entry. Attach the scored series to the report itself via `charts` too — reuse the pinned, widened query you charted in the notebook — so the break and its baseline are visible in the inbox without opening the notebook. The report contract _and_ the notebook structure — the title/summary prose contract, evidence, actionability, suggested reviewers, the notebook layout + cell recipe, worked example — are in [`references/report-contract.md`](references/report-contract.md). In legacy mode, a report-worthy anomaly is: robust z ≥ ~3.5 on the latest complete bucket, the move not explained by seasonality or a known data-pipeline gap, with the insight `short_id`, the bucket value, the baseline, the z-score, and the time window in the evidence. **Search the inbox first** (`inbox-reports-list`, plus your `report:` scratchpad pointer) — the channel is not idempotent, so never author a duplicate.
- **Edit** the existing report via `scout-edit-report` when one already covers this insight's anomaly (found via the inbox search or a `report:anomaly_detection:insight:<short_id>` pointer) and you have a material update — it's still firing, escalated, or correlates with a fresh deploy. Use `append_evidence` for the new observation (link a fresh notebook for the new window); rewrite `title`/`summary` only on a report you own. Don't author a second report for the same ongoing move.
- **Remember** if it's suggestive but below the bar, or to refresh a baseline / record what you ruled out.
- **Skip** if a `noise:` / `addressed:` / `report:` entry already covers it without new evidence.
## Close out
One paragraph: the mode and the candidate source, which watchlist items or candidates you checked, what you added, which anomalies you reported (authored or updated), and what you ruled out and why. The harness saves this as the run summary; future runs read it via `scout-runs-list`. Do **not** write a separate "run metadata" scratchpad entry. "Checked the due watchlist, everything within baseline" and "judged 3 scored candidates, all artifacts" are real outcomes.
## Disqualifiers (skip these)
- **Seasonal swings** — the regular daily/weekly rhythm (weekday vs weekend, business-hours vs overnight). Only real once the move clears the **seasonality-matched** baseline.
- **The current partial bucket** — the in-progress hour/day is incomplete; never score it.
- **Data-pipeline gaps, not real drops** — a metric that flat-lines to zero across _every_ insight at the same timestamp is almost always missing/late data or a deploy gap, not a product anomaly. Don't report it as a metric anomaly per insight. Run the source probe in [`references/source-liveness.md`](references/source-liveness.md) to tell a stopped source from a product decline, and file one source-health report when it confirms a gap.
- **Low-count noise** — series whose baseline counts are tiny; a few events of movement is not signal. Enforce the minimum relative-change and minimum-absolute-count floors.
- **Dev / test / internal-only segments** — bursts whose `properties.$environment` or service is `dev`/`local`/`test`, or single-user/single-session quirks.
- **Expected one-offs the team already knows about** — launches, migrations, backfills, known experiments. If a `noise:` / `addressed:` entry names it, skip.
When in doubt, refresh the baseline memory instead of reporting.
## MCP tools
Direct (read-only):
- `anomaly-search` — scored mode: which series of the project's saved trends insights were flagged in a date range, and `scoring_enabled` with `scored_history_ready`, which together pick the mode on a run with no `<precheck_result>` block.
- `anomaly-insight-get` — scored mode: the full scored history of one insight, every series with each bucket's value, score, flag and detector `details`.
- `alert-simulate` — legacy mode only, the primary scorer: run PostHog's statistical anomaly detectors on a saved insight's series (no alert required); returns per-point scores + triggered dates. Never with the AI detector (`llm`).
- `insights-trending-retrieve` — legacy mode: most-viewed insights (discovery / explore).
- `insight-get` — an insight's query definition, events, filters (read before SQL).
- `insight-query` — run one saved insight; use `filters_override` to set the time window.
- `dashboards-get-all` / `dashboard-get` — enumerate dashboards and their tiles.
- `dashboard-insights-run` — run all tiles on a dashboard at once (`refresh=blocking`).
- `insights-list` / `execute-sql` over `system.*` — search insights/dashboards by name.
- `execute-sql` over `events` — legacy mode fallback scorer: hourly/daily series + trailing baseline for non-saved series or custom baselines.
- `read-data-schema` — confirm events/properties before any SQL.
- `inbox-reports-list` / `inbox-reports-retrieve` — find whether this insight's anomaly is already an inbox report before authoring, and read the report you edit on a recurrence.
Local: `Bash` + `python3` — legacy mode, the distribution-shift lens: run a pure-stdlib two-sample KS / changepoint inline, or fetch the bundled `scripts/ks2.py` via `llma-skill-file-get` and write it to `/tmp` first (not on disk in a scheduled run). Feed it histograms from `execute-sql`.
Write (user-facing):
- `scout-emit-report` / `scout-edit-report` (gated on `signal_scout_report:write`) — the report channel: author a full inbox report for an anomaly, or update the existing one on a recurrence. Field-level contract in [`references/report-contract.md`](references/report-contract.md).
- `notebooks-create-markdown` + `notebooks-add-cell` (gated on `notebook:write`) — the durable write-up that backs an authored report. Build it _before_ authoring and reference its URL from the report `summary` and an `evidence` entry. Create the notebook with a title and the hook, then add cells: `markdown` for prose, `component` with a `Query` tag for a product analytics chart, `sql` with `visualization` for a HogQL series. The layout and cell recipe is in [`references/report-contract.md`](references/report-contract.md).
- `notebooks-destroy` — clean up the write-up if the report did not surface (preflight gate-skip, or the safety judge suppressed it) so a non-surfacing run leaves no orphan artifact. See [`references/report-contract.md`](references/report-contract.md).
- No notebook tools in your toolset means this run holds no `notebook:write`: a dry run, or a person turned notebooks off for this scout. Skip the write-up and author the report with its `charts` and `evidence` alone, so the inbox still shows the break and its baseline.
- `scout-record-output` — structured output, only when your run prompt has a `# Structured output` section.
Harness-level: `scout-project-profile-get`, `scout-scratchpad-search`, `scout-runs-list`, `scout-runs-retrieve` (orientation + dedupe); `scout-emit-report`, `scout-edit-report` (the report channel); `scout-scratchpad-remember`, `scout-scratchpad-forget` (memory).
## When to stop
- Scored mode: you judged every candidate, at most 20 → close out.
- Scored mode with nothing flagged, and every source live → close out empty.
- Legacy mode, nothing worth checking (quick close-out: no due watchlist item and every source live, or no usable source history) → close out empty.
- Legacy mode, you've checked the due watchlist items and added a couple of new ones → close out, even if more remain. Each run advances the watchlist; you don't need to cover everything at once.
- A candidate matches a `noise:` / `addressed:` / `dedupe:` entry → skip.
Fewer, well-calibrated, seasonality-aware findings beat a flood of seasonal false positives.
© PostHog · MIT · PostHog/skills