Skip to main content
skill CX Codex

signals-scout-error-tracking — Claude Code skill: what it does

The signals-scout-error-tracking skill monitors PostHog $exception activity for bursts, stuck loops, fingerprint clusters, and status regressions. Use it for recurring error-signal checks.

MIT 319 installs ★ 80 updated 2026-10-24

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add PostHog/skills --skill signals-scout-error-tracking -a claude-code
Open on GitHub

What signals-scout-error-tracking does

The skill examines $exception events in PostHog Signals and looks for meaningful changes, including sudden bursts, repeating loops, multi-fingerprint clusters, and regressions after changes. It compares event volume with affected-user counts to separate a local issue from background noise.

It uses error-tracking tools, SQL against the events table, and activity logs when available. Confirmed findings go through emit_report or edit_report; an existing inbox item is updated instead of duplicated. The skill requires the PostHog Signals agent, read-only analytics access, write access to the scratchpad and report channel, and the listed error-tracking MCP tools.

What it helps with

  • spot a burst of $exception events
  • separate a broad failure from a local issue
  • find errors trapped in a repeating loop
  • connect a regression with a deploy
  • update an existing report for a recurring issue

Example prompts for the agent

  • Check the $exception stream for new bursts, stuck errors, and status regressions.
  • Compare count with distinct_users, then report only confirmed issues.
  • Look for an existing report on this error and edit it if the problem is already covered.

How to install signals-scout-error-tracking

1Skills CLI (Vercel)

npx skills add PostHog/skills --skill signals-scout-error-tracking -a claude-code

2Manually: put the files in place

~/.claude/skills/signals-scout-error-tracking/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/signals-scout-error-tracking/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about signals-scout-error-tracking

How do I install signals-scout-error-tracking in Claude Code?

Run npx skills add PostHog/skills --skill signals-scout-error-tracking -a claude-code, then check ~/.claude/skills/signals-scout-error-tracking/SKILL.md.

What does signals-scout-error-tracking do?

It reviews $exception activity for bursts, stuck loops, fingerprint clusters, and status regressions. Confirmed findings become reports, while known issues are edited instead of duplicated.

What access does the skill need?

It is designed for the PostHog Signals agent in a Claude sandbox. The setup uses read-only analytics, signal_scout_internal:write, signal_scout_report:write, and the error-tracking MCP tools.

Source text

SKILL.md · MIT
---
name: signals-scout-error-tracking
scout-display-name: Error tracking
description: >
  Signals scout for PostHog error tracking. Watches `$exception` bursts, stuck loops, multi-
  fingerprint clusters, and status regressions.
compatibility: >
  PostHog Signals agent (Claude sandbox). Read-only analytics + signal_scout_internal:write
  (scratchpad) + signal_scout_report:write (report channel), plus the error-tracking tools in
  the MCP tools section (query-error-tracking-issues-list / -issue, execute-sql over the
  events table, advanced-activity-logs-list).
allowed_tools:
  - emit_report
  - edit_report
metadata:
  owner_team: signals
  scope: error_tracking
scout-precheck-query: |
  SELECT 1 AS has_exceptions
  FROM events
  WHERE event = '$exception'
    AND timestamp > {now} - INTERVAL 7 DAY
  LIMIT 1
---

# Signals scout: error tracking

You are a focused error tracking scout. Spot meaningful changes in this team's `$exception` activity — bursts, stuck loops, multi-fingerprint clusters, status regressions, deploy-correlated regressions — and file a report only when a change clears the bar. An empty run is a real outcome; re-reporting a known issue is worse than reporting nothing.

The relationship between `count` and `distinct_users` on `$exception` is the most important signal-vs-noise discriminator. Internalize that shape.

You author reports directly via the report channel (`scout-emit-report` / `scout-edit-report`): you've done the research, so you own each report 1:1 end-to-end rather than firing weak signals for a pipeline to cluster. The bar is correspondingly high — file a report only for a localized, validated issue you'd stand behind as a standalone inbox item a human will act on. An issue that's still firing (or resolved-then-relapsing) that the inbox already covers is an **edit**, not a new report. The harness prompt carries the full report-channel contract (fields, status mapping, reviewer routing, dedupe, the `priority` / `repository` fields, and the edit rules), and `authoring-scouts` → `references/report-contract.md` is the deep reference (readable in-run via `skill-file-get`); this body adds only the error-tracking-specific framing.

## Activity-history availability

Activity history is optional. Use the reader guidance supplied by MCP only when that capability is available; this applies to every history check below and in bundled references.

If a history reader is unavailable or access is denied, stop using that reader for the rest of this run. Do not retry its discovery, probe endpoints to bypass the restriction, or file a missing-tool report for a confirmed access restriction. Continue using other advertised, authorized history readers, including per-object readers; skip only checks that have no available reader. Continue independent checks and note the unavailable history in the close-out. Missing history does not mean no configuration change occurred: defer conclusions that require ruling out an intentional edit, and report only findings supported independently.

## Quick close-out: is error tracking even loud?

A scheduled run can start with a `<precheck_result>` block. The block means that at least one `$exception` event arrived in the last 7 days, so error tracking is in use. When the block is present, skip the `not-in-use` check. The block holds no count, so the baseline close-out still applies.

`top_events` holds only the project's top-ranked events.
A quiet but active `$exception` stream can fall outside that list, so absence from `top_events` does not prove that error tracking is inactive.

If `$exception` is in `top_events` at baseline (no fresh 24h activity, `recent_24h_count` ≪ `count / 7`), error tracking probably isn't where the signal is today. Cheap scratchpad entry + close out empty:

- key: `pattern:error_tracking:baseline-team{team_id}`
- content: `"$exception baseline ~{count}/day, no fresh 24h burst at {timestamp}"`

If `$exception` is absent from `top_events`, run one bounded activity check before you decide anything:

```sql
SELECT count() AS exceptions_7d,
       countIf(timestamp >= now() - INTERVAL 1 DAY) AS exceptions_24h
FROM events
WHERE event = '$exception'
  AND timestamp >= now() - INTERVAL 7 DAY
  AND timestamp <= now() + INTERVAL 1 DAY
```

If `execute-sql` is unavailable or the query fails, use `query-error-tracking-issues-list` (`status=active`, sorted by `last_seen_at`) as the check. An active issue seen in the last 7 days is activity.

| Check result                        | Action                                                                                                                                                         |
| ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Exceptions or active issues found   | The stream is active. Use these counts in place of the missing profile row and continue the investigation. Do not write `not-in-use`.                          |
| Zero exceptions and no active issue | Write key `not-in-use:error_tracking:team{team_id}`, content `"$exception absent, activity check found 0 in 7d at {timestamp}"`. Close out empty.              |
| No check could run                  | Activity is unknown. Do not write or refresh `not-in-use`. State the unavailable check in the close-out summary, and continue with any other available reader. |

A `not-in-use:error_tracking:*` entry from an earlier run does not stop this run.
Run the activity check again before you close out on it.
If the check finds activity, remove the entry with `scout-scratchpad-forget` and continue.

## How a run works

Cycle between these moves; skip what's not useful.

### Get oriented

Four cheap reads cold-start a run:

- `scout-scratchpad-search` (`text=error` or `text=exception`) — durable team steering from past error-tracking runs. Entries with `pattern:`, `noise:`, `addressed:`, `dedupe:`, `report:`, or `reviewer:` key prefixes tell you what's normal, what's already surfaced, what to skip, which report covers an issue, and who owns it.
- `scout-runs-list` (last 7d) — what prior error-tracking scouts found and ruled out.
- `scout-project-profile-get` — the `$exception` row in `top_events` carries `count`, `distinct_users`, `recent_24h_count`, `recent_24h_users` (pattern the count/users ratio against the table below), plus `existing_inbox_reports` for what's already in the inbox.
- `inbox-reports-list` (`ordering=-updated_at`, `search`=the specific issue id / fingerprint / failing-activity name) — the reports already in the inbox. Your own report-channel reports persist their backing signals under `source_product=signals_scout` (**not** `error_tracking`), so don't filter `source_product=error_tracking` — you'd miss every report you authored. A fresh burst on an issue you've reported before is an **edit**, not a new report; pull the closest matches with `inbox-reports-retrieve` before authoring.

### Profile shape — count vs distinct_users

| Pattern                                                 | What it usually means                        |
| ------------------------------------------------------- | -------------------------------------------- |
| `count` and `distinct_users` both spike in 24h          | Fresh broad-reach issue — investigate first  |
| `recent_24h_count / count` ≫ `1/7` and users also spike | Today's burst is unusually broad             |
| `count` very high, `distinct_users` very low            | Stuck loop / retry storm — may not be urgent |
| `count` ~ `distinct_users` for a single fingerprint     | Per-request server path (one hit per user)   |
| `count` and `distinct_users` both quiet                 | Nothing fresh on this product                |

### Explore

Patterns to watch — starting points, not a checklist.

#### Burst with broad reach

`recent_24h_count` and `recent_24h_users` both spike together. Usually a fresh regression — many users hitting it independently. Drill in:

1. `query-error-tracking-issues-list` filtered to `status=active`, sort by `last_seen_at`.
2. `execute-sql` against `events` with `event = '$exception' AND properties.$exception_issue_id = '<id>'` grouped by `toStartOfHour(timestamp)`.
3. Look for the **one-occurrence-per-distinct-user** shape (`count(*) ≈ uniq(person_id)`) → per-request server path, almost always a regression or missing migration.

#### Stuck loop (narrow reach)

`recent_24h_count` very high but `recent_24h_users` is small. A worker, cron, websocket, or retry is looping. Look at the issue's stack trace for the activity / job name. Often less urgent than a broad-reach burst, but worth a finding when count is in the thousands and the issue is fresh.

#### Multi-fingerprint cluster

Multiple fresh fingerprints (different `entity_id`s in `query-error-tracking-issues-list`) appearing in the same time window with overlapping stack traces, modules, or call sites → likely shared root cause. Bundle them in one finding (single description, evidence list with all fingerprint ids, dedupe key per fingerprint).

#### Status regression

An issue with `status=resolved` that's now firing again. Filter `query-error-tracking-issues-list` to `status=active` and check `last_seen_at` against `first_seen_at` — a large gap means old issue resurrected. Strong findings: the team explicitly closed them once.

#### Stack-trace activity name

When the issue is server-side, the stack trace usually names the failing activity / view / management command. Extract it (top frame, look for `<activity>_activity`, `def view_name`, etc.) and pair with `advanced-activity-logs-list` to find a recent deploy or model change correlation. Cross-source convergence is where this scout earns its keep.

### Save memory as you go

Memory is a continuous activity. Write a scratchpad entry whenever you observe something a future error-tracking run should know. Encode the "category" in the key prefix — `pattern:`, `noise:`, `addressed:`, `dedupe:`, `report:`, `reviewer:` — so future runs find it with a single `text=` search:

- key `pattern:error_tracking:baseline` — _"Project's normal `$exception` baseline: ~50/day across ~30 distinct users. Anything materially above that is fresh."_
- key `dedupe:error_tracking:019de34e` — _"Issue 019de34e — surfaced 2026-05-01 11:31–13:22Z, then quiet. If quiet next run, treat as already-surfaced; if firing, escalate."_
- key `noise:error_tracking:ci-runner-timeoutexpired` — _"`TimeoutExpired` raised by the team's own CI runner is recurring noise — internal tooling, not user-facing."_
- key `pattern:error_tracking:fetch_signals_for_report_activity` — _"Server activity `fetch_signals_for_report_activity` was a regression source on 2026-05-01 — if it appears in a fresh stack trace, double-check it's not the same root cause."_
- key `report:error_tracking:019de34e` — the `report_id` of a report you authored for issue `019de34e`, so the next run edits it (`append_evidence` with the fresh window) instead of duplicating.
- key `reviewer:error_tracking:ingestion` — a resolved owner (bare lowercase GitHub login) for a service / module / activity area, so reports route to a human faster.

By run #5 you'll have a local map of what's normal versus what warrants investigation, and burn less time on cold-start exploration.

### Decide

The generic report mechanics — search the inbox first (via the `report:error_tracking:<issue_id>` pointer, else an `inbox-reports-list` search on the issue's _specific_ terms — the issue id, the fingerprint, the failing activity name, not a broad word like `error`), edit-vs-author, the status rules, reviewer routing, the retry and cross-run deduplication rules, and the `priority` / `repository` / actionability fields — live in the harness prompt and in `authoring-scouts` → `references/report-contract.md`. Do not re-derive them here. This section is only the error-tracking judgment layered on top:

- **Edit** when a still-live report already tracks the same issue and it's still moving — a burst still elevated, a stuck loop still looping, a cluster still growing. A persistent issue is one report across runs: a fresh window confirming it's ongoing is a re-escalation (`append_evidence` with the fresh hourly counts and distinct-user numbers), not a new report per tick. A **status regression** is the exception — an issue the team explicitly `resolved` that's firing again is a genuinely new event; if its prior report is already closed, author a fresh report (per the status rules) and repoint `report:error_tracking:<issue_id>` rather than appending to a resolved item.
- **Author** when nothing live covers the issue. A report-worthy finding names the issue (issue id + fingerprint), shows the count-vs-distinct_users shape that makes it signal, quantifies the burst against baseline with an hourly breakdown, dates the onset, and — when the stack trace names a server activity / view — cites it with an `advanced-activity-logs-list` deploy correlation, all in the `evidence`. Attach the burst as a report chart — the issue's hourly or daily series with the baseline window on screen — so the spike and its onset are visible next to the numbers. Most findings are investigations → `actionability=requires_human_input` + `repository=NO_REPO`. The exception this surface earns: a well-localized bug whose stack trace points at a specific named file / module in a known repo can be `actionability=immediately_actionable` + `repository=owner/repo` to open a draft fix PR. Priority: a fresh broad-reach regression (count and distinct_users both spiking, per-request server path) or a resolved-issue status regression is **P1**, **P2** when reach is moderate; a stuck loop or narrow-reach cluster is **P3**, **P2** when count is in the thousands and fresh.
- **Remember** if it's below the bar but worth carrying forward (an issue drifting inside the noise band, a fingerprint building history), or to record what you ruled out and why.
- **Skip** with a one-line note if a `noise:` / `addressed:` / `dedupe:` entry, or an existing inbox report, already covers it.

Sibling courtesy: raw log-line rate/level shifts belong to the logs scout; LLM `$ai_*` errors to the ai-observability scout; CSP `$csp_violation` blocks to the csp-violations scout; errors surfaced through session friction to the session-replay scout. Honor their `dedupe:` entries — your unique angle is always the `$exception` issue-level burst / regression frame.

### Close out

**Summarize the run** — one paragraph: looked at what, which reports you authored or edited, what you remembered, what you ruled out. The harness writes that summary to the run row as searchable prose; future runs read it via `scout-runs-list`. Do **not** write a separate "run metadata" scratchpad entry — the run summary already serves that role.

## Disqualifiers (skip these)

- **Single user, single session, single occurrence** — almost always a personal browser quirk. Confirmed via low `count` AND low `distinct_users`.
- **Known upstream provider errors** — Anthropic / OpenAI rate limits, third-party API outages already covered by past memory. Skip unless volume / shape changes meaningfully.

When in doubt, write a memory entry instead of filing a report.

## MCP tools

Direct calls (read-only):

- `query-error-tracking-issues-list` — start here. Filter `status=active`, sort by `last_seen_at` desc.
- `query-error-tracking-issue` — drill into one issue (frames, sample events, occurrence counts).
- `execute-sql` against `events` — for hourly breakdowns, distinct-user counts, per-fingerprint correlation, time-window aggregations.
- `advanced-activity-logs-list` — pair stack-trace activity names with recent deploys or model changes for cross-source convergence.

Inbox & reviewer routing (mechanics in `authoring-scouts` → `references/report-contract.md`):

- `inbox-reports-list` / `inbox-reports-retrieve` — the reports already in the inbox; check before authoring so you edit instead of duplicating (`ordering=-updated_at`).
- `inbox-report-artefacts-list` — a comparable report's artefact log; reviewer precedent.
- `scout-members-list` — the in-run roster for routing `suggested_reviewers` to a service / module / activity owner.

Harness-level:

- `scout-project-profile-get` / `scout-scratchpad-search` / `scout-runs-list` / `scout-runs-retrieve` — orientation + dedupe.
- `scout-emit-report` / `scout-edit-report` — author a report / edit an existing one (the report-channel contract is in the harness prompt).
- `scout-scratchpad-remember` / `scout-scratchpad-forget` — remember / prune stale memory keys.

## When to stop

- `$exception` row in profile is at baseline → close out empty.
- `$exception` absent from profile and this run's activity check finds zero → `not-in-use` entry, close out empty.
- A candidate matches a scratchpad entry with `noise:` / `addressed:` / `dedupe:` key prefix, or an existing inbox report → edit-or-skip with a one-line note.
- You've validated some hypotheses and filed reports for what's solid → close out, even if there's more you could look at. Fewer, better reports.

"Looked but found nothing meaningful" is a real outcome.

© PostHog · MIT · PostHog/skills

Similar skills

All skills for Agent setup: memory, context, orchestration →

From the same set112

The whole set →
auditing-endpoints auditing-endpoints scans every endpoint in your PostHog project to find unused, broken, and stale versions. Generates a prioritised report with remediation steps but requires your approval before making changes. CX Codex +1 321installs diagnosing-ci-and-merge-bottlenecks The diagnosing-ci-and-merge-bottlenecks Claude Code skill examines CI health and the path from pull request creation to merge. It spots slow workflows, stalled PRs, and CI spend. CX Codex +1 320installs creating-an-endpoint creating-an-endpoint — a skill for designing PostHog endpoints with the right configuration on the first try. CX Codex +1 320installs signals-scout-csp-violations signals-scout-csp-violations monitors PostHog $csp_violation events for blocked domain clusters, per-directive bursts, and regression patterns. CX Codex +1 320installs signals-scout-experiments signals-scout-experiments monitors running PostHog A/B tests for sample ratio mismatch, flag mutations, broken exposure events, and lifecycle drift. Catches instrumentation flaws before teams act on biased results. CX Codex +1 319installs signals-scout-surveys signals-scout-surveys monitors PostHog surveys for score regressions, volume drops, and clusters open-text feedback into clear themes. CX Codex +1 319installs exploring-endpoint-execution-logs exploring-endpoint-execution-logs inspects execution records for a single PostHog endpoint to troubleshoot errors, cache misses, and zero-row outputs. Useful when an API query fails. CX Codex +1 318installs signals-scout-feature-flags signals-scout-feature-flags — a Claude Code skill that audits PostHog feature flags against incoming $feature_flag_called telemetry. It catches ghost flags, abrupt evaluation drops, and uncleaned code checks. CX Codex +1 318installs

Project sponsors

Companies that keep this analytics open