Skip to main content
skill CX Codex

signals-scout-mcp-tool-calls — Claude Code skill: what it does

signals-scout-mcp-tool-calls — a PostHog Signals skill that sifts $mcp_tool_call telemetry for MCP tools agents struggle with, then files one report per product category.

no license 194 installs ★ 101 updated 2026-11-16

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add PostHog/ai-plugin --skill signals-scout-mcp-tool-calls -a claude-code
Open on GitHub

What signals-scout-mcp-tool-calls does

Inside the PostHog Signals agent (Claude sandbox), this skill queries the $mcp_tool_call event and looks for MCP tools agents cannot use cleanly. Error share counts only when volume and reach back it up, so failures get weighed against distinct users and sessions. Retry loops inside one session, slow p95 $mcp_duration_ms, oversized responses and a jump in call frequency against the previous window are findings too. Tools are bucketed by $mcp_tool_category, the product team that owns them.

Field coverage comes first. A precheck query over the last 7 days tells whether the event arrives at all, then the skill checks which properties the project actually sends: PostHog's own hono server carries $mcp_error_type and the category, external SDK servers carry the raw error text. Two reference files, references/queries.md and references/metrics-dashboard.md, hold the SQL and the dashboard recipe and are opened as needed. Every problem tool leaves with a fix hypothesis attached, written through emit_report; a category that already has a live report gets amended with edit_report rather than a second one.

Access is the gate: execute-sql, read-data-schema, write on the scratchpad and on the report channel. Without the Signals agent in the Claude sandbox nothing runs. A run that turns up nothing is a valid outcome.

What it helps with

  • spot MCP tools failing at a high rate on real traffic
  • separate a genuine break from single-user noise
  • explain why agents keep calling the same tool in a loop
  • file a per-category report with a concrete fix
  • catch a tool whose call volume jumped

Example prompts for the agent

  • Run the scout over our MCP tool calls and list the tools that fail most.
  • Check $mcp_tool_call for the past week and file a report for the category with problem tools.
  • Update the report for that category, a new tool joined the broken ones.

How to install signals-scout-mcp-tool-calls

1Skills CLI (Vercel)

npx skills add PostHog/ai-plugin --skill signals-scout-mcp-tool-calls -a claude-code

2Manually: put the files in place

~/.claude/skills/signals-scout-mcp-tool-calls/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/signals-scout-mcp-tool-calls/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about signals-scout-mcp-tool-calls

How do I install signals-scout-mcp-tool-calls in Claude Code?

Run npx skills add PostHog/ai-plugin --skill signals-scout-mcp-tool-calls -a claude-code. The skill file then sits at ~/.claude/skills/signals-scout-mcp-tool-calls/SKILL.md.

What does signals-scout-mcp-tool-calls do?

It reads $mcp_tool_call telemetry, finds MCP tools that get in the way of agents, and groups them by product category. Each category with problem tools gets one report carrying a suggested fix.

What access does the skill need?

The PostHog Signals agent in a Claude sandbox, plus analytics access: execute-sql, read-data-schema, and write on the scratchpad and the report channel. Reports go out through emit_report, edits through edit_report.

Similar skills

All skills for Agent setup: memory, context, orchestration →

From the same set82

The whole set →
authoring-scouts authoring-scouts is a Claude Code skill for creating, tuning, and testing PostHog Signals scouts that inspect project metrics on a schedule and submit findings. CX Codex +1 196installs creating-online-evaluations creating-online-evaluations — a Claude Code skill that turns production failure modes into continuously-running online evaluations in PostHog. It starts when new generations need automatic scoring against specific errors. CX Codex +1 195installs signals-scout-insight-alerts signals-scout-insight-alerts — a Claude Code skill for reviewing insight alert history in PostHog. It scans recent alert triggers and flags material incidents that missed notifying the team. CX Codex +1 195installs signals-scout-data-warehouse signals-scout-data-warehouse monitors warehouse imports for failures, stale data, and sharp row-count drops, then flags recurring query patterns that may merit materialized views. CX Codex +1 195installs signals-scout-web-vitals signals-scout-web-vitals tracks page p75 LCP, INP, CLS, and FCP from PostHog events, spots performance degradation against Google thresholds, and links regressions to deploys. CX Codex +1 194installs signals-scout-skills-store signals-scout-skills-store audits shared PostHog agent skills for vague descriptions, oversized bodies, broken links, and leaked secrets. Produces reports with copy-ready fixes. CX Codex +1 193installs analyzing-expensive-users analyzing-expensive-users — a skill for Claude Code, Codex, Cursor and other agents that finds the top spenders in PostHog AI observability and explains what drives their costs. CX Codex +1 192installs turning-engineering-analytics-into-insights turning-engineering-analytics-into-insights — a skill for Claude Code and Codex: converts PR and CI metrics into saved PostHog insights, dashboards, and automated subscriptions. Triggers when visualizing engineering team performance. CX Codex +1 189installs

Project sponsors

Companies that keep this analytics open