Skip to main content
skill CX Codex

skill-logger — Claude Code skill: usage logging and quality scoring

skill-logger — a skill for Claude Code, Codex, Cursor and more: records every skill invocation, computes a quality score, and captures feedback signals. Activate it when you want to know which skills underperform and why.

MIT 197 installs ★ 274 updated 2026-10-30

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add curiositech/some_claude_skills --skill skill-logger -a claude-code
Open on GitHub

What skill-logger does

skill-logger runs on Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, and OpenCode. Each invocation captures the user query, execution timing, tool calls (Read, Write, Edit, Bash, Grep, Glob), token counts, and any errors that occurred.

Every run gets a score from 0 to 100 across four weighted components: error-free completion (25%), token efficiency (20%), output quality based on user edits and acceptance (30%), and satisfaction signals like follow-up questions (25%). Results go into a SQLite schema indexed by skill name, timestamp, and quality score — or into a portable JSON log if you prefer.

This skill does not create or document other skills; agent-creator and skill-coach handle those jobs. A score below 50 flags a skill for review; below 25 means it needs immediate attention, so the data has a direct path to action.

What it helps with

  • record skill invocations with query context, timing, and tool call counts
  • compute a weighted quality score across completion, efficiency, acceptance, and satisfaction
  • collect feedback signals: user edits ratio, acceptance, and follow-up frequency
  • surface skills with high error rates or consistently low quality scores
  • compare two skill variants in an A/B test using aggregated score data

Example prompts for the agent

  • Start logging the wedding-immortalist skill and define quality signals
  • Show weekly aggregate metrics for all skills in this project
  • Set up a scoring pipeline: calculate a 0–100 quality score for every skill run

How to install skill-logger

1Skills CLI (Vercel)

npx skills add curiositech/some_claude_skills --skill skill-logger -a claude-code

2Manually: put the files in place

~/.claude/skills/skill-logger/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/skill-logger/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about skill-logger

How do I install skill-logger in Claude Code?

Run npx skills add curiositech/some_claude_skills --skill skill-logger -a claude-code in your terminal. The skill file lands at ~/.claude/skills/skill-logger/SKILL.md.

What does skill-logger do?

It records each skill invocation — input query, tokens used, tool calls, errors, and timestamps — then computes a 0–100 quality score from four weighted components and stores everything in SQLite or JSON.

How is skill-logger different from skill-coach?

skill-coach handles skill documentation and guidance; skill-logger measures runtime quality of skills already in use, collecting feedback signals and scoring each invocation to identify where improvements are needed.

Source text

SKILL.md · MIT
---
name: skill-logger
description: Logs and scores skill usage quality, tracking output effectiveness, user satisfaction signals, and improvement opportunities. Expert in skill analytics, quality metrics, feedback loops, and
  continuous improvement. Activate on "skill logging", "skill quality", "skill analytics", "skill scoring", "skill performance", "skill metrics", "track skill usage", "skill improvement". NOT for creating
  skills (use agent-creator), skill documentation (use skill-coach), or runtime debugging (use debugger skills).
allowed-tools: Read,Write,Edit,Bash,Grep,Glob
metadata:
  category: Productivity & Meta
  pairs-with:
  - skill: automatic-stateful-prompt-improver
    reason: Data for prompt optimization
  - skill: skill-coach
    reason: Quality tracking feeds coaching
  tags:
  - logging
  - analytics
  - metrics
  - quality
  - improvement
---

# Skill Logger

Track, measure, and improve skill quality through systematic logging and scoring.

## When to Use This Skill

**Use for:**
- Setting up skill usage logging
- Defining quality metrics for skill outputs
- Analyzing skill performance over time
- Identifying skills that need improvement
- Building feedback loops for skill enhancement
- A/B testing skill variations

**NOT for:**
- Creating new skills → use agent-creator
- Skill documentation → use skill-coach
- Runtime debugging → use appropriate debugger skills
- General logging/monitoring → use devops-automator

## Core Logging Architecture

```
┌────────────────────────────────────────────────────────────────┐
│                    SKILL LOGGING PIPELINE                       │
├────────────────────────────────────────────────────────────────┤
│                                                                 │
│  1. CAPTURE          2. ANALYZE           3. SCORE              │
│  ├─ Invocation       ├─ Output parse      ├─ Quality metrics    │
│  ├─ Input context    ├─ Token usage       ├─ User satisfaction  │
│  ├─ Output           ├─ Tool calls        ├─ Goal completion    │
│  └─ Timing           └─ Error patterns    └─ Efficiency         │
│                                                                 │
│  4. AGGREGATE        5. ALERT             6. IMPROVE            │
│  ├─ Per-skill stats  ├─ Quality drops     ├─ Identify patterns  │
│  ├─ Trend analysis   ├─ Error spikes      ├─ Suggest changes    │
│  └─ Comparisons      └─ Underuse          └─ Track experiments  │
│                                                                 │
└────────────────────────────────────────────────────────────────┘
```

## What to Log

### Invocation Data

```json
{
  "invocation_id": "uuid",
  "timestamp": "ISO8601",
  "skill_name": "wedding-immortalist",
  "skill_version": "1.2.0",

  "input": {
    "user_query": "Create a 3D model from my wedding photos",
    "context_tokens": 1500,
    "files_referenced": ["photos/", "config.json"]
  },

  "execution": {
    "duration_ms": 45000,
    "tool_calls": [
      {"tool": "Bash", "count": 5},
      {"tool": "Write", "count": 3}
    ],
    "tokens_used": {
      "input": 8500,
      "output": 3200
    },
    "errors": []
  },

  "output": {
    "type": "code_generation",
    "artifacts_created": ["pipeline.py", "config.yaml"],
    "response_length": 3200
  }
}
```

### Quality Signals

```python
QUALITY_SIGNALS = {
    # Implicit signals (automated)
    'completion': 'Did the skill complete without errors?',
    'token_efficiency': 'Output quality per token used',
    'tool_success_rate': 'Tool calls that succeeded',
    'retry_count': 'How many retries needed?',

    # Explicit signals (user feedback)
    'user_edit_ratio': 'How much did user modify output?',
    'user_accepted': 'Did user accept/use the output?',
    'follow_up_needed': 'Did user need to ask for fixes?',
    'explicit_rating': 'Thumbs up/down if available',

    # Outcome signals (delayed)
    'code_ran_successfully': 'Did generated code work?',
    'tests_passed': 'Did it pass tests?',
    'reverted': 'Was the output later reverted?',
}
```

## Scoring Framework

### Multi-Dimensional Quality Score

```python
def calculate_skill_score(invocation_log):
    """Score a skill invocation 0-100."""

    scores = {
        # Completion (25%)
        'completion': (
            25 if invocation_log['errors'] == [] else
            15 if invocation_log['recovered'] else
            0
        ),

        # Efficiency (20%)
        'efficiency': min(20, 20 * (
            BASELINE_TOKENS / invocation_log['tokens_used']
        )),

        # Output Quality (30%)
        'quality': (
            30 if invocation_log['user_accepted'] else
            20 if invocation_log['user_edit_ratio'] < 0.2 else
            10 if invocation_log['user_edit_ratio'] < 0.5 else
            0
        ),

        # User Satisfaction (25%)
        'satisfaction': (
            25 if invocation_log['explicit_rating'] == 'positive' else
            15 if invocation_log['no_follow_up'] else
            5 if invocation_log['follow_up_resolved'] else
            0
        ),
    }

    return sum(scores.values())
```

### Score Interpretation

| Score Range | Quality Level | Action |
|-------------|---------------|--------|
| 90-100 | Excellent | Document as exemplar |
| 75-89 | Good | Monitor for consistency |
| 50-74 | Acceptable | Review for improvements |
| 25-49 | Poor | Prioritize fixes |
| 0-24 | Failing | Immediate intervention |

## Log Storage Schema

### SQLite Schema (Local)

```sql
CREATE TABLE skill_invocations (
    id TEXT PRIMARY KEY,
    skill_name TEXT NOT NULL,
    skill_version TEXT,
    timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,

    -- Input
    user_query TEXT,
    context_tokens INTEGER,

    -- Execution
    duration_ms INTEGER,
    tokens_input INTEGER,
    tokens_output INTEGER,
    tool_calls_json TEXT,
    errors_json TEXT,

    -- Output
    output_type TEXT,
    artifacts_json TEXT,
    response_length INTEGER,

    -- Quality signals
    user_accepted BOOLEAN,
    user_edit_ratio REAL,
    follow_up_needed BOOLEAN,
    explicit_rating TEXT,

    -- Computed
    quality_score REAL,

    INDEX idx_skill_name (skill_name),
    INDEX idx_timestamp (timestamp),
    INDEX idx_quality (quality_score)
);

CREATE TABLE skill_aggregates (
    skill_name TEXT,
    period TEXT,  -- 'daily', 'weekly', 'monthly'
    period_start DATE,

    invocation_count INTEGER,
    avg_quality_score REAL,
    error_rate REAL,
    avg_tokens_used INTEGER,
    avg_duration_ms INTEGER,

    PRIMARY KEY (skill_name, period, period_start)
);
```

### JSON Log Format (Portable)

```json
{
  "logs_version": "1.0",
  "skill_name": "wedding-immortalist",
  "entries": [
    {
      "id": "uuid",
      "timestamp": "2025-01-15T14:30:00Z",
      "input": {...},
      "execution": {...},
      "output": {...},
      "quality": {
        "signals": {...},
        "score": 85,
        "computed_at": "2025-01-15T14:35:00Z"
      }
    }
  ]
}
```

## Analytics Queries

### Skill Performance Dashboard

```sql
-- Overall skill rankings
SELECT
    skill_name,
    COUNT(*) as uses,
    AVG(quality_score) as avg_quality,
    AVG(tokens_output) as avg_tokens,
    SUM(CASE WHEN errors_json != '[]' THEN 1 ELSE 0 END) * 100.0 / COUNT(*) as error_rate
FROM skill_invocations
WHERE timestamp > datetime('now', '-30 days')
GROUP BY skill_name
ORDER BY avg_quality DESC;

-- Quality trend (weekly)
SELECT
    skill_name,
    strftime('%Y-%W', timestamp) as week,
    AVG(quality_score) as avg_quality,
    COUNT(*) as uses
FROM skill_invocations
GROUP BY skill_name, week
ORDER BY skill_name, week;

-- Problem detection
SELECT skill_name, COUNT(*) as failures
FROM skill_invocations
WHERE quality_score < 50
  AND timestamp > datetime('now', '-7 days')
GROUP BY skill_name
HAVING failures >= 3
ORDER BY failures DESC;
```

### Improvement Opportunities

```python
def identify_improvement_opportunities(skill_name, logs):
    """Analyze logs to suggest skill improvements."""

    opportunities = []

    # Pattern 1: Common follow-up questions
    follow_ups = extract_follow_up_patterns(logs)
    if follow_ups:
        opportunities.append({
            'type': 'missing_capability',
            'description': f'Users frequently ask: {follow_ups[0]}',
            'suggestion': 'Add guidance for this common need'
        })

    # Pattern 2: High edit ratio in specific output types
    edit_patterns = analyze_edit_patterns(logs)
    if edit_patterns['code'] > 0.4:
        opportunities.append({
            'type': 'code_quality',
            'description': 'Users frequently edit generated code',
            'suggestion': 'Review code examples and templates'
        })

    # Pattern 3: Repeated errors
    error_patterns = cluster_errors(logs)
    for error_type, count in error_patterns:
        if count >= 3:
            opportunities.append({
                'type': 'recurring_error',
                'description': f'{error_type} occurred {count} times',
                'suggestion': 'Add error handling or documentation'
            })

    return opportunities
```

## Implementation Guide

### Basic Logger Hook

```python
# hooks/skill_logger.py
import json
import sqlite3
from datetime import datetime
from pathlib import Path

LOG_DB = Path.home() / '.claude' / 'skill_logs.db'

def log_skill_invocation(
    skill_name: str,
    user_query: str,
    output: str,
    tool_calls: list,
    duration_ms: int,
    tokens: dict,
    errors: list = None
):
    """Log a skill invocation to the database."""

    conn = sqlite3.connect(LOG_DB)
    cursor = conn.cursor()

    cursor.execute('''
        INSERT INTO skill_invocations
        (id, skill_name, timestamp, user_query, duration_ms,
         tokens_input, tokens_output, tool_calls_json, errors_json,
         response_length)
        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
    ''', (
        str(uuid.uuid4()),
        skill_name,
        datetime.utcnow().isoformat(),
        user_query,
        duration_ms,
        tokens.get('input', 0),
        tokens.get('output', 0),
        json.dumps(tool_calls),
        json.dumps(errors or []),
        len(output)
    ))

    conn.commit()
    conn.close()
```

### Quality Signal Collection

```python
def collect_quality_signals(invocation_id: str, signals: dict):
    """Update an invocation with quality signals."""

    conn = sqlite3.connect(LOG_DB)
    cursor = conn.cursor()

    # Update with user feedback
    cursor.execute('''
        UPDATE skill_invocations
        SET user_accepted = ?,
            user_edit_ratio = ?,
            follow_up_needed = ?,
            explicit_rating = ?,
            quality_score = ?
        WHERE id = ?
    ''', (
        signals.get('accepted'),
        signals.get('edit_ratio'),
        signals.get('follow_up'),
        signals.get('rating'),
        calculate_score(signals),
        invocation_id
    ))

    conn.commit()
    conn.close()
```

## Alerting & Notifications

### Alert Conditions

```python
ALERT_CONDITIONS = {
    'quality_drop': {
        'condition': 'avg_quality_7d < avg_quality_30d * 0.8',
        'message': 'Skill {skill} quality dropped 20%+ in past week',
        'severity': 'warning'
    },
    'error_spike': {
        'condition': 'error_rate_24h > error_rate_7d * 2',
        'message': 'Skill {skill} error rate doubled in past 24h',
        'severity': 'critical'
    },
    'underused': {
        'condition': 'uses_7d < uses_30d_avg * 0.5',
        'message': 'Skill {skill} usage down 50%+ this week',
        'severity': 'info'
    },
    'high_performer': {
        'condition': 'avg_quality_7d > 90 AND uses_7d > 10',
        'message': 'Skill {skill} performing excellently',
        'severity': 'positive'
    }
}
```

## Anti-Patterns

### "Log Everything"
**Wrong**: Logging complete input/output for every invocation.
**Why**: Privacy concerns, storage explosion, noise.
**Right**: Log metadata, summaries, and opt-in detailed logging.

### "Score Once, Forget"
**Wrong**: Calculating quality score immediately after completion.
**Why**: Misses delayed signals (did code work? was it reverted?).
**Right**: Collect signals over time, recalculate periodically.

### "Averages Only"
**Wrong**: Only tracking average quality scores.
**Why**: Hides distribution, misses failure modes.
**Right**: Track percentiles, failure rates, and patterns.

### "No Baseline"
**Wrong**: Measuring quality without establishing baselines.
**Why**: Can't detect improvement or regression.
**Right**: Establish baselines per skill, compare trends.

## Output Reports

### Weekly Skill Health Report

```markdown
# Skill Health Report - Week of 2025-01-13

## Overview
- Total invocations: 247
- Average quality: 78.3 (up 2.1 from last week)
- Error rate: 4.2% (down 1.8%)

## Top Performers
1. **wedding-immortalist** - 92.1 avg quality, 18 uses
2. **skill-coach** - 89.4 avg quality, 34 uses
3. **api-architect** - 87.2 avg quality, 22 uses

## Needs Attention
1. **legacy-code-converter** - 52.3 avg quality (down 15%)
   - Common issue: Missing dependency detection
   - Suggested fix: Add dependency scanning step

## Improvement Opportunities
- `partner-text-coach`: Users frequently ask for tone adjustment
- `yard-landscaper`: High edit ratio on plant recommendations
```

## Integration Points

- **skill-coach**: Feed quality data for skill improvements
- **agent-creator**: Use metrics when designing new skills
- **automatic-stateful-prompt-improver**: Quality signals for prompt optimization

---

**Core Philosophy**: What gets measured gets improved. Skill logging transforms intuition about skill quality into actionable data, enabling continuous improvement of the entire skill ecosystem.

© curiositech · MIT · curiositech/some_claude_skills

Similar skills

All skills for Agent setup: memory, context, orchestration →
code-review-quality proffesor-for-testing code-review-quality is a Claude Code skill for reviewing code changes with attention to bugs, security, and testability. It can guide PR feedback or help set team review practices. CX Codex +1 1,600installs metrics-dashboard phuryn Helps you design a product metrics dashboard with North Star Metric, Input Metrics, Health Metrics, and Business Metrics. Use it when defining KPIs, setting up product analytics, or building a data monitoring plan. CX Codex +1 3,200installs receipts Karan Bansal receipts — a skill that displays your personal quality card across sessions: failure rate, edit churn, and tokens per task broken down by model version. Runs silently in the background and surfaces shifts in model performance. CX Codex +1 ★ 586repo stars bvssh-check haabe bvssh-check — a skill for Claude Code, Cursor and other editors that checks whether your current work aligns with Better Value Sooner Safer Happier principles. Run at diamond completion and periodically throughout development. CX Codex +1 ★ 54repo stars analytics OrchestKit analytics queries local Claude Code logs to report agent usage, hook latency, and token expenses. Runs whenever you inspect session costs or runtime bottlenecks. CX Codex +1 210installs developer-community-health Samuel Berthe developer-community-health — a skill that turns "how is our community doing?" into a short set of metrics you can actually collect. Useful when a chat or forum goes quiet. CX Codex +1 13,300installs

From the same set180

The whole set →
playwright-e2e-tester playwright-e2e-tester is a Claude Code skill that generates Playwright test suites, sets up Page Object models, and configures CI/CD for browser automation. CX Codex +1 200installs sober-addict-protector sober-addict-protector is a Claude Code skill for daily recovery check-ins, spotting triggers, assessing risk, and planning ways to protect sobriety before a lapse. CX Codex +1 199installs product-appeal-analyzer product-appeal-analyzer is a Claude Code skill that assesses product appeal, trust, and positioning. It reviews landing pages and product messaging. CX Codex +1 199installs background-job-orchestrator background-job-orchestrator designs background queues with BullMQ, Celery, or cloud tasks. Offloads heavy logic from HTTP handlers, configures retries, and scales worker processes. CX Codex +1 198installs site-reliability-engineer The site-reliability-engineer skill checks Docusaurus builds for MDX issues, validates links, and helps catch site problems before deployment. CX Codex +1 196installs skill-documentarian skill-documentarian keeps a Claude Skills site in sync with its source files, metadata, tags, and badges. Use it when adding or checking skills. CX Codex +1 196installs vr-avatar-engineer vr-avatar-engineer guides work on VR avatar systems with facial tracking, network sync, and skin rendering. Useful for projects targeting Vision Pro, Quest, or PC VR. CX Codex +1 195installs real-time-collaboration-engine real-time-collaboration-engine guides real-time collaborative editing using WebSockets, OT, and CRDTs. Useful for building multi-user document editors, shared boards, and presence tracking. CX Codex +1 195installs

Project sponsors

Companies that keep this analytics open