Skip to main content
skill CX Codex

vision-bench — Claude Code skill: image scoring

vision-bench is a Claude Code skill that scores and compares images with vision LLM judges using YAML rubrics. It evaluates image generation, OCR, charts, UI screenshots, and style.

MIT 178 installs ★ 414 updated 2026-11-07

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add glebis/claude-skills --skill vision-bench -a claude-code
Open on GitHub

What vision-bench does

This skill runs bench.py to score one image or compare several files. It sends them to one or more vision LLM judges, applies a selected YAML rubric, and writes the result as Markdown, JSON, or a table. The bundled presets cover 11 tasks, including text_to_image, photorealism, document OCR, chart reading, UI screenshots, invoices, portraits, scientific images, alt text, and artistic style.

Choose a judge with --judge or several with --judges. Provider prefixes cover OpenAI, Anthropic, Gemini, Mistral, and OpenRouter. A custom .yaml file can replace the bundled criteria through --criteria. The repository also contains the judge logic, report writer, SOPS vault handling, and the criteria/ preset directory.

The Python setup uses pyyaml, openai, anthropic, and mistralai. A provider key is needed, such as OPENAI_API_KEY or GEMINI_API_KEY; encrypted values can live in secrets.enc.yaml and be handled with SOPS and age. Historical scores from gemini-3.5-flash-lite should not be treated as comparable until a live image/JSON smoke test and rubric check have been run.

What it helps with

  • compare two generated images against one rubric
  • score photorealism or artistic style
  • measure text extraction from a document
  • check how a model reads a chart or invoice
  • export multi-judge image scores as a report

Example prompts for the agent

  • Compare img_a.png and img_b.png with the text_to_image rubric, the prompt “a fox in a snowy forest”, and judge gpt-4o.
  • Score portrait.png with gpt-4o, gemini-3.5-flash-lite, and claude-opus-4-5-20251022, then compare their results.
  • Evaluate chart.png with chart_analysis and save the report to report.md.

How to install vision-bench

1Skills CLI (Vercel)

npx skills add glebis/claude-skills --skill vision-bench -a claude-code

2Manually: put the files in place

~/.claude/skills/vision-bench/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/vision-bench/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about vision-bench

How do I install vision-bench in Claude Code?

Run npx skills add glebis/claude-skills --skill vision-bench -a claude-code. The skill file will be placed at ~/.claude/skills/vision-bench/SKILL.md.

What does vision-bench do?

It scores and compares images with one or more vision LLM judges against structured criteria. Results can be written as Markdown, JSON, or a table.

Which API keys does vision-bench use?

It supports OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, and OPENROUTER_API_KEY. Keys may come from secrets.enc.yaml through SOPS and age or from environment variables.

Source text

SKILL.md · MIT
---
name: vision-bench
description: Score and compare images using vision LLMs as judges. YAML-defined criteria presets for 11 use cases (text-to-image, photorealism, document OCR, charts, UI, portrait, product, scientific, invoice, alt-text, artistic style). Supports OpenAI, Anthropic, Gemini, Mistral, and OpenRouter as judge providers. Keys auto-decrypted via SOPS + age.
---

# Vision Bench — LLM Image Evaluation

Compare images by scoring them with one or more vision LLM judges against structured rubric criteria.

## Quick Start

```bash
# Install dependencies
pip install pyyaml openai anthropic mistralai

# Score a single image
python bench.py image.png --criteria photorealism --judge gemini-3.5-flash-lite

# Compare two AI-generated images
python bench.py img_a.png img_b.png \
  --criteria text_to_image \
  --prompt "a fox in a snowy forest" \
  --judge gpt-4o

# Multi-judge consensus
python bench.py img.png \
  --criteria portrait \
  --judges gpt-4o gemini-3.5-flash-lite claude-opus-4-5-20251022

# OpenRouter models (any vision-capable model)
python bench.py img_a.png img_b.png \
  --criteria artistic_style \
  --judges "openrouter/meta-llama/llama-4-maverick" "openrouter/mistralai/pixtral-large-2411"

# List all presets
python bench.py --list-presets

# Save report to file
python bench.py img.png --criteria chart_analysis --save report.md
```

## Presets

| Preset | Use Case |
|--------|----------|
| `text_to_image` | Compare AI image generators (Midjourney, DALL-E, Flux) |
| `photorealism` | How convincingly an image looks like a photo |
| `artistic_style` | Style consistency, composition, color harmony |
| `portrait` | AI-generated portrait quality and realism |
| `product_photo` | E-commerce product image quality |
| `document_ocr` | Document text extraction and layout understanding |
| `chart_analysis` | Chart and data visualization comprehension |
| `invoice` | Financial document field extraction accuracy |
| `ui_screenshot` | App/web screenshot understanding |
| `scientific` | Scientific/medical image accuracy |
| `alt_text` | Accessibility image description quality |

Custom criteria: pass any `.yaml` file as `--criteria path/to/my.yaml`.

## Judge Providers

| Prefix | Provider | Example |
|--------|----------|---------|
| `gpt-`, `o1`, `o3`, `o4` | OpenAI | `gpt-4o` |
| `claude-` | Anthropic | `claude-sonnet-4-5-20251022` |
| `gemini-` | Google Gemini | `gemini-3.5-flash-lite` |
| `pixtral-`, `mistral-`, `ministral-` | Mistral | `pixtral-12b-2409` |
| `openrouter/` | OpenRouter (any model) | `openrouter/meta-llama/llama-4-maverick` |

## API Keys

Keys are loaded from `secrets.enc.yaml` (SOPS + age encrypted) with fallback to environment variables.

Supported keys: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `OPENROUTER_API_KEY`

To encrypt your own keys:
```bash
sops --config .sops.yaml --encrypt --input-type yaml --output-type yaml secrets.yaml > secrets.enc.yaml
```

## Output Formats

`--output markdown` (default) · `--output json` · `--output table`

## Files

- `bench.py` — CLI entry point
- `judge.py` — Multi-provider LLM judge logic
- `report.py` — Report generation
- `vault.py` — SOPS secrets decryption
- `criteria/` — 11 YAML preset files
- `.sops.yaml` — Age key config for encryption
- `secrets.enc.yaml` — Encrypted API keys

Use `--judge` or `--judges` to select another model explicitly. The Gemini path sends one image/text turn with JSON output and no tools, history, thinking or sampling settings. The 3.5 Flash-Lite default requires a live image/JSON smoke test and rubric comparison before relying on historical score comparability; offline request tests do not establish model quality.

© glebis · MIT · glebis/claude-skills

Similar skills

All skills for AI and ML →

From the same set51

The whole set →

Project sponsors

Companies that keep this analytics open