vision-bench — Claude Code skill: image scoring
vision-bench is a Claude Code skill that scores and compares images with vision LLM judges using YAML rubrics. It evaluates image generation, OCR, charts, UI screenshots, and style.
MIT 178 installs ★ 414 updated 2026-11-07
npx skills add glebis/claude-skills --skill vision-bench -a claude-codeWhat vision-bench does
This skill runs bench.py to score one image or compare several files. It sends them to one or more vision LLM judges, applies a selected YAML rubric, and writes the result as Markdown, JSON, or a table. The bundled presets cover 11 tasks, including text_to_image, photorealism, document OCR, chart reading, UI screenshots, invoices, portraits, scientific images, alt text, and artistic style.
Choose a judge with --judge or several with --judges. Provider prefixes cover OpenAI, Anthropic, Gemini, Mistral, and OpenRouter. A custom .yaml file can replace the bundled criteria through --criteria. The repository also contains the judge logic, report writer, SOPS vault handling, and the criteria/ preset directory.
The Python setup uses pyyaml, openai, anthropic, and mistralai. A provider key is needed, such as OPENAI_API_KEY or GEMINI_API_KEY; encrypted values can live in secrets.enc.yaml and be handled with SOPS and age. Historical scores from gemini-3.5-flash-lite should not be treated as comparable until a live image/JSON smoke test and rubric check have been run.
What it helps with
- compare two generated images against one rubric
- score photorealism or artistic style
- measure text extraction from a document
- check how a model reads a chart or invoice
- export multi-judge image scores as a report
Example prompts for the agent
- Compare
img_a.pngandimg_b.pngwith thetext_to_imagerubric, the prompt “a fox in a snowy forest”, and judgegpt-4o. - Score
portrait.pngwithgpt-4o,gemini-3.5-flash-lite, andclaude-opus-4-5-20251022, then compare their results. - Evaluate
chart.pngwithchart_analysisand save the report toreport.md.
How to install vision-bench
1Skills CLI (Vercel)
npx skills add glebis/claude-skills --skill vision-bench -a claude-code2Manually: put the files in place
~/.claude/skills/vision-bench/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/vision-bench/
1Skills CLI (Vercel)
npx skills add glebis/claude-skills --skill vision-bench -a codex2Manually: put the files in place
.agents/skills/vision-bench/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add glebis/claude-skills --skill vision-bench -a cursor2Manually: put the files in place
.cursor/skills/vision-bench/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install glebis/claude-skills vision-bench2Manually: put the files in place
.github/skills/vision-bench/SKILL.md1Skills CLI (Vercel)
npx skills add glebis/claude-skills --skill vision-bench -a gemini-cli2Platform CLI
gemini skills install https://github.com/glebis/claude-skills --path vision-benchManually: put the files in place
.devin/skills/vision-bench/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add glebis/claude-skills --skill vision-bench -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about vision-bench
How do I install vision-bench in Claude Code?
Run npx skills add glebis/claude-skills --skill vision-bench -a claude-code. The skill file will be placed at ~/.claude/skills/vision-bench/SKILL.md.
What does vision-bench do?
It scores and compares images with one or more vision LLM judges against structured criteria. Results can be written as Markdown, JSON, or a table.
Which API keys does vision-bench use?
It supports OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, and OPENROUTER_API_KEY. Keys may come from secrets.enc.yaml through SOPS and age or from environment variables.
Source text
SKILL.md · MIT
--- name: vision-bench description: Score and compare images using vision LLMs as judges. YAML-defined criteria presets for 11 use cases (text-to-image, photorealism, document OCR, charts, UI, portrait, product, scientific, invoice, alt-text, artistic style). Supports OpenAI, Anthropic, Gemini, Mistral, and OpenRouter as judge providers. Keys auto-decrypted via SOPS + age. --- # Vision Bench — LLM Image Evaluation Compare images by scoring them with one or more vision LLM judges against structured rubric criteria. ## Quick Start ```bash # Install dependencies pip install pyyaml openai anthropic mistralai # Score a single image python bench.py image.png --criteria photorealism --judge gemini-3.5-flash-lite # Compare two AI-generated images python bench.py img_a.png img_b.png \ --criteria text_to_image \ --prompt "a fox in a snowy forest" \ --judge gpt-4o # Multi-judge consensus python bench.py img.png \ --criteria portrait \ --judges gpt-4o gemini-3.5-flash-lite claude-opus-4-5-20251022 # OpenRouter models (any vision-capable model) python bench.py img_a.png img_b.png \ --criteria artistic_style \ --judges "openrouter/meta-llama/llama-4-maverick" "openrouter/mistralai/pixtral-large-2411" # List all presets python bench.py --list-presets # Save report to file python bench.py img.png --criteria chart_analysis --save report.md ``` ## Presets | Preset | Use Case | |--------|----------| | `text_to_image` | Compare AI image generators (Midjourney, DALL-E, Flux) | | `photorealism` | How convincingly an image looks like a photo | | `artistic_style` | Style consistency, composition, color harmony | | `portrait` | AI-generated portrait quality and realism | | `product_photo` | E-commerce product image quality | | `document_ocr` | Document text extraction and layout understanding | | `chart_analysis` | Chart and data visualization comprehension | | `invoice` | Financial document field extraction accuracy | | `ui_screenshot` | App/web screenshot understanding | | `scientific` | Scientific/medical image accuracy | | `alt_text` | Accessibility image description quality | Custom criteria: pass any `.yaml` file as `--criteria path/to/my.yaml`. ## Judge Providers | Prefix | Provider | Example | |--------|----------|---------| | `gpt-`, `o1`, `o3`, `o4` | OpenAI | `gpt-4o` | | `claude-` | Anthropic | `claude-sonnet-4-5-20251022` | | `gemini-` | Google Gemini | `gemini-3.5-flash-lite` | | `pixtral-`, `mistral-`, `ministral-` | Mistral | `pixtral-12b-2409` | | `openrouter/` | OpenRouter (any model) | `openrouter/meta-llama/llama-4-maverick` | ## API Keys Keys are loaded from `secrets.enc.yaml` (SOPS + age encrypted) with fallback to environment variables. Supported keys: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `OPENROUTER_API_KEY` To encrypt your own keys: ```bash sops --config .sops.yaml --encrypt --input-type yaml --output-type yaml secrets.yaml > secrets.enc.yaml ``` ## Output Formats `--output markdown` (default) · `--output json` · `--output table` ## Files - `bench.py` — CLI entry point - `judge.py` — Multi-provider LLM judge logic - `report.py` — Report generation - `vault.py` — SOPS secrets decryption - `criteria/` — 11 YAML preset files - `.sops.yaml` — Age key config for encryption - `secrets.enc.yaml` — Encrypted API keys Use `--judge` or `--judges` to select another model explicitly. The Gemini path sends one image/text turn with JSON output and no tools, history, thinking or sampling settings. The 3.5 Flash-Lite default requires a live image/JSON smoke test and rubric comparison before relying on historical score comparability; offline request tests do not establish model quality.
© glebis · MIT · glebis/claude-skills