qwencloud-vision — Claude Code skill: image and video understanding

---
name: qwencloud-vision
description: "Understand images and videos with Qwen vision models. TRIGGER when: user wants to analyze, describe, or extract information from images or videos, OCR text extraction, chart/table reading, visual reasoning, multi-image comparison, screenshot understanding, video comprehension, or explicitly invokes this skill by name (e.g. use qwencloud-vision). DO NOT TRIGGER when: user wants to generate/create images (use qwencloud-image-generation), generate videos (use qwencloud-video-generation), text-only tasks without visual input, or non-Qwen vision tasks."
compatibility: "Requires Python 3.9+; curl is PAYG-only. Cursor: auto-loaded. Claude Code: read this skill's SKILL.md before first use."
---

# Qwen Vision (Image & Video Understanding)

Analyze images and videos using Qwen VL and QVQ models.
This skill is part of **qwencloud/qwencloud-ai**.

## Skill directory

Use this skill's internal files to execute and learn. Load reference files on demand when the default path fails or you need details.

| Location | Purpose |
|----------|---------|
| `scripts/analyze.py` | Image/video understanding, multi-image, thinking mode |
| `scripts/reason.py` | Visual reasoning (QVQ, chain-of-thought, streaming) |
| `scripts/ocr.py` | OCR text extraction |
| `scripts/vision_lib.py` | Shared helpers (base64, upload, streaming) |
| `references/execution-guide.md` | Fallback: curl, code generation |
| `references/curl-examples.md` | Curl for base64, multi-image, video, OCR |
| `references/visual-reasoning.md` | QVQ and thinking mode details |
| `references/prompt-guide.md` | Query prompt templates by task, thinking mode decision |
| `references/ocr.md` | OCR parameters and examples |
| `references/sources.md` | Official documentation URLs |

## Security

**NEVER output any API key or credential in plaintext.** Always use variable references (`$QWENCLOUD_API_KEY` in shell, `os.environ["QWENCLOUD_API_KEY"]` in Python). The scripts accept `QWENCLOUD_API_KEY`, then `QWEN_API_KEY`, then `DASHSCOPE_API_KEY`. Any check or detection of credentials must be **non-plaintext**: report only status (e.g. "set" / "not set", "valid" / "invalid"), never the value. Never display contents of `.env` or config files that may contain secrets.

**When the API key is not configured, NEVER ask the user to provide it directly.** Instead, help create a `.env` file with a placeholder (`QWENCLOUD_API_KEY=sk-your-key-here`) and instruct the user to replace it with their actual key from the [QwenCloud Console](https://home.qwencloud.com/api-keys). Only write the actual key value if the user explicitly requests it.

## Key Compatibility

Scripts support both **standard QwenCloud API keys** (`sk-...`) and **Token Plan keys** (`sk-sp-...`). Token Plan keys are automatically routed to the Token Plan endpoint for supported multimodal models — see the [Token Plan model catalog](https://alioth-intl.alicdn.com/skills-info/models/references/qwencloud-token-plan-models.md). If CDN access fails, use the [local fallback](cdn/references/qwencloud-token-plan-models.md).

**Token Plan: do not use curl; always use the bundled Python scripts.**

Coding Plan keys (also `sk-sp-` prefix but purchased via Coding Plan subscription) cannot be used for direct API calls. The scripts detect key type at startup and route accordingly. If qwencloud-ops-auth is installed, see its `references/codingplan.md` for current model coverage, endpoint mapping, and error details.

Detect the API key type without exposing the key:

```bash
python3 -c "
import sys; sys.path.insert(0, 'scripts')
from qwencloud_lib import detect_api_key_type
print(detect_api_key_type('scripts/qwencloud_lib.py'))
"
```

| Output | Meaning |
|--------|---------|
| `token-plan` | Token Plan key detected (`sk-sp-` prefix) |
| `payg` | Standard PAYG key detected |
| `not-set` | No API key found in environment |

## Model Selection

> **🚫 CRITICAL — Never override user-specified parameters.** If the user explicitly specifies a model (in prompt or request JSON), you MUST use exactly that model. Do NOT:
> - Replace it with a "better suited" or newer model (e.g., swapping the user's `qwen3-vl-plus` for `qwen3.7-plus`/`qwen3.8-max` because newer docs "prefer" it)
> - Add parameters the user did not ask for (`enable_thinking`, `thinking_budget`, `vl_high_resolution_images`, `detail`), except that the bundled scripts must send `enable_thinking: false` when the user requests `json_mode`/`schema` and omitted the setting
> - "Optimize" any explicit user choice
>
> Selection guidance below applies **only when the user has NOT specified a model**.

Before selecting, recommending, or defaulting a model, fetch and read the current [QwenCloud vision model catalog](https://alioth-intl.alicdn.com/skills-info/models/references/qwencloud-vision-models.md). It contains the model list, basic model information, task recommendations, compatibility notes, and defaults. If CDN access fails, use the [local fallback](cdn/references/qwencloud-vision-models.md).

1. **User specified a model** → **MANDATORY: use that exact model** — do not substitute or "optimize" it. Do not add unrequested parameters except for the structured-output compatibility behavior described above; reject a thinking-only model instead of silently replacing it.
2. **Consult the qwencloud-model-selector skill** when model choice depends on requirement, scenario, or pricing.
3. **No signal, clear task** → use the default and task-specific alternatives from the model catalog. Dedicated-script defaults remain independent of key type; Token Plan users should select a compatible model from the catalog when a default is unavailable on their plan.

> **⚠️ Important**: The model catalog is a **point-in-time snapshot** and may be outdated. Model availability
> changes frequently. **Always check the [official model list](https://www.qwencloud.com/models)
> for the authoritative, up-to-date catalog before making model decisions.**

> **Model details**: For more information about a specific model, direct the user to its detail page: `https://www.qwencloud.com/models/<model-name>` (replace `<model-name>` with the exact model ID, e.g. `qwen3.6-plus` → https://www.qwencloud.com/models/qwen3.6-plus). NEVER modify or guess the model name in the URL.

> **Dynamic model queries**: If the **qwencloud-model-selector** skill or **QwenCloud CLI** (`qwencloud models info <model>`) is available, use it for real-time model data. CLI requires authentication — see the **qwencloud-usage** skill for login flow.

## Execution

### Prerequisites

- **API Key**: Check `QWENCLOUD_API_KEY`, `QWEN_API_KEY`, then `DASHSCOPE_API_KEY` using a **non-plaintext** check only (e.g. in shell:
  `[ -n "$QWENCLOUD_API_KEY" ]`; report only "set" or "not set", never the key value). If not set: run the *
  *qwencloud-ops-auth** skill if available; otherwise guide the user to obtain a key from [QwenCloud Console](https://home.qwencloud.com/api-keys) and set it via `.env` file (
  `echo 'QWENCLOUD_API_KEY=sk-your-key-here' >> .env` in project root or current directory) or environment variable. The
  script searches for `.env` in the current working directory and the project root. Skills may be installed
  independently — do not assume qwencloud-ops-auth is present.
  **Note**: The script auto-loads `.env` from the current directory and the project root (in addition to any exported environment
  variable). A shell check showing `$QWENCLOUD_API_KEY` as "not set" does NOT mean the script will fail — it may still find the
  key in `.env`. Treat the shell check as informational only; the authoritative test is simply running the script (it exits with
  a clear error if no key is found anywhere).
- Python 3.9+ (stdlib only, **no pip install needed**)

### Environment Check

Before first execution, verify Python is available:

```bash
python3 --version  # must be 3.9+
```

If `python3` is unavailable or below 3.9, PAYG may use **Path 2 (curl)**; Token Plan must install Python 3.9+ instead.

### Default: Run Script

**Script path**: Scripts are in the `scripts/` subdirectory **of this skill's directory** (the directory containing this SKILL.md). **You MUST first locate this skill's installation directory, then ALWAYS use the full absolute path to execute scripts.** Do NOT assume scripts are in the current working directory. Do NOT use `cd` to switch directories before execution. Shared infrastructure lives in `scripts/vision_lib.py`.

**Execution note:** Run all scripts in the **foreground** — wait for stdout; do not background.

**Discovery:** Run `python3 <this-skill-dir>/scripts/analyze.py --help` (or `reason.py`, `ocr.py`) first to see all available arguments.

| Script | Purpose | Default Model |
|--------|---------|---------------|
| `scripts/analyze.py` | Image understanding, multi-image, video, thinking mode, high-res | Read the current default from the model catalog above |
| `scripts/reason.py` | Visual reasoning with chain-of-thought, video reasoning (always streaming) | Read the current default from the model catalog above |
| `scripts/ocr.py` | OCR text extraction from documents, receipts, tables | Read the current default from the model catalog above |

**Input type fields** (use exactly one in `--request` JSON):

| Field | Use for | Example |
|-------|---------|--------|
| `"image"` | Single image (URL or local path) | `"image": "photo.jpg"` |
| `"images"` | Multi-image comparison (array) | `"images": ["a.jpg", "b.jpg"]` |
| `"video"` | Video file (URL or local path) | `"video": "clip.mp4"` |
| `"video_frames"` | Video as frame array | `"video_frames": ["f1.jpg", "f2.jpg"]` |

> **⚠️ Common mistake**: Do NOT use `"image"` for video files — use `"video"` instead.

```bash
# Image analysis
python3 <this-skill-dir>/scripts/analyze.py \
  --request '{"prompt":"What is in this image?","image":"https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"}' \
  --output output/qwencloud-vision/result.json --print-response

# Video analysis (local file — add --upload-files for files >= 7 MB)
python3 <this-skill-dir>/scripts/analyze.py \
  --request '{"prompt":"Describe what happens in this video","video":"clip.mp4"}' \
  --upload-files --print-response

python3 <this-skill-dir>/scripts/reason.py \
  --request '{"prompt":"Solve this math problem step by step","image":"problem.png"}' \
  --print-response

python3 <this-skill-dir>/scripts/ocr.py \
  --request '{"image":"invoice.jpg"}' \
  --print-response
```

| Argument | Description |
|----------|-------------|
| `--request '{...}'` | JSON request body |
| `--file path.json` | Load request from file |
| `--output path` | Save response JSON to path |
| `--print-response` | Print response to stdout |
| `--stream` | Enable streaming (auto for thinking/QVQ) |
| `--upload-files` | Upload local files to temp storage (for files > 7 MB) |
| `--schema path.json` | JSON Schema for structured extraction |

### Verify Result

- Exit code `0` + output contains valid JSON with `choices` field → **success**
- Non-zero exit, HTTP error, empty response, or JSON with `"code"`/`"message"` error → **fail**
- **Post-execution check**: When `--output` is used, verify the response JSON file exists and contains expected content
- **MANDATORY — stderr signal check**: After confirming the result, scan the command's stderr output for `[ACTION_REQUIRED]` or `[UPDATE_AVAILABLE]`. If either signal is present, you **MUST** follow the instructions in [Update Check](#update-check-mandatory-post-execution) below before responding to the user.

### On Failure

If scripts fail, match the error output against the diagnostic table below to determine the resolution. If no match, read [execution-guide.md](references/execution-guide.md) for alternative paths: curl commands (Path 2), code generation (Path 3), and autonomous resolution (Path 5).

**If Python is not available at all** → PAYG may use Path 2 (curl); Token Plan must install Python 3.9+.

| Error Pattern                    | Diagnosis                        | Resolution                                                                                                                                                  |
|----------------------------------|----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `command not found: python3`     | Python not on PATH               | Try `python` or `py -3`; install Python 3.9+ if missing                                                                                                     |
| `Python 3.9+ required`           | Script version check failed      | Upgrade Python to 3.9+                                                                                                                                      |
| `SyntaxError` near type hints    | Python < 3.9                     | Upgrade Python to 3.9+                                                                                                                                      |
| `QWENCLOUD_API_KEY/QWEN_API_KEY/DASHSCOPE_API_KEY not found` | Missing API key | Obtain key from [QwenCloud Console](https://home.qwencloud.com/api-keys); add to `.env`: `echo 'QWENCLOUD_API_KEY=sk-...' >> .env`; or run **qwencloud-ops-auth** if available |
| `HTTP 401`                       | Invalid or mismatched key        | Run **qwencloud-ops-auth** (non-plaintext check only); verify key is valid                                                                                  |
| `SSL: CERTIFICATE_VERIFY_FAILED` | SSL cert issue (proxy/corporate) | macOS: run `Install Certificates.command`; else set `SSL_CERT_FILE` env var                                                                                 |
| `URLError` / `ConnectionError`   | Network unreachable              | Check internet; set `HTTPS_PROXY` if behind proxy                                                                                                           |
| `HTTP 429`                       | Rate limited                     | Wait and retry with backoff                                                                                                                                 |
| `HTTP 5xx`                       | Server error                     | Retry with backoff                                                                                                                                          |
| `PermissionError`                | Can't write output               | Use `--output` to specify writable directory                                                                                                                |

## File Input

The API accepts: **HTTP/HTTPS URL**, **Base64 data URI**, and **`oss://` URL**. Local file paths are NOT directly supported — scripts handle conversion automatically. **Pass local paths directly; no manual upload step needed.**

**Large file rule: If the local file is >= 7 MB, always add `--upload-files`.** Base64 encoding inflates size by ~33% and will exceed the 10 MB API limit. Small files (including short video clips < 7 MB) can use the default base64 path.

| Method | When to use | How |
|--------|-------------|-----|
| **Online URL** | File already hosted | Pass URL directly — **preferred for large files** |
| **Base64** (default) | Local files < 7 MB (images or short video clips) | Script auto-converts to `data:` URI |
| **Temp upload** | Local files >= 7 MB | Add `--upload-files` flag → uploads to DashScope temp storage (`oss://` URL, 48h TTL) |

> **Production**: Default temp storage has **48h TTL** and **100 QPS upload limit** — not suitable for production, high-concurrency, or load-testing. To use your own OSS bucket, set `QWEN_TMP_OSS_BUCKET` and `QWEN_TMP_OSS_REGION` in `.env`, install `pip install alibabacloud-oss-v2`, and provide credentials via `QWEN_TMP_OSS_AK_ID` / `QWEN_TMP_OSS_AK_SECRET` or the standard `OSS_ACCESS_KEY_ID` / `OSS_ACCESS_KEY_SECRET`. Use a RAM user with least-privilege (`oss:PutObject` + `oss:GetObject` on target bucket only). The `--upload-files` flag is still required for vision scripts to trigger upload. If qwencloud-ops-auth is installed, see its `references/custom-oss.md` for the full setup guide.

## Input from Other Skills

When the input file comes from another skill's output (e.g., image-gen, video-gen):
- **Pass the URL directly** (e.g., `"image": "<image_url from image-gen>"`) — do NOT download the URL first
- Downloading and re-passing as a local path wastes bandwidth and triggers unnecessary base64 encoding or OSS upload
- All URL types are supported: `https://`, `oss://`, `data:`

## Thinking Mode

Use the vision model catalog linked above for current thinking defaults and model compatibility. Do NOT set `enable_thinking` unless the user explicitly asks to change it — neither to enable it "because the task looks complex" nor to disable it "because the task looks simple". The sole compatibility exception is structured output: `json_mode` and `schema` require non-thinking mode, so the bundled scripts automatically send `enable_thinking: false` when the setting is omitted and reject explicit `true`. Thinking-only models cannot be used for structured output.

See [visual-reasoning.md](references/visual-reasoning.md) for details.

## OCR

Use the vision model catalog linked above for the current OCR default, alternatives, and model capabilities. See [ocr.md](references/ocr.md) for parameters and examples.

## Input Limits

**Images**: BMP/JPEG/PNG/TIFF/WEBP/HEIC. Min 10px sides, aspect ratio <= 200:1. Check the model catalog above for model-specific size limits.

**Videos**: MP4/AVI/MKV/MOV/FLV/WMV. Check the model catalog above for model-specific duration and size limits. fps range [0.1, 10], default 2.0.

## Error Handling

| HTTP | Meaning | Action |
|------|---------|--------|
| 401 | Invalid or missing API key | Run **qwencloud-ops-auth** if available; else prompt user to set key (non-plaintext check only) |
| 400 | Bad request (invalid format) | Verify messages format and image URL/format |
| 429 | Rate limited | Retry with exponential backoff |
| 5xx | Server error | Retry with exponential backoff |

> **Usage & billing**: Use the **qwencloud-usage** skill to check usage, free tier quota, and billing directly. Alternatively, the user can visit the QwenCloud console:
> [Usage Analytics](https://home.qwencloud.com/analytics) |
> [Pay-as-you-go Billing](https://home.qwencloud.com/billing/pay-as-you-go) |
> [Coding Plan Billing](https://home.qwencloud.com/billing/coding-plan)
>
> **NEVER fabricate, guess, or construct usage/billing/console URLs.** Only provide the exact links listed in this skill. If a URL is not listed here, do not invent one.

## Output Location

Prefer the **current working directory**. Default subdirectory: `./output/qwencloud-vision/`.

**Write prohibition**: NEVER write output files into this skill's installation directory or any `skills/` hierarchy. All generated content must go to `output/` under the current working directory or a user-specified path.

## Token Plan Support

Token Plan keys (`sk-sp-...`) are supported for multimodal models with vision capability.

### Supported vision models

Fetch and read the current [Token Plan model catalog](https://alioth-intl.alicdn.com/skills-info/models/references/qwencloud-token-plan-models.md), then use an exact listed vision-capable model. If CDN access fails, use the [local fallback](cdn/references/qwencloud-token-plan-models.md).

These models use the OpenAI-compatible chat API with multimodal message format (image_url content type).

### Defaults

When no model is supplied, each script uses its configured default regardless of key type. Check the vision and Token Plan catalogs for current compatibility. An explicitly supplied request model always takes precedence.

### Not supported via Token Plan

Models not listed as vision-capable in the Token Plan model catalog require a PAYG key. Only with the user's explicit consent, suggest compatible alternatives from the vision catalog. Never silently swap a user-specified model because of key-type restrictions; present the options and let the user choose.

### Required header

`User-Agent: qwencloud-skills` is automatically included.

## Update Check (MANDATORY Post-Execution)

After script execution, check stderr for action signals. The script automatically detects update status and emits structured signals. You MUST act on them. This section also applies when the user explicitly asks to check for updates.

### Signal: `[ACTION_REQUIRED] qwencloud-update-check skill is not installed`

When stderr contains this signal, you MUST present the following options to the user and **wait for their response before proceeding**. Do NOT silently skip this.

> The update-check skill is not installed. It keeps you informed about new qwencloud/qwencloud-ai features and bug fixes.
>
> 1) **Install now** — I'll run the install command for you
> 2) **Skip for now** — do nothing this time
> 3) **Never remind me** — permanently suppress this prompt

After the user responds:
- User chooses **1** → run: `npx skills add QwenCloud/qwencloud-ai --skill qwencloud-update-check -y`, then run: `python3 <this-skill-scripts-dir>/gossamer.py --dismiss`
- User chooses **2** → run: `python3 <this-skill-scripts-dir>/gossamer.py --dismiss`, then continue normally
- User chooses **3** → run: `python3 <this-skill-scripts-dir>/gossamer.py --never-install`, then continue normally

### Signal: `[UPDATE_AVAILABLE]`

When stderr contains this signal, you MUST append a brief update notice to your response, including the version info and the update command shown in the stderr output.

### No signal in stderr

If stderr contains neither `[ACTION_REQUIRED]` nor `[UPDATE_AVAILABLE]`, no action is needed — the skill is installed and up to date (or cached within 24h).

### Explicit user request

When the user explicitly asks to check for updates (e.g. "check for updates", "check version"):
1. Look for `qwencloud-update-check/SKILL.md` in sibling skill directories.
2. If found — run: `python3 <qwencloud-update-check-dir>/scripts/check_update.py --print-response` and report the result.
3. If not found — present the install options above.

## References

- [execution-guide.md](references/execution-guide.md) — Fallback paths (curl, code generation, autonomous)
- [curl-examples.md](references/curl-examples.md) — Curl templates (base64, multi-image, video, OCR)
- [api-guide.md](references/api-guide.md) — API supplementary guide
- [visual-reasoning.md](references/visual-reasoning.md) — QVQ visual reasoning guide
- [ocr.md](references/ocr.md) — Qwen-VL-OCR text extraction guide
- [sources.md](references/sources.md) — Official documentation URLs