# runai-diagnose-failed-workload

runai-diagnose-failed-workload diagnoses crashed, stopped, OOMKilled, and startup-failed Run:ai workloads using structured evidence and read-only cluster queries.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf (Devin Desktop), Claude.ai · ★ 4106 repo stars
- Author: Run:ai
- License: not stated · policy=link-only · description only; the file stays with the author
- Source: https://github.com/NVIDIA/skills/tree/HEAD/skills/runai-diagnose-failed-workload/SKILL.md
- Updated: 2026-10-23
- Security check: automated, 2026-11-16 (text) — no signs of malicious code found.
- HTML: https://affpapa.org/en/nevibecode/skills/runai-diagnose-failed-workload

## What it does

The skill investigates terminated or broken Run:ai jobs, focusing on issues like `CrashLoopBackOff`, container startup failures, volume mount issues, and `ImagePullBackOff`. It gathers context from workload specifications, metrics, and pod events. The workflow remains strictly read-only: it reports findings and suggests remedies without editing, stopping, or restarting resources.

Inspection operates on two tiers. Tier 1 requires an MCP setup with read access to the Run:ai service account for specifications, events, and metrics. Tier 2 is optional, relying on read-only Kubernetes credentials to pull prior container logs, node descriptions, and namespace events via `kubectl`.

Every run requires an explicit `cluster_id`. You must provide either a direct Run:ai UUID or both the workload name and project name. Normal completions, suspended jobs, and scheduling preemption fall outside this skill's scope.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill runai-diagnose-failed-workload -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/runai-diagnose-failed-workload/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/runai-diagnose-failed-workload/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill runai-diagnose-failed-workload -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/runai-diagnose-failed-workload/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill runai-diagnose-failed-workload -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/runai-diagnose-failed-workload/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install NVIDIA/skills skills/runai-diagnose-failed-workload
```

2. **Manually: put the files in place**
- File: `.github/skills/runai-diagnose-failed-workload/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add NVIDIA/skills --skill runai-diagnose-failed-workload -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/NVIDIA/skills --path skills/runai-diagnose-failed-workload
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/runai-diagnose-failed-workload/SKILL.md` — The legacy .windsurf/skills/ path still works

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/NVIDIA/skills/tree/HEAD/skills/runai-diagnose-failed-workload/SKILL.md

## When to use it
- categorize terminal failures into root causes like app crashes or OOM terminations
- inspect failed pod startups caused by invalid images or init scripts
- trace storage issues from persistent volume claim mounts
- separate worker node outages from application errors

## Example prompts
- Diagnose why workload `w-98765` failed on cluster `ai-prod-1`
- Check the CrashLoopBackOff cause for `finetune-llm` in project `research` on cluster `k8s-gpu`
- Find why job `eval-batch` stopped in project `testing` on cluster `dev-cluster` over the last `48h`

## Similar
- [pulumi-debug-failed-operation](https://affpapa.org/en/nevibecode/skills/pulumi-debug-failed-operation.md) — skill · 405 installs
- [gke-workload-security](https://affpapa.org/en/nevibecode/skills/gke-workload-security.md) — skill · 5000 installs
- [debug-openshell-cluster](https://affpapa.org/en/nevibecode/skills/debug-openshell-cluster.md) — skill · 739 installs
- [crontap-debug-failed-runs](https://affpapa.org/en/nevibecode/skills/crontap-debug-failed-runs.md) — skill · 1300 installs
- [diagnose-deployment](https://affpapa.org/en/nevibecode/skills/diagnose-deployment.md) — skill · ★ 1167 repo stars
- [physical-ai-infrastructure-setup-and-resilient-scaling](https://affpapa.org/en/nevibecode/skills/physical-ai-infrastructure-setup-and-resilient-scaling.md) — skill · ★ 8052 repo stars

From the same set: [nvidia/skills](https://affpapa.org/en/nevibecode/by/nvidia/skills.md) — 312 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md
