databricks-serverless-migration — Claude Code skill
A skill that checks Databricks notebooks and jobs for serverless compatibility, names the broken parts and walks the migration step by step.
npx skills add databricks/databricks-agent-skills --skill databricks-serverless-migration -a claude-codeWhat databricks-serverless-migration does
Point the agent at your notebooks, jobs or pipelines and it tells you whether the workload runs on serverless compute. It first gathers the classic cluster setup: instance type, DBR version, libraries, init scripts, networking. Then it reads the code and sorts every finding into one of three buckets. Things Databricks is still adding support for, things you have to change yourself, and things that will never run on serverless.
The fixes are specific. RDD calls get rewritten to the DataFrame API, DBFS paths move to UC Volumes, Hive Metastore tables go to Unity Catalog, ProcessingTime triggers become AvailableNow. Cluster-level Spark configs are dropped, since serverless tunes them on its own. Scala JAR jobs take a separate path: build against 2.13.16, and depend on databricks-connect with % Provided scope rather than bundling spark-core.
You need databricks CLI v0.292.0 or newer. The skill also leans on the parent databricks-core skill for authentication and profile selection, so those CLI steps stall without it. Upgrading classic DBR between versions is a different job with a different compatibility surface, and this skill does not cover it. When a workload spans several notebooks, the agent handles them one at a time because the context window fills up fast otherwise.
What it helps with
- check whether an existing notebook runs on serverless compute
- rewrite RDD code with the DataFrame API
- move DBFS paths onto UC Volumes
- rebuild a Scala JAR job for serverless
- strip Spark configs that serverless sets itself
Example prompts for the agent
- Check
etl_daily.pyfor anything that breaks on serverless compute - Rewrite the DBFS reads in this notebook to use UC Volumes
- Rebuild
build.sbtfor thespark_jar_taskjob so it runs on serverless
How to install databricks-serverless-migration
1Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-serverless-migration -a claude-code2Manually: put the files in place
~/.claude/skills/databricks-serverless-migration/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/databricks-serverless-migration/
1Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-serverless-migration -a codex2Manually: put the files in place
.agents/skills/databricks-serverless-migration/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-serverless-migration -a cursor2Manually: put the files in place
.cursor/skills/databricks-serverless-migration/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install databricks/databricks-agent-skills databricks-serverless-migration2Manually: put the files in place
.github/skills/databricks-serverless-migration/SKILL.md1Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-serverless-migration -a gemini-cli2Platform CLI
gemini skills install https://github.com/databricks/databricks-agent-skills --path skills/databricks-serverless-migrationManually: put the files in place
.devin/skills/databricks-serverless-migration/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add databricks/databricks-agent-skills --skill databricks-serverless-migration -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about databricks-serverless-migration
How do I install databricks-serverless-migration in Claude Code?
Run npx skills add databricks/databricks-agent-skills --skill databricks-serverless-migration -a claude-code, then open ~/.claude/skills/databricks-serverless-migration/SKILL.md.
What does databricks-serverless-migration do?
It reviews Databricks code against the serverless compute architecture and Spark Connect: RDD usage, DBFS paths, Hive Metastore, streaming triggers, init scripts, JAR tasks. Each finding comes with a concrete fix.
What does the skill need to run?
databricks CLI v0.292.0 or newer, plus the parent databricks-core skill for authentication and profile selection.