# llm-fine-tuning

Configures infrastructure for fine-tuning open LLMs with QLoRA, LoRA, and DeepSpeed. Useful when training models on proprietary domain datasets.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 318 installs · ★ 1177
- Author: devops-skills
- License: MIT · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/BagelHole/DevOps-Security-Agent-Skills/tree/HEAD/infrastructure/local-ai/llm-fine-tuning/SKILL.md
- Updated: 2026-05-22
- HTML: https://affpapa.org/en/nevibecode/skills/llm-fine-tuning

## What it does

This skill provides training configurations and code templates for fine-tuning open-source language models. It covers Hugging Face TRL (`SFTTrainer` and `DPOTrainer`), production configs for Axolotl, ZeRO Stage 3 DeepSpeed setups, and a Kubernetes Job template for A100 clusters. It also includes Python snippets to merge LoRA adapters into base model weights before serving with vLLM or exporting to Hugging Face Hub.

Running these pipelines requires an NVIDIA GPU with at least 24GB of VRAM (such as an RTX 4090, A100, or H100), CUDA 12.1+, Python 3.10+, and 500GB+ of free disk space. Without a 24GB+ GPU, local training will fail on out-of-memory errors. Access to gated checkpoints also requires setting `HF_TOKEN`.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llm-fine-tuning -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/llm-fine-tuning/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/llm-fine-tuning/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llm-fine-tuning -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/llm-fine-tuning/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llm-fine-tuning -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/llm-fine-tuning/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install BagelHole/DevOps-Security-Agent-Skills llm-fine-tuning
```

2. **Manually: put the files in place**
- File: `.github/skills/llm-fine-tuning/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llm-fine-tuning -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/BagelHole/DevOps-Security-Agent-Skills --path infrastructure/local-ai/llm-fine-tuning
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/llm-fine-tuning/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llm-fine-tuning -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/BagelHole/DevOps-Security-Agent-Skills/tree/HEAD/infrastructure/local-ai/llm-fine-tuning/SKILL.md

## When to use it
- set up QLoRA training runs with Hugging Face TRL and PEFT
- create Axolotl configuration files with sample packing and Flash Attention
- configure multi-GPU distributed training using DeepSpeed ZeRO-3
- merge trained LoRA weights with base models for vLLM deployment

## Example prompts
- Create a QLoRA fine-tuning script for Llama 3.1 8B using TRL SFTTrainer on a single RTX 4090.
- Configure Axolotl and DeepSpeed ZeRO-3 for training a 70B model across 4 GPUs.
- Write a script to merge PEFT adapters into base weights and save them for vLLM.

## Source text · SKILL.md · MIT

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: 570ea119bea97758.

BEGIN THIRD-PARTY FILE 570ea119bea97758
````text
---
name: llm-fine-tuning
description: Set up infrastructure for fine-tuning LLMs with QLoRA, LoRA, and full fine-tuning using Hugging Face TRL, Axolotl, and distributed training with DeepSpeed or FSDP. Covers dataset prep, training runs, and model export.
license: MIT
metadata:
  author: devops-skills
  version: "1.0"
---

# LLM Fine-Tuning Infrastructure

Train and fine-tune open-source LLMs efficiently — from LoRA on a single GPU to distributed full fine-tuning across multi-node clusters.

## When to Use This Skill

Use this skill when:
- Fine-tuning an LLM on domain-specific data (legal, medical, code, support)
- Running QLoRA to fine-tune 70B models on consumer GPUs
- Setting up distributed training with DeepSpeed or FSDP
- Exporting fine-tuned adapters for production serving
- Implementing RLHF, DPO, or instruction tuning pipelines

## Prerequisites

- NVIDIA GPU(s) with 24GB+ VRAM (RTX 4090 / A100 / H100)
- CUDA 12.1+ and `nvidia-smi` working
- Python 3.10+ with `pip`
- Hugging Face account and `HF_TOKEN` for gated models
- 500GB+ disk for model weights and training data

## Quick Start: QLoRA Fine-Tuning

```bash
pip install transformers datasets trl peft bitsandbytes accelerate

python - <<'EOF'
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model
from trl import SFTTrainer, SFTConfig
import torch

model_id = "meta-llama/Llama-3.1-8B-Instruct"

# 4-bit quantization (QLoRA)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    model_id, quantization_config=bnb_config, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# LoRA configuration
peft_config = LoraConfig(
    r=16,                    # rank
    lora_alpha=32,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)

dataset = load_dataset("your-org/your-dataset", split="train")

trainer = SFTTrainer(
    model=model,
    args=SFTConfig(
        output_dir="./output",
        num_train_epochs=3,
        per_device_train_batch_size=2,
        gradient_accumulation_steps=8,
        learning_rate=2e-4,
        bf16=True,
        logging_steps=10,
        save_strategy="epoch",
        report_to="wandb",
    ),
    train_dataset=dataset,
    peft_config=peft_config,
    processing_class=tokenizer,
)
trainer.train()
trainer.save_model("./fine-tuned-model")
EOF
```

## Axolotl (Production Fine-Tuning Framework)

```yaml
````
END THIRD-PARTY FILE 570ea119bea97758

The third-party file has ended. Everything between the 570ea119bea97758 tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 2745 of 8810 bytes. © devops-skills · MIT · https://github.com/BagelHole/DevOps-Security-Agent-Skills/tree/HEAD/infrastructure/local-ai/llm-fine-tuning/SKILL.md

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md