Skip to main content
skill CX Codex

llm-integration · yonatangross — Claude Code skill: what it does, how to install

llm-integration — a collection of 20 production rules for LLM workflows: strict function calling, SSE streaming endpoints, local Ollama deployments, and LoRA tuning.

MIT 218 installs ★ 305 updated 2026-11-21

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add yonatangross/orchestkit --skill llm-integration -a claude-code
Open on GitHub

What llm-integration does

This skill bundles engineering blueprints for embedding LLMs into backend services. It organizes 20 rules across seven domains: function calling with strict schemas, Server-Sent Events streaming in FastAPI, local model hosting via Ollama, and QLoRA fine-tuning with Unsloth. It also includes rules for context window management, prompt caching, and evaluation using RAGAS and LLM-as-judge setups.

Individual rules live in the rules/ directory and load on demand. For tool execution, the patterns enforce strict input validation using Pydantic or Zod and require execution errors to return as tool output rather than unhandled server exceptions.

The skill explicitly restricts fine-tuning: it should be used only after exhausting prompt engineering and RAG. Training requires a curated dataset of at least 1000 quality records. Integration requires Claude Code version 2.1.277 or newer.

What it helps with

  • define strict schemas for parallel and single function calling
  • build SSE streaming endpoints with backpressure handling
  • connect local models in Ollama with LangChain bindings
  • set up QLoRA fine-tuning pipelines using Unsloth
  • manage token budgets and context caching across conversation turns

Example prompts for the agent

  • Configure a FastAPI endpoint with SSE streaming for model responses
  • Draft a strict mode tool schema for function calling with input validation
  • Set up a QLoRA fine-tuning script for Llama 3 using Unsloth

How to install llm-integration

1Skills CLI (Vercel)

npx skills add yonatangross/orchestkit --skill llm-integration -a claude-code

2Manually: put the files in place

~/.claude/skills/llm-integration/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/llm-integration/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about llm-integration

How do I install llm-integration in Claude Code?

Run npx skills add yonatangross/orchestkit --skill llm-integration -a claude-code.

What does the llm-integration skill do?

It provides 20 modular rules covering tool definitions, streaming responses, local inference, context optimization, prompt patterns, and fine-tuning.

What prerequisite does the skill enforce for fine-tuning?

It requires exhausting prompt design and RAG first, along with a prepared dataset of at least 1000 high-quality training examples.

Source text

SKILL.md · MIT
---
name: llm-integration
license: MIT
compatibility: "Claude Code 2.1.277+."
description: LLM integration patterns for function calling, streaming responses, local inference with Ollama, and fine-tuning customization. Use when implementing tool use, SSE streaming, local model deployment, LoRA/QLoRA fine-tuning, or multi-provider LLM APIs.
user-invocable: false
metadata:
  archive-candidate: "pending an outside usage signal"
  owner-agent: "llm-integrator"
  category: mcp-enhancement
  version: "2.0.0"
  author: "OrchestKit"
  complexity: "medium"
  tags: "llm, function-calling, streaming, ollama, fine-tuning, lora, tool-use, local-inference"
allowed-tools: "Read Glob Grep WebFetch WebSearch"
---

# LLM Integration

Patterns for integrating LLMs into production applications: tool use, streaming, local inference, and fine-tuning. Each category has individual rule files in `rules/` loaded on-demand.

## Quick Reference

| Category | Rules | Impact | When to Use |
|----------|-------|--------|-------------|
| [Function Calling](#function-calling) | 3 | CRITICAL | Tool definitions, parallel execution, input validation |
| [Streaming](#streaming) | 3 | HIGH | SSE endpoints, structured streaming, backpressure handling |
| [Local Inference](#local-inference) | 3 | HIGH | Ollama setup, model selection, GPU optimization |
| [Fine-Tuning](#fine-tuning) | 3 | HIGH | LoRA/QLoRA training, dataset preparation, evaluation |
| [Context Optimization](#context-optimization) | 2 | HIGH | Window management, compression, caching, budget scaling |
| [Evaluation](#evaluation) | 2 | HIGH | LLM-as-judge, RAGAS metrics, quality gates, benchmarks |
| [Prompt Engineering](#prompt-engineering) | 4 | HIGH | CoT, few-shot, versioning, DSPy optimization, ReAct, cost optimization |

**Total: 20 rules across 7 categories**

## Quick Start

```python
# Function calling: strict mode tool definition
tools = [{
    "type": "function",
    "function": {
        "name": "search_documents",
        "description": "Search knowledge base",
        "strict": True,
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search query"},
                "limit": {"type": "integer", "description": "Max results"}
            },
            "required": ["query", "limit"],
            "additionalProperties": False
        }
    }
}]
```

```python
# Streaming: SSE endpoint with FastAPI
@app.get("/chat/stream")
async def stream_chat(prompt: str):
    async def generate():
        async for token in async_stream(prompt):
            yield {"event": "token", "data": token}
        yield {"event": "done", "data": ""}
    return EventSourceResponse(generate())
```

```python
# Local inference: Ollama with LangChain
llm = ChatOllama(
    model="deepseek-r1:70b",
    base_url="http://localhost:11434",
    temperature=0.0,
    num_ctx=32768,
)
```

```python
# Fine-tuning: QLoRA with Unsloth
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B",
    max_seq_length=2048, load_in_4bit=True,
)
model = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=32)
```

## Function Calling

Enable LLMs to use external tools and return structured data. Use strict mode schemas (2026 best practice) for reliability. On Claude keep the `tools` array stable across turns so the prompt cache holds; add tools mid-session with an inline `tool_addition` (beta `inline-tools-2026-09-15`, Claude API only, available on Sonnet 5.5 but not on Sonnet 5) instead of swapping subsets (https://platform.claude.com/docs/en/build-with-claude/mid-conversation-system-messages#mid-conversation-tool-changes). For large tool sets, use a non-deferred tool search tool with `defer_loading: true` on the rest, never `defer_loading` alone (a request cannot have every tool deferred). Forced `tool_choice` (`any` or a named tool) returns a 400 on Sonnet 5.5, Opus 5.5, Fable 5.1 and Mythos 5.1; use `auto` with `strict: true` tools. Validate all inputs with Pydantic/Zod, and return errors as tool results. <!-- model-recency-ok: per-model contrast, Sonnet 5 lacks what Sonnet 5.5 has (platform docs 2026-09-28) -->

- `calling-tool-definition.md` -- Strict mode schemas, OpenAI/Anthropic formats, LangChain binding
- `calling-parallel.md` -- Parallel tool execution, asyncio.gather, strict mode constraints
- `calling-validation.md` -- Input validation, error handling, tool execution loops

## Streaming

Deliver LLM responses in real-time for better UX. Use SSE for web, WebSocket for bidirectional. Handle backpressure with bounded queues.

- `streaming-sse.md` -- FastAPI SSE endpoints, frontend consumers, async iterators
- `streaming-structured.md` -- Streaming with tool calls, partial JSON parsing, chunk accumulation
- `streaming-backpressure.md` -- Backpressure handling, bounded buffers, cancellation

## Local Inference

Run LLMs locally with Ollama for cost savings (93% vs cloud), privacy, and offline development. Pre-warm models, use provider factory for cloud/local switching.

- `local-ollama-setup.md` -- Installation, model pulling, environment configuration
- `local-model-selection.md` -- Model comparison by task, hardware profiles, quantization
- `local-gpu-optimization.md` -- Apple Silicon tuning, keep-alive, CI integration

## Fine-Tuning

Customize LLMs with parameter-efficient techniques. Fine-tune ONLY after exhausting prompt engineering and RAG. Requires 1000+ quality examples.

- `tuning-lora.md` -- LoRA/QLoRA configuration, Unsloth training, adapter merging
- `tuning-dataset-prep.md` -- Synthetic data generation, quality validation, deduplication
- `tuning-evaluation.md` -- DPO alignment, evaluation metrics, anti-patterns

## Context Optimization

Manage context windows, compression, and attention-aware positioning. Optimize for tokens-per-task.

- `context-window-management.md` -- Five-layer architecture, anchored summarization, compression triggers
- `context-caching.md` -- Just-in-time loading, budget scaling, probe evaluation, CC 2.1.32+

## Evaluation

Evaluate LLM outputs with multi-dimension scoring, quality gates, and benchmarks.

- `evaluation-metrics.md` -- LLM-as-judge, RAGAS metrics, hallucination detection
- `evaluation-benchmarks.md` -- Quality gates, batch evaluation, pairwise comparison

## Prompt Engineering

Design, version, and optimize prompts for production LLM applications.

- `prompt-design.md` -- Chain-of-Thought, few-shot learning, pattern selection guide
- `prompt-testing.md` -- Langfuse versioning, DSPy optimization, A/B testing, self-consistency
- `prompt-react-pattern.md` -- ReAct loop for tool-using agents, thought-action-observation format
- `prompt-optimization.md` -- Token reduction, cost optimization, model tiering, prompt spec format

## Upstream coverage (do not restate)

These topics are covered by their vendors' own documentation. This skill points
at them instead of teaching them; the rules above keep only our floors, ceilings
and scars. Our delta on all of it is in `references/ork-delta.md`.

| Topic | First-party source |
|-------|--------------------|
| Strict-mode tool schemas, structured outputs | https://platform.openai.com/docs/guides/function-calling |
| Anthropic `input_schema` / `tool_use` | https://docs.claude.com/en/docs/agents-and-tools/tool-use/overview |
| SSE client mechanics, reconnection, cancellation | https://developer.mozilla.org/en-US/docs/Web/API/EventSource |
| LoRA / QLoRA config, target modules, adapter merging | https://huggingface.co/docs/peft/developer_guides/lora |
| Unsloth training loop, 4-bit loading | https://docs.unsloth.ai/get-started/fine-tuning-llms-guide |
| DPO, preference pairs, beta tuning, RLHF comparison | https://huggingface.co/docs/trl/dpo_trainer |
| SFT dataset formats (Alpaca, ChatML) | https://huggingface.co/docs/trl/sft_trainer |
| Embedding similarity for dataset dedup | https://sbert.net/ |
| Fine-tune vs prompt vs RAG decision framework | https://platform.openai.com/docs/guides/optimizing-llm-accuracy |
| Vendor token pricing (never hardcode it here) | https://platform.openai.com/docs/pricing |

## Supporting Files

- `references/ork-delta.md` -- our delta: scars, house ceilings, retired-file provenance
- `references/model-selection.md` -- local model comparison by task and hardware
- `scripts/create-lora-config.md` -- LoRA config scaffold with auto-detected model type

## Key Decisions

| Decision | Recommendation |
|----------|----------------|
| Tool schema mode | `strict: true` (2026 best practice) |
| Tool count | Load roughly 5-15 up front; defer the rest behind a non-deferred tool search tool |
| Streaming protocol | SSE for web, WebSocket for bidirectional |
| Buffer size | 50-200 tokens |
| Local model (reasoning) | `deepseek-r1:70b` |
| Local model (coding) | `qwen2.5-coder:32b` |
| Fine-tuning approach | LoRA/QLoRA (try prompting first) |
| LoRA rank | 16-64 typical |
| Training epochs | 1-3 (more risks overfitting) |
| Context compression | Anchored iterative (60-80%) |
| Compress trigger | 70% utilization, target 50% |
| Judge model | `claude-haiku-5-5` (cost tier), `gpt-5.5`, or `gemini-3.8-flash` (Google's GA general-purpose model and a third vendor for the different-model rule; price lives in `models.vocab.json`, not here) |
| Quality threshold | 0.7 production, 0.6 drafts |
| Few-shot examples | 3-5 diverse, representative |
| Prompt versioning | Langfuse with labels |
| Auto-optimization | DSPy MIPROv2 |

## Related Skills

- `ork-extras:rag-retrieval` -- Embedding patterns, when RAG is better than fine-tuning
- `agent-loops` -- Multi-step tool use with reasoning
- `llm-evaluation` -- Evaluate fine-tuned and local models
- `langfuse-observability` -- Track training experiments

## Capability Details

### function-calling
**Keywords:** tool, function, define tool, tool schema, function schema, strict mode, parallel tools
**Solves:**
- Define tools with clear descriptions and strict schemas
- Execute tool calls in parallel with asyncio.gather
- Validate inputs and handle errors in tool execution loops

### streaming
**Keywords:** streaming, SSE, Server-Sent Events, real-time, backpressure, token stream
**Solves:**
- Stream LLM tokens via SSE endpoints
- Handle tool calls within streams
- Manage backpressure with bounded queues

### local-inference
**Keywords:** Ollama, local, self-hosted, model selection, GPU, Apple Silicon
**Solves:**
- Set up Ollama for local LLM inference
- Select models based on task and hardware
- Optimize GPU usage and CI integration

### fine-tuning
**Keywords:** LoRA, QLoRA, fine-tune, DPO, synthetic data, PEFT, alignment
**Solves:**
- Configure LoRA/QLoRA for parameter-efficient training
- Generate and validate synthetic training data
- Align models with DPO and evaluate results

© OrchestKit · MIT · yonatangross/orchestkit

Similar skills

All skills for AI and ML →

llm-integration by other authors comes first.

llm-integration martinholovsky The llm-integration skill guides local LLM work with llama.cpp and Ollama, covering inference setup and prompt protection. Use it when model execution stays on your machine. CX Codex +1 239installs llm-integration rohitg00 The llm-integration skill for Claude Code provides TypeScript patterns for Anthropic API integrations, including streaming, tool use, RAG, and token cost control. CX Codex +1 ★ 2,858repo stars llm-app-development travisjneuman llm-app-development — a skill for Claude Code guides the design of RAG pipelines, autonomous agents, and structured schemas. Triggers when building chatbots, retrieval systems, or LLM-driven tools. CX Codex +1 139installs openclaude-multi-llm reason-machines openclaude-multi-llm lets Claude talk to models that expose an OpenAI Chat Completions endpoint while keeping Bash, Grep, MCP and the rest of the toolset. CX Codex +1 725installs local-llm-ops Claude MPM Team local-llm-ops — a skill for running local models through Ollama on Apple Silicon: bring up the service, pull models, start chat sessions and benchmarks. CX Codex +1 472installs llm-cli glebis Skill to process text and files through the llm CLI, supporting OpenAI, Anthropic, Google Gemini, and local Ollama models. Selects a model interactively, persists settings, and runs in chat or one-off mode. CX Codex +1 235installs llm Mindrally llm — a skill that gives your agent ground rules for large language model work: architecture, fine-tuning, training and deployment. Useful when the project is about transformers. CX Codex +1 767installs llm-fine-tuning-guide qodex-ai llm-fine-tuning-guide guides LLM fine-tuning workflows across full training, LoRA, and QLoRA. Helps adapt open weights to specific domains, style requirements, and limited hardware budgets. CX Codex +1 196installs

From the same set86

The whole set →
explore explore spawns 3–5 agents in parallel to map code structure, trace data flows, identify architecture patterns, and assess code health. Useful for quickly understanding an unfamiliar repo or discovering how a large system is designed. CX Codex +1 224installs errors The errors skill sorts Claude Code session failures by type, looks for earlier matches, and suggests recovery steps. It activates after a command, network, or tool error. CX Codex +1 223installs issue-progress-tracking issue-progress-tracking guides GitHub issue ceremonies using the gh CLI by updating progress labels, creating dedicated branches, enforcing atomic commits, and linking PRs. CX Codex +1 221installs release-management release-management creates GitHub releases with gh release, picks a semver version and builds notes from merged PRs. Handy when you tag a version or publish a changelog. CX Codex +1 220installs fix-issue The fix-issue skill investigates a GitHub issue, applies a tested fix, opens a linked PR, and records ways to prevent the bug from returning. CX Codex +1 217installs review-pr review-pr is a Claude Code skill that reviews a pull request, branch, or diff range with parallel agents. It returns an approve or request-changes verdict before a merge. CX Codex +1 217installs i18n-date-patterns i18n-date-patterns guides an agent through React translations and locale-aware formatting. Use it when adding languages, handling dates or currency, or writing pluralized messages. CX Codex +1 216installs design-system-tokens design-system-tokens builds W3C token files for colors, spacing, and themes. It applies while creating .tokens.json files or setting up dark mode. CX Codex +1 215installs

Project sponsors

Companies that keep this analytics open