Skip to main content
skill CX Codex

llm-provider — Claude Code skill: add an LLM provider

The llm-provider skill guides you through adding an LLM backend to a project, from its provider class to configuration and factory registration. Use it when connecting a third-party API or another model.

MIT ★ 1,319 updated 2026-09-24

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add caliber-ai-org/ai-setup --skill llm-provider -a claude-code
Open on GitHub

What llm-provider does

The skill walks through creating a class that implements LLMProvider and wiring it into the project. The provider needs regular and streaming calls, must honor the selected model, and should report token usage. If the API supplies no usage figures, the instructions describe estimating them from text.

The steps cover edits to src/llm/types.ts, src/llm/config.ts, and src/llm/index.ts: adding the provider type, configuration and environment-variable detection, then registering it with the factory. Error handling and tests are covered too. An API key is needed when the chosen provider requires one; other configuration depends on that API. Keep the type, config, and factory changes in sync or the build or runtime can fail.

What it helps with

  • add a provider class for a new LLM API
  • handle streamed model responses
  • read provider settings from environment variables
  • register a backend in the provider factory
  • track token usage and handle API errors

Example prompts for the agent

  • Add a provider for this LLM API and register it with the factory.
  • Implement streaming and token tracking for the new backend.
  • Wire this provider's environment variables into its config and add tests.

How to install llm-provider

1Skills CLI (Vercel)

npx skills add caliber-ai-org/ai-setup --skill llm-provider -a claude-code

2Manually: put the files in place

~/.claude/skills/llm-provider/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/llm-provider/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about llm-provider

How do I install llm-provider in Claude Code?

Run npx skills add caliber-ai-org/ai-setup --skill llm-provider -a claude-code. The skill file is at ~/.claude/skills/llm-provider/SKILL.md.

What does the llm-provider skill do?

It guides you through adding an LLM backend, including regular and streaming calls, configuration, and factory registration.

Which project files need changes when adding a provider?

Update the ProviderType union, settings in src/llm/config.ts, and the createProvider() switch in src/llm/index.ts together. Missing one can cause a build error or an unknown-provider runtime error.

Source text

SKILL.md · MIT
---
name: llm-provider
description: Adds a new LLM provider implementing LLMProvider interface with call() and stream() methods. Integrates with provider factory in src/llm/index.ts, config detection in src/llm/config.ts, and error handling via tracking and recovery. Use when adding a new model backend, integrating a third-party LLM API, or extending LLM platform support. Do NOT use for fixing bugs in existing providers, modifying existing provider behavior, or changing the LLMProvider interface.
paths:
  - src/llm/**/*.ts
  - src/llm/__tests__/**/*.ts
---
# LLM Provider

## Critical

1. **All providers MUST implement the LLMProvider interface** from src/llm/types.ts with three methods:
   - call(options: LLMCallOptions): Promise<string> — single non-streaming call returning text
   - stream(options: LLMStreamOptions, callbacks: LLMStreamCallbacks): Promise<void> — streaming call invoking callbacks
   - listModels?(): Promise<string[]> — optional; list available models from the API

2. **Initialize client in constructor and store defaultModel from config**. Example: `this.client = new YourSDK({ apiKey: config.apiKey })`. Never lazy-initialize on first call — providers are instantiated once and cached in src/llm/index.ts.

3. **For EVERY response in call() and stream(), invoke trackUsage(model, usage)** from src/llm/usage.js before returning/ending. This is mandatory — it captures token metrics for CLI telemetry and cost analysis. If the API doesn't return usage data, estimate via estimateTokens(text), which assumes ~4 chars per token.

4. **Both call() and stream() must respect the model parameter** using pattern: `options.model || this.defaultModel`. Never hardcode model names. Callers supply model overrides via LLMCallOptions.model.

5. **Error handling: catch all errors, preserve error messages unchanged**. The retry logic in src/llm/index.ts handles transient errors (ECONNRESET, socket hang up, 529 overload). For seat-based providers (Cursor, Claude CLI), wrap stderr via parseSeatBasedError() for user-friendly messages.

6. **Always update ProviderType union** (Step 2), DEFAULT_MODELS (Step 4), and createProvider() switch case (Step 5) in lock-step. Missing any one breaks the build or causes runtime Unknown provider error.

## Instructions

### Step 1: Create provider class file
Verify directory exists: `ls -la src/llm/`. Create `src/llm/your-provider.ts`. Match existing provider patterns (src/llm/anthropic.ts, src/llm/openai-compat.ts).

Minimal structure:
```typescript
import type { LLMProvider, LLMCallOptions, LLMStreamOptions, LLMStreamCallbacks, LLMConfig, TokenUsage } from './types.js';
import { trackUsage } from './usage.js';
import { estimateTokens } from './utils.js';

export class YourProviderProvider implements LLMProvider {
  private client: YourSDKType;
  private defaultModel: string;

  constructor(config: LLMConfig) {
    if (!config.apiKey) throw new Error('API key required');
    this.client = new YourSDK({ apiKey: config.apiKey, ...(config.baseUrl && { baseURL: config.baseUrl }) });
    this.defaultModel = config.model;
  }

  async call(options: LLMCallOptions): Promise<string> {
    const model = options.model || this.defaultModel;
    const response = await this.client.messages.create({ model, max_tokens: options.maxTokens || 4096, system: options.system, messages: [{ role: 'user', content: options.prompt }] });
    trackUsage(model, { inputTokens: response.usage?.input_tokens || 0, outputTokens: response.usage?.output_tokens || 0 });
    return response.content?.[0]?.text || '';
  }

  async stream(options: LLMStreamOptions, callbacks: LLMStreamCallbacks): Promise<void> {
    const model = options.model || this.defaultModel;
    const messages = [...(options.messages || []), { role: 'user' as const, content: options.prompt }];
    try {
      const stream = await this.client.stream({ model, max_tokens: options.maxTokens || 10240, system: options.system, messages });
      let stopReason: string | undefined, usage: TokenUsage | undefined;
      for await (const chunk of stream) {
        if (chunk.delta?.text) callbacks.onText(chunk.delta.text);
        if (chunk.delta?.stop_reason) stopReason = chunk.delta.stop_reason;
        if (chunk.usage) usage = { inputTokens: chunk.usage.input_tokens, outputTokens: chunk.usage.output_tokens };
      }
      if (usage) trackUsage(model, usage);
      callbacks.onEnd({ stopReason, usage });
    } catch (error) { callbacks.onError(error instanceof Error ? error : new Error(String(error))); }
  }
}
```

Verify: File exports the class; imports match existing providers.

### Step 2: Add to ProviderType union
Edit `src/llm/types.ts` line 1. Add your provider in kebab-case:
```typescript
export type ProviderType = 'anthropic' | 'vertex' | 'openai' | 'cursor' | 'claude-cli' | 'your-provider';
```
Verify: `npx tsc --noEmit` shows no ProviderType errors.

### Step 3: Add config fields
If your provider needs fields beyond apiKey, model, baseUrl, extend LLMConfig in src/llm/types.ts:
```typescript
export interface LLMConfig {
  provider: ProviderType;
  model: string;
  fastModel?: string;
  apiKey?: string;
  baseUrl?: string;
  yourProviderSecret?: string;
}
```

### Step 4: Update config.ts
Edit `src/llm/config.ts`:

**Line 9:** Add to DEFAULT_MODELS:
```typescript
export const DEFAULT_MODELS: Record<ProviderType, string> = {
  anthropic: 'claude-sonnet-4-6',
  vertex: 'claude-sonnet-4-6',
  openai: 'gpt-5.4-mini',
  cursor: 'sonnet-4.6',
  'claude-cli': 'default',
  'your-provider': 'your-provider/default-model',
};
```

**Line 17:** Add to MODEL_CONTEXT_WINDOWS if known:
```typescript
export const MODEL_CONTEXT_WINDOWS: Record<string, number> = {
  'your-provider/model-name': 128_000,
};
```

**Line 59:** In resolveFromEnv(), add env detection before final return null:
```typescript
if (process.env.YOUR_PROVIDER_API_KEY) {
  return {
    provider: 'your-provider',
    apiKey: process.env.YOUR_PROVIDER_API_KEY,
    model: process.env.CALIBER_MODEL || DEFAULT_MODELS['your-provider'],
    baseUrl: process.env.YOUR_PROVIDER_BASE_URL,
  };
}
```

**Line 115:** In readConfigFile() validation, add 'your-provider' to includes list.

Verify: `npm run test -- src/llm/__tests__/ -t config` confirms env var detection works.

### Step 5: Register in factory
Edit `src/llm/index.ts`. Add import (line ~4):
```typescript
import { YourProviderProvider } from './your-provider.js';
```

In createProvider() switch (line ~24), add before default case:
```typescript
case 'your-provider':
  return new YourProviderProvider(config);
```

Verify: `npx tsc --noEmit` passes; no type errors on switch cases.

### Step 6: Write tests
Create `src/llm/__tests__/your-provider.test.ts`:
```typescript
import { describe, it, expect, beforeEach } from 'vitest';
import { YourProviderProvider } from '../your-provider.js';

describe('YourProviderProvider', () => {
  let provider: YourProviderProvider;
  beforeEach(() => {
    provider = new YourProviderProvider({ provider: 'your-provider', model: 'test', apiKey: 'test' });
  });

  it('implements LLMProvider interface', () => {
    expect(typeof provider.call).toBe('function');
    expect(typeof provider.stream).toBe('function');
  });

  it('call() returns string', async () => {
    const result = await provider.call({ system: 'helpful', prompt: 'hi' });
    expect(typeof result).toBe('string');
  });

  it('stream() invokes callbacks', async () => {
    const texts: string[] = [];
    let ended = false;
    await provider.stream({ system: 'helpful', prompt: 'hi' }, {
      onText: (t) => texts.push(t),
      onEnd: () => { ended = true; },
      onError: () => {},
    });
    expect(ended).toBe(true);
  });
});
```

Verify: `npm run test -- src/llm/__tests__/your-provider.test.ts` passes.

### Step 7: Integration test
Run factory tests with your provider env var:
```bash
YOUR_PROVIDER_API_KEY=test npm run test -- src/llm/__tests__/index.test.ts
```
Verify: getProvider() instantiates your provider; llmCall() dispatches correctly.

## Examples

### Example 1: Local LM Studio server
User says: "I need caliber to use my local LM Studio instance."

Actions: Create src/llm/lm-studio.ts extending OpenAICompatProvider. Add 'lm-studio' to ProviderType. In config.ts:
```typescript
if (process.env.LM_STUDIO_BASE_URL) {
  return { provider: 'lm-studio', apiKey: '', model: 'local', baseUrl: process.env.LM_STUDIO_BASE_URL };
}
```
Register in createProvider() case. User: `export LM_STUDIO_BASE_URL=http://localhost:8000/v1`. Result: caliber uses local LM Studio; tokens estimated via estimateTokens().

### Example 2: Ollama (seat-based)
User says: "Ollama is auto-detected; no API key needed."

Actions: Create src/llm/ollama.ts extending OpenAICompatProvider. Add 'ollama' to ProviderType and SEAT_BASED_PROVIDERS. In config.ts:
```typescript
if (process.env.OLLAMA_HOST) {
  return { provider: 'ollama', model: 'mistral', baseUrl: process.env.OLLAMA_HOST || 'http://localhost:11434/v1' };
}
```
Result: Offline per-machine LLM without API keys.

## Common Issues

**Unknown provider: your-provider**
- Cause: ProviderType updated but createProvider() case missing.
- Fix: Add case in src/llm/index.ts and import the class.

**Cannot find module './your-provider.js'**
- Cause: File named your_provider.ts (underscore) not your-provider.ts (kebab).
- Fix: Rename file to use kebab-case.

**API key is required for YourProvider**
- Cause: Env var YOUR_PROVIDER_API_KEY not set; resolveFromEnv() didn't detect it.
- Fix: Verify env var name in config.ts matches. Test: `YOUR_PROVIDER_API_KEY=test npm run test -- src/llm/__tests__/index.test.ts`.

**LLM response did not include usage tokens**
- Cause: Provider API doesn't return usage (local models).
- Fix: Estimate tokens: `trackUsage(model, { inputTokens: estimateTokens(options.system + options.prompt), outputTokens: estimateTokens(response.text) });`

**Stream callbacks never fire; onEnd not called**
- Cause: Async iterator not fully consumed before method returns.
- Fix: Ensure iteration completes before onEnd(): `for await (const chunk of stream) { } callbacks.onEnd({ stopReason, usage });`

**My model parameter is ignored**
- Cause: call()/stream() doesn't use `options.model || this.defaultModel`.
- Fix: Replace hardcoded model: `const model = options.model || this.defaultModel; const response = await this.client.create({ model, ... });`

**trackUsage() is never called**
- Cause: Forgot to call trackUsage() in call() or stream().
- Fix: Add after every response: `trackUsage(model, { inputTokens: ..., outputTokens: ... });`

**Type error: Provider doesn't implement LLMProvider**
- Cause: Missing method or wrong signature.
- Fix: Verify all required methods exist with exact signatures from src/llm/types.ts. Copy-paste from anthropic.ts as reference.

© caliber-ai-org · MIT · caliber-ai-org/ai-setup

Similar skills

All skills for AI and ML →

From the same set17

The whole set →
find-skills find-skills discovers community skills from the Caliber registry and installs them directly into your coding session. CX Codex +1 ★ 1,319repo stars save-learning save-learning turns a user's lasting preferences into project learnings. Use it when an agent should carry a rule from one session into the next. CX Codex +1 ★ 1,319repo stars setup-caliber setup-caliber walks a project through Caliber setup: installs the CLI, hooks agent config sync into every commit, and creates the config files your agents are missing. CX Codex +1 ★ 1,319repo stars adding-a-command adding-a-command generates Commander.js CLI commands in src/commands/, connects them to src/cli.ts, and sets up telemetry tracking. CX Codex +1 ★ 1,319repo stars scoring-checks Adds deterministic scoring checks that measure AI agent config quality. Evaluates file structure, standards compliance, and configuration patterns — with no randomness or external calls. CX Codex +1 ★ 1,319repo stars caliber-testing caliber-testing is a Claude Code skill that generates Vitest tests with process.env resets, vi.hoisted mocks, and test files placed in __tests__ directories. CX Codex +1 ★ 1,319repo stars writers-pattern The writers-pattern skill guides you through adding a module that writes agent config files for a new platform. Use it when extending caliber to another coding tool. CX Codex +1 ★ 1,319repo stars caliber-jev-compaction caliber-jev-compaction compacts Claude Code sessions using TypeSafe's Jev model to score and drop stale tool calls instead of summarizing. CX Codex ★ 1,319repo stars

Project sponsors

Companies that keep this analytics open