Skip to main content
affpapa.org
RU EN
skill CX Codex

indexion-segment — Claude Code skill: chunk text for RAG

Splits documents into contextual chunks for RAG and embedding pipelines. Useful when you want topic boundaries instead of a fixed character count.

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓ 
npx skills add trkbt10/indexion-skills --skill indexion-segment -a claude-code
Open on GitHub

What indexion-segment does

The skill drives the indexion segment command: it takes an input file, cuts it into segments and writes them to an output directory. Three strategies are available — window with sliding-window divergence detection, tfidf for topic shifts, and punctuation for sentence boundaries. The default is window with adaptive thresholding.

Chunk sizes are set explicitly through --min-size, --max-size and --target-size. A hybrid mode (--hybrid) blends NCD and TF-IDF, weighted by --ncd-weight and --tfidf-weight. That is a reasonable starting point for mixed-content documents, though the weights need tuning against your own corpus.

The command takes an input file and an output folder, and --prefix controls the output file names. If segments come out too large or too choppy, adjust --threshold and --target-size.

What it helps with

  • chunk a long document for embeddings
  • split text on topic changes rather than character count
  • prepare a corpus for similarity search
  • extract meaningful sections from mixed-content documents
  • enforce a fixed chunk size in a pipeline

Example prompts for the agent

  • Split document.txt into segments in the output/ folder
  • Segment this text with the tfidf strategy, target size 800
  • Run hybrid mode with NCD weight 0.6 on this file

How to install indexion-segment

1Skills CLI (Vercel)

npx skills add trkbt10/indexion-skills --skill indexion-segment -a claude-code

2Manually: put the files in place

~/.claude/skills/indexion-segment/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/indexion-segment/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about indexion-segment

How do I install indexion-segment in Claude?

Run npx skills add trkbt10/indexion-skills --skill indexion-segment -a claude-code. The skill file then lands at ~/.claude/skills/indexion-segment/SKILL.md.

What does indexion-segment do?

It splits text into contextual segments for RAG and embedding pipelines. You pick the strategy per task: window divergence, TF-IDF or punctuation.

How do the segmentation strategies differ?

window looks for divergence in a sliding window and is the default. tfidf detects topic changes, punctuation cuts at sentence boundaries. Mixed documents can use --hybrid.

Source text

SKILL.md · Apache-2.0 
---
name: indexion-segment
description: Split text into contextual chunks for RAG/embedding pipelines. Document segmentation and section extraction using window, tfidf, punctuation, or hybrid strategies chosen by intent.
---

# indexion segment

Split text into contextual segments using divergence-based, TF-IDF, or punctuation strategies.

## When to Use

- User needs to chunk text for RAG or embedding pipelines
- User wants to split a document into meaningful sections
- User asks to segment text for processing
- Preparing text for similarity analysis at sub-document level

## Usage

```bash
# Default window divergence strategy
indexion segment <input-file> <output-dir>

# TF-IDF based segmentation
indexion segment --strategy=tfidf <input-file> <output-dir>

# Punctuation-based segmentation
indexion segment --strategy=punctuation <input-file> <output-dir>

# Custom segment sizes
indexion segment --min-size=200 --max-size=3000 --target-size=800 document.txt output/

# Custom divergence threshold
indexion segment --threshold=0.5 document.txt output/

# Adaptive threshold mode (default)
indexion segment --adaptive document.txt output/

# Hybrid NCD+TF-IDF mode
indexion segment --hybrid --ncd-weight=0.6 --tfidf-weight=0.4 document.txt output/

# Custom window size
indexion segment --window-size=5 document.txt output/

# Custom output prefix
indexion segment --prefix=chunk document.txt output/
```

## Options

| Option | Default | Description |
|--------|---------|-------------|
| `--strategy=NAME` | window | Strategy: window, tfidf, punctuation |
| `--min-size=INT` | 100 | Minimum segment characters |
| `--max-size=INT` | 2000 | Maximum segment characters |
| `--target-size=INT` | 500 | Target segment characters |
| `--threshold=FLOAT` | 0.42 | Divergence threshold |
| `--window-size=INT` | 3 | Window size |
| `--adaptive` | true | Adaptive threshold mode |
| `--hybrid` | false | NCD+TF-IDF hybrid mode |
| `--ncd-weight=FLOAT` | 0.5 | NCD weight in hybrid mode |
| `--tfidf-weight=FLOAT` | 0.5 | TF-IDF weight in hybrid mode |
| `--prefix=NAME` | segment | Output file prefix |

## Strategies

| Strategy | Description |
|----------|-------------|
| `window` (default) | Sliding window divergence detection |
| `tfidf` | TF-IDF based topic change detection |
| `punctuation` | Punctuation/sentence boundary based |

## Workflow

1. Run `indexion segment <input-file> <output-dir>` to split text with defaults
2. Adjust `--threshold` and `--target-size` to tune segmentation granularity
3. Use `--hybrid` mode for better accuracy on mixed-content documents

© trkbt10 · Apache-2.0 · trkbt10/indexion-skills

Similar skills

Everything for AI and ML →
langchain-rag langchain-ai Equips coding agents with reference RAG implementations in Python and TypeScript using LangChain loaders, splitters, and vector stores. CX Codex +1 17,500installs rag giuseppe-trisciuoglio Guides the creation of Retrieval-Augmented Generation pipelines, covering document chunking, embedding generation, vector database selection, and retrieval. CX Codex +1 3,700installs rag-retrieval OrchestKit A skill with ready-made retrieval-augmented generation patterns: hybrid search, HyDE, reranking, pgvector. Useful when you are wiring a RAG pipeline and want working recipes. CX Codex +1 1,500installs rag-engineer sickn33 A skill for designing retrieval-augmented generation systems with vector databases. Covers semantic chunking, embedding model selection, and search quality optimization before results reach the LLM. CX Codex +1 1,000installs embedding-strategies Seth Hobson This skill helps choose embedding models, tune chunking, and assess quality for semantic search and RAG projects. CX Codex 11,391installs similarity-search-patterns Seth Hobson This skill helps build similarity search with vector databases for semantic search, RAG retrieval, nearest-neighbor queries, and hybrid search. CX Codex 10,700installs

From the same set6

indexion-readme This skill assembles a project README from templates, doc comments and static files, then checks that your edits only added text. CX Codex +1 5,100installs indexion-documentation This skill measures documentation coverage, finds drift between code and docs, and draws dependency graphs. CX Codex +1 5,000installs indexion-sdd Turns an RFC or spec into SDD requirements and checks them against the code quantitatively. Useful when implementation has to stay inside the document. CX Codex +1 5,000installs indexion-identity This skill checks whether function, file and folder names still match their contents, then plans verified renames. Useful during refactoring. CX Codex +1 4,900installs indexion-kgf Shows tokens, parse events and extracted dependency edges for source files, so you can see how indexion processes them. CX Codex +1 4,900installs

Project sponsors

Companies that keep this analytics open
Gold sponsor
Nutra.Club

Impressions, clicks and code copies — open statistics. We keep the numbers public for the same reason NeBlask itself is public. Become a sponsor