Skip to main content
skill CX Codex

data-formats — Claude Code skill for inspecting files

The data-formats skill guides file inspection before parsing, from checking binary signatures to sampling large structured files. Use it when a file’s format is unclear.

MIT ★ 1,125 updated 2026-10-27

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add vstorm-co/pydantic-deepagents --skill data-formats -a claude-code
Open on GitHub

What data-formats does

The skill starts with file inspection: identify a likely type, examine a short text preview, or read the opening bytes. Its examples use file, xxd, head, and a small Python snippet.

It covers binary data, structured text, SQLite files, and model checkpoints for PyTorch, TensorFlow, NumPy, HuggingFace, and ONNX. For large files, it recommends sampling records and checking field counts, missing values, mixed types, and encoding. Reading binary content as text can corrupt it, so choosing the right reading method matters.

For an unfamiliar binary format, the steps include looking for signatures, version markers, string tables, and offsets, then testing a small parser against known values. The examples use Python and shell commands; format-specific tools such as torch, numpy, or onnx appear where relevant.

What it helps with

  • identify a file format from its signature and contents
  • inspect CSV delimiters, quoting, and header rows
  • probe an unfamiliar binary file before parsing it
  • check a multi-file dataset for naming patterns and inconsistencies

Example prompts for the agent

  • Inspect data.bin and suggest a way to identify and parse its format.
  • Check records.csv for its delimiter, header, and encoding issues.
  • Review these model files and explain how to load them.

How to install data-formats

1Skills CLI (Vercel)

npx skills add vstorm-co/pydantic-deepagents --skill data-formats -a claude-code

2Manually: put the files in place

~/.claude/skills/data-formats/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/data-formats/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about data-formats

How do I install data-formats in Claude Code?

Run npx skills add vstorm-co/pydantic-deepagents --skill data-formats -a claude-code. The skill file is at ~/.claude/skills/data-formats/SKILL.md.

What does the data-formats skill do?

It guides the agent through inspecting files and choosing a parsing approach. Its examples cover binary and structured text, SQLite databases, and model files.

How does the skill approach an unknown binary format?

It suggests examining the opening bytes for signatures, version markers, and string tables, then checking file size and testing a minimal parser against known values.

Source text

SKILL.md · MIT
---
name: data-formats
description: "Working with diverse data formats: binary, text, structured, and custom"
tags: [data, parsing, formats, benchmark]
version: "1.0.0"
---

# Data Formats

How to work with diverse and unknown data formats.

## Format Detection

Always inspect before parsing:

```bash
file <filename>                    # MIME type detection
xxd <filename> | head -5           # hex dump (first bytes)
head -3 <filename>                 # text preview
python3 -c "
with open('<filename>', 'rb') as f:
    h = f.read(16)
    print(h, h.hex())
"
```

## Common Formats

### Binary
- **Magic bytes**: Most binary formats start with a signature (ELF: `\x7fELF`, PNG: `\x89PNG`)
- **Endianness**: Check if little-endian or big-endian (`struct.unpack('<I', ...)` vs `'>I'`)
- **Alignment**: Fields are often aligned to 4 or 8 bytes
- **Offsets**: Binary headers often contain offsets to other sections

### Structured text
- **CSV/TSV**: Check delimiter (comma, tab, pipe), quoting, header row
- **JSON**: `python3 -c "import json; json.load(open('f'))"`
- **YAML**: Check indentation, anchors/aliases
- **TOML**: `python3 -c "import tomllib; ..."`
- **XML**: Check encoding declaration, namespaces

### Checkpoints / Model files
- **PyTorch**: `.pt`, `.pth` → `torch.load(f, map_location='cpu')`
- **TensorFlow**: `.ckpt` → index + data files, use `tf.train.load_checkpoint()`
- **NumPy**: `.npy`, `.npz` → `numpy.load()`
- **HuggingFace**: `config.json` + `model.safetensors`
- **ONNX**: `onnx.load()`

### Database files
- **SQLite**: `file` says "SQLite 3.x database" → `sqlite3 <file> ".tables"`
- **WAL files**: SQLite write-ahead log — recover with `sqlite3` PRAGMA
- **CSV dumps**: Often need schema inference

## Parsing Strategies

### Unknown binary format
1. Hex dump first 256 bytes: `xxd file | head -16`
2. Look for magic bytes, version numbers, string tables
3. Check file size — does it suggest a pattern? (e.g., N * record_size)
4. Look for documentation of the format online
5. Write a minimal parser, test on known values

### Large structured files
1. Never load entirely — sample first: `head`, `tail`, `shuf -n 10`
2. Check consistency: are all lines the same format?
3. Count fields: `head -1 file | awk -F',' '{print NF}'`
4. Watch for: mixed types, missing values, encoding issues

### Multi-file datasets
1. List all files and sizes
2. Look for manifest/index files (often JSON or CSV)
3. Check naming patterns — timestamps, sequence numbers, shards
4. Process one file first, then generalize

## Common Pitfalls

- Assuming UTF-8 when the file is Latin-1 or binary
- Assuming CSV when it's TSV (or vice versa)
- Ignoring the header row
- Not handling quoted fields with embedded delimiters
- Reading binary files as text (corrupts data)
- Endianness mismatch (x86 is little-endian, network byte order is big-endian)

© vstorm-co · MIT · vstorm-co/pydantic-deepagents

Similar skills

All skills for Data and databases →

From the same set14

The whole set →
environment-discovery The environment-discovery skill trains Claude Code to systematically explore unfamiliar environments before making changes. It checks file structure, data formats, available tools, and existing code—saving debugging time. CX Codex +1 ★ 1,125repo stars report-writing report-writing provides guidelines for structuring analytical reports, drafting stand-alone executive summaries, and citing sources. CX Codex +1 ★ 1,125repo stars build-and-compile The build-and-compile skill helps Claude Code choose build commands from a project's files and troubleshoot compiler or dependency errors. Use it when a build fails or the project setup is unclear. CX Codex +1 ★ 1,125repo stars research-methodology research-methodology — a skill with methodology for structured research: three-phase search, source reliability ranking, and note-taking with confidence levels for each finding. CX Codex +1 ★ 1,125repo stars performant-code performant-code guides Claude Code to write fast routines for large files and strict memory caps. Helps prevent runtime timeouts on heavy workloads. CX Codex +1 ★ 1,125repo stars systematic-debugging systematic-debugging walks the agent through five debugging steps, from reproducing the failure to confirming the fix. Useful once random edits stop working. CX Codex +1 ★ 1,125repo stars verification-strategy verification-strategy — a structured checklist skill for Claude Code to audit file outputs, compiler flags, exit codes, and edge cases before declaring a task finished. CX Codex +1 ★ 1,125repo stars diagram-design The diagram-design skill guides research diagram creation on an Excalidraw canvas through MCP. Use it to map comparisons, processes, architectures, or timelines. CX Codex +1 ★ 1,125repo stars

Project sponsors

Companies that keep this analytics open