Skip to main content
skill CX Codex

entropy — Claude Code skill: information theory entropy calculations

A skill for Shannon entropy, conditional and differential entropy, KL divergence, and maximum entropy principle problems.

MIT ★ 3,951 updated 2026-01-26

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add parcadei/Continuous-Claude-v3 --skill entropy -a claude-code
Open on GitHub

What entropy does

The skill provides a decision tree for entropy calculations in discrete and continuous distributions. It includes ready-made scipy.stats.entropy and sympy_compute.py commands for integrals and simplifications. Allowed tools are Bash and Read.

It gives formulas for Shannon entropy H(X) = -sum p(x) log₂ p(x), joint H(X,Y), and conditional H(Y|X), along with chain rule properties and inequalities. For continuous cases, differential entropy h(X) can be negative.

The maximum entropy principle selects the least biased distribution under given constraints: uniform with no constraints, exponential for E[X] = μ, Gaussian for E[X] and Var[X].

What it helps with

  • compute Shannon entropy for a discrete distribution
  • find KL divergence between two distributions
  • calculate differential entropy of a continuous variable
  • verify conditional entropy properties H(Y|X)
  • determine the maximum entropy distribution under constraints

Example prompts for the agent

  • Calculate entropy for probabilities [0.25, 0.25, 0.25, 0.25]
  • Find KL divergence between p = [0.5, 0.5] and q = [0.9, 0.1]
  • Simplify the entropy expression -p*log(p, 2) - (1-p)*log(1-p, 2) in SymPy

How to install entropy

1Skills CLI (Vercel)

npx skills add parcadei/Continuous-Claude-v3 --skill entropy -a claude-code

2Manually: put the files in place

~/.claude/skills/entropy/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/entropy/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about entropy

How to install entropy in Claude Code?

Run npx skills add parcadei/Continuous-Claude-v3 --skill entropy -a claude-code. The skill will be placed in ~/.claude/skills/entropy/SKILL.md.

What does entropy do?

A skill for information theory tasks: computes Shannon, joint, conditional, and differential entropy, KL divergence, applies the maximum entropy principle. Uses scipy.stats.entropy and sympy_compute.py.

Which tools are allowed in entropy?

Bash and Read. The skill runs Python scripts via uv run and reads documentation files.

Source text

SKILL.md · MIT
---
name: entropy
description: "Problem-solving strategies for entropy in information theory"
allowed-tools: [Bash, Read]
---

# Entropy

## When to Use

Use this skill when working on entropy problems in information theory.

## Decision Tree


1. **Shannon Entropy**
   - H(X) = -sum p(x) log2 p(x)
   - Maximum for uniform distribution: H_max = log2(n)
   - Minimum = 0 for deterministic (one outcome certain)
   - `scipy.stats.entropy(p, base=2)` for discrete

2. **Entropy Properties**
   - Non-negative: H(X) >= 0
   - Concave in p
   - Chain rule: H(X,Y) = H(X) + H(Y|X)
   - `z3_solve.py prove "entropy_nonnegative"`

3. **Joint and Conditional Entropy**
   - H(X,Y) = -sum sum p(x,y) log2 p(x,y)
   - H(Y|X) = H(X,Y) - H(X)
   - H(Y|X) <= H(Y) with equality iff independent

4. **Differential Entropy (Continuous)**
   - h(X) = -integral f(x) log f(x) dx
   - Can be negative!
   - Gaussian: h(X) = 0.5 * log2(2*pi*e*sigma^2)
   - `sympy_compute.py integrate "-f(x)*log(f(x))" --var x`

5. **Maximum Entropy Principle**
   - Given constraints, max entropy distribution is least biased
   - Uniform for no constraints
   - Exponential for E[X] = mu constraint
   - Gaussian for E[X], Var[X] constraints


## Tool Commands

### Scipy_Entropy
```bash
uv run python -c "from scipy.stats import entropy; p = [0.25, 0.25, 0.25, 0.25]; H = entropy(p, base=2); print('Entropy:', H, 'bits')"
```

### Scipy_Kl_Div
```bash
uv run python -c "from scipy.stats import entropy; p = [0.5, 0.5]; q = [0.9, 0.1]; kl = entropy(p, q); print('KL divergence:', kl)"
```

### Sympy_Entropy
```bash
uv run python -m runtime.harness scripts/sympy_compute.py simplify "-p*log(p, 2) - (1-p)*log(1-p, 2)"
```

## Key Techniques

*From indexed textbooks:*

- [Elements of Information Theory] Elements of Information Theory -- Thomas M_ Cover &amp; Joy A_ Thomas -- 2_, Auflage, New York, NY, 2012 -- Wiley-Interscience -- 9780470303153 -- 2fcfe3e8a16b3aeefeaf9429fcf9a513 -- Anna’s Archive. What is the channel capacity of this channel? This is the multiple\-access channel solved by Liao and Ahlswede.

## Cognitive Tools Reference

See `.claude/skills/math-mode/SKILL.md` for full tool documentation.

© parcadei · MIT · parcadei/Continuous-Claude-v3

Similar skills

All skills for AI and ML →

From the same set161

The whole set →
modular-arithmetic Handles modular arithmetic problems in graph number theory. Activates when you need the extended Euclidean algorithm, the Chinese remainder theorem, Euler's theorem, quadratic residues, or order of elements. CX Codex +1 ★ 3,951repo stars agentica-spawn agentica-spawn — a skill that spawns multi-agent patterns via Agentica: swarms for parallel research, hierarchies for role-based tasks, generator-critic loops for iterative refinement, and juries for consensus. CX Codex +1 ★ 3,951repo stars parallel-agent-contracts parallel-agent-contracts enforces type ownership rules across parallel agents in TypeScript projects, requiring grep checks and tsc verification before any subtask completes. CX Codex +1 ★ 3,951repo stars spark spark is a Claude Code subagent for small, focused code changes. It handles quick fixes and tweaks, then writes a summary to a project file. CX Codex ★ 3,951repo stars proactive-delegation The proactive-delegation rule directs Claude Code to proactively hand off tasks to specialized subagents. It preserves the main context by detecting research, bug fixes, and parallel work automatically. CX Codex +1 ★ 3,951repo stars search-hierarchy The search-hierarchy skill picks a search tool based on whether a question concerns code structure, meaning, or an exact identifier. It helps locate a relevant file before reading it. CX Codex +1 ★ 3,951repo stars channel-capacity A skill for solving information-theory problems: calculating channel capacity, mutual information, and applying the Blahut-Arimoto algorithm. Use it when working with information theory, coding, or communication channels. CX Codex +1 ★ 3,951repo stars rudin-real-complex-analysis rudin-real-complex-analysis is a Claude Code skill for solving problems and checking proofs with Walter Rudin's Real and Complex Analysis textbook, a theorem tree, and Python scripts. CX Codex +1 ★ 3,951repo stars

Project sponsors

Companies that keep this analytics open