flash-moe-inference — Claude Code skill: what it does, how to install
Runs the Qwen3.5-397B Mixture-of-Experts model on Apple Silicon by streaming expert weights directly from an SSD in pure C and Metal. Designed for Macs with 48GB of memory.
npx skills add reason-machines/trending-skills --skill flash-moe-inference -a claude-codeWhat flash-moe-inference does
This skill guides coding assistants through building and running the Flash-MoE execution runtime. Instead of attempting to fit the entire 209GB parameter set into memory, the engine streams four active experts per layer from NVMe storage on demand. Hand-written Metal compute kernels handle matrix multiplication directly on the GPU, achieving speeds above 4.4 tokens per second without Python runtime overhead.
The project compiles infer.m, an interactive terminal chat chat.m, and an isolated MoE benchmark main.m using a standard Makefile. Python is utilized strictly offline to extract tensors from safetensors archives into binary files. Requantizing experts down to 2-bit saves roughly 89GB of storage, though doing so compromises structured JSON generation and tool calls.
Execution strictly requires an Apple Silicon machine like M3 Max, 48GB unified RAM, macOS 26 or Darwin 25, and at least 210GB free SSD space. Systems missing these hardware prerequisites cannot run the pipeline.
What it helps with
- compile the C and Metal inference engine using make
- pack safetensors files into quantized binary slices for SSD streaming
- start a local terminal chat with function calling support
- benchmark forward-pass expert throughput on Apple Silicon hardware
Example prompts for the agent
- Build the Flash-MoE engine with make and inspect the compiled binaries
- Help unpack the Qwen3.5 safetensors directory into 4-bit expert chunks
- Run infer with the prompt 'Explain quantum computing' for 100 tokens
How to install flash-moe-inference
1Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill flash-moe-inference -a claude-code2Manually: put the files in place
~/.claude/skills/flash-moe-inference/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/flash-moe-inference/
1Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill flash-moe-inference -a codex2Manually: put the files in place
.agents/skills/flash-moe-inference/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill flash-moe-inference -a cursor2Manually: put the files in place
.cursor/skills/flash-moe-inference/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install reason-machines/trending-skills flash-moe-inference2Manually: put the files in place
.github/skills/flash-moe-inference/SKILL.md1Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill flash-moe-inference -a gemini-cli2Platform CLI
gemini skills install https://github.com/reason-machines/trending-skills --path skills/flash-moe-inferenceManually: put the files in place
.devin/skills/flash-moe-inference/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add reason-machines/trending-skills --skill flash-moe-inference -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about flash-moe-inference
How to install flash-moe-inference in Claude Code?
Run npx skills add reason-machines/trending-skills --skill flash-moe-inference -a claude-code in your terminal.
What does flash-moe-inference do?
It equips the agent with steps to compile and execute a native C and Metal engine that streams 397B MoE weights from disk on Apple Silicon Macs.
What is the penalty for using 2-bit expert weights?
Downscaling weights to 2-bit shrinks total disk footprint from 209GB to 120GB, but it breaks reliable JSON outputs and prevents tool calling.
