web-to-markdown · softaworks — Claude Code skill: what it does, how to install
web-to-markdown converts JavaScript-heavy web pages into clean Markdown via a local browser CLI. Helpful for archiving articles and docs without layout junk.
MIT 796 installs ★ 2,707 updated 2026-03-05
npx skills add softaworks/agent-toolkit --skill web-to-markdown -a claude-codeWhat web-to-markdown does
The skill turns dynamic pages into Markdown using the local web2md utility. It drives an installed Chromium browser through puppeteer-core, strips extraneous layout elements using Readability, and transforms the body with Turndown while cleaning up links.
A strict gate protects activation. The agent halts and refuses to run unless your prompt includes use the skill web-to-markdown or use a skill web-to-markdown.
Execution fails if web2md is absent from your PATH when checked by command -v web2md. For gated content or verification screens, passing --interactive launches a visible browser window with --user-data-dir so you can solve checks manually before parsing starts.
What it helps with
- extract readable article text from JavaScript-rendered sites
- pass manual login checks or captchas before saving content
- strip banners and navbars from web pages into clean Markdown
- batch save several pages into auto-named Markdown files
Example prompts for the agent
- use the skill web-to-markdown: convert the page to ./page.md
- use the skill web-to-markdown: print page contents to stdout
- use the skill web-to-markdown: run interactive mode with profile ./tmp/web2md-profile and save to ./out/
How to install web-to-markdown
1Skills CLI (Vercel)
npx skills add softaworks/agent-toolkit --skill web-to-markdown -a claude-code2Manually: put the files in place
~/.claude/skills/web-to-markdown/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/web-to-markdown/
1Skills CLI (Vercel)
npx skills add softaworks/agent-toolkit --skill web-to-markdown -a codex2Manually: put the files in place
.agents/skills/web-to-markdown/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add softaworks/agent-toolkit --skill web-to-markdown -a cursor2Manually: put the files in place
.cursor/skills/web-to-markdown/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install softaworks/agent-toolkit skills/web-to-markdown2Manually: put the files in place
.github/skills/web-to-markdown/SKILL.md1Skills CLI (Vercel)
npx skills add softaworks/agent-toolkit --skill web-to-markdown -a gemini-cli2Platform CLI
gemini skills install https://github.com/softaworks/agent-toolkit --path skills/web-to-markdownManually: put the files in place
.devin/skills/web-to-markdown/SKILL.mdThe legacy .windsurf/skills/ path still works
Skills CLI (Vercel)
npx skills add softaworks/agent-toolkit --skill web-to-markdown -a opencodeClaude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about web-to-markdown
How do I install web-to-markdown in Claude Code?
Run npx skills add softaworks/agent-toolkit --skill web-to-markdown -a claude-code in your terminal.
What does the web-to-markdown skill do?
It controls a local Chromium browser using web2md, extracts text via Readability, and outputs Markdown with Turndown.
Why does the skill ignore generic conversion prompts?
It enforces a mandatory activation trigger. The assistant will refuse the task unless the request contains the exact phrase use the skill web-to-markdown.
Source text
SKILL.md · MIT
---
name: web-to-markdown
description: "Use ONLY when the user explicitly says: 'use the skill web-to-markdown ...' (or 'use a skill web-to-markdown ...'). Converts webpage URLs to clean Markdown by calling the local web2md CLI (Puppeteer + Readability), suitable for JS-rendered pages."
metadata:
version: 0.1.0
---
# web-to-markdown
Convert web pages to clean Markdown by driving a locally installed browser (via `web2md`).
## Hard trigger gate (must enforce)
This skill MUST NOT be used unless the user explicitly wrote **exactly** a phrase like:
- `use the skill web-to-markdown ...`
- `use a skill web-to-markdown ...`
If the user did not explicitly request this skill by name, stop and ask them to re-issue the request including: `use the skill web-to-markdown`.
## What this skill does
- Handles JS-rendered pages (Puppeteer → user Chrome).
- Works best with Chromium-family browsers (Chrome/Chromium/Brave/Edge) via `puppeteer-core`.
- Extracts main content (Readability).
- Converts to Markdown (Turndown) with cleaned links and optional YAML frontmatter.
## Non-goals
- Do not use Playwright or other browser automation stacks; the mechanism is `web2md`.
## Inputs you should collect (ask only if missing)
- `url` (or a list of URLs)
- Output preference:
- Print to stdout (`--print`), OR
- Save to a file (`--out ./file.md`), OR
- Save to a directory (`--out ./some-dir/` to auto-name by page title)
- Optional rendering controls for tricky pages:
- `--chrome-path <path>` (if Chrome auto-detection fails)
- `--interactive` (show Chrome and pause so the user can complete human checks/login, then press Enter)
- `--wait-until load|domcontentloaded|networkidle0|networkidle2`
- `--wait-for '<css selector>'`
- `--wait-ms <milliseconds>`
- `--headful` (debug)
- `--no-sandbox` (sometimes required in containers/CI)
- `--user-data-dir <dir>` (login/session; use a dedicated profile directory)
## Workflow
1) Confirm the user explicitly invoked the skill (`use the skill web-to-markdown`).
2) Validate URL(s) start with `http://` or `https://`.
3) Ensure `web2md` is installed:
- Run: `command -v web2md`
- If missing, instruct the user to install it (assume the project exists at `~/workspace/softaworks/projects/web2md`):
- `cd ~/workspace/softaworks/projects/web2md && npm install && npm run build && npm link`
- Or: `cd ~/workspace/softaworks/projects/web2md && npm install && npm run build && npm install -g .`
4) Convert:
- Single URL → file:
- `web2md '<url>' --out ./page.md`
- Single URL → auto-named file in directory:
- `mkdir -p ./out && web2md '<url>' --out ./out/`
- Human verification / login walls (interactive):
- `mkdir -p ./out && web2md '<url>' --interactive --user-data-dir ./tmp/web2md-profile --out ./out/`
- Then: complete the check in the browser window and press Enter in the terminal to continue.
- Print to stdout:
- `web2md '<url>' --print`
- Multiple URLs (batch):
- Create output dir (e.g. `./out/`) then run one `web2md` command per URL using `--out ./out/`
5) Validate output:
- If writing files, verify they exist and are non-empty (e.g. `ls -la <path>` and `wc -c <path>`).
6) Return:
- The saved file path(s), or the Markdown (stdout mode).
## Defaults (recommended)
- For most pages: `--wait-until networkidle2`
- For heavy apps: start with `--wait-until domcontentloaded --wait-ms 2000`, then add `--wait-for 'main'` (or another stable selector) if needed.
© softaworks · MIT · softaworks/agent-toolkit
Similar skills
All skills for Browser and scraping →web-to-markdown by other authors comes first.