# webshift

webshift searches the web, removes HTML noise, and returns size-controlled text to an agent. Use it when raw page markup is too costly.

- MCP server · `type=mcp` · Claude Code, Codex, Cursor, GitHub Copilot, Windsurf (Devin Desktop), Cline, Zed · ★ 8
- Author: x-hannibal
- License: MIT · policy=link-only · description only; the file stays with the author
- Source: https://github.com/x-hannibal/webshift
- Updated: 2026-05-09
- HTML: https://affpapa.org/en/nevibecode/mcp/webshift

## What it does

webshift is a Rust MCP server for turning web search results into compact text. Its pipeline can expand a query with an LLM, search through a configured backend, remove duplicate and binary URLs, fetch pages with size caps, clean HTML, and rerank results with BM25. The response is structured JSON.

The server exposes `webshift_query`, `webshift_fetch`, and `webshift_onboarding`. The first runs the search, fetch, cleanup, and ranking flow; the second handles one page. Supported backends include SearXNG, Brave, Tavily, Exa, SerpAPI, Google, Bing, and HTTP. The setup example starts SearXNG with `docker run -d -p 8080:8080 searxng/searxng`.

Install the native binary with Cargo; its executable name is `mcp-webshift`. webshift parses static HTML and does not run JavaScript, so it cannot act as a headless browser for JavaScript-heavy SPAs.

## Requirements
- Needs: a prebuilt binary from the repository releases

## Install

### Claude Code (native platform)

**Install**
- The MCP registry has no launch command for it: see the repository README

## When to use it
- search the web from an AI agent
- strip ads, scripts, and navigation from HTML
- cap page and total response size
- rerank sources with BM25
- fetch clean text from a single URL

## Example prompts
- Use `webshift_query` to research this topic, keep the relevant sources, and summarize them with inline citations.
- Run `webshift_fetch` for this URL and return the page text without navigation or scripts.
- Call `webshift_onboarding` and show the available backends and budget guidance.

## Similar
- [rag-retrieval](https://affpapa.org/en/nevibecode/skills/rag-retrieval.md) — skill · 1500 installs
- [nemotron-retrieval-recipes](https://affpapa.org/en/nevibecode/skills/nemotron-retrieval-recipes.md) — skill · 2000 installs
- [tavily-best-practices](https://affpapa.org/en/nevibecode/skills/tavily-best-practices.md) — skill · 20700 installs
- [exa-web-search-free](https://affpapa.org/en/nevibecode/skills/exa-web-search-free.md) — skill · 2800 installs
- [exa-search](https://affpapa.org/en/nevibecode/skills/exa-labs-exa-search.md) — skill · 1400 installs
- [qdrant-search-quality](https://affpapa.org/en/nevibecode/skills/qdrant-search-quality.md) — skill · 61 installs

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md