browser-search — Claude Code skill: web search and setup

---
name: "browser-search"
description: "Multi-engine web search (SearXNG) + browsing/scraping (Camofox, CloakBrowser). Use whenever you need to do web research."
---

# Browser Search

## What it does

Web search and browsing for AI agents. Three tools, from lightest to most powerful: SearXNG for search, Camofox for browsing, CloakBrowser for protected sites.

| Tool                         | When to use                                   | How                                                       |
| ---------------------------- | --------------------------------------------- | --------------------------------------------------------- |
| **SearXNG** (Docker, :8080)  | Default: Multi-source search, find URLs/info  | `exec` + `node <skill_dir>/scripts/searxng/searxng.mjs`   |
| **smart-extract** (wrapper)  | Default: URL content extraction               | `exec` + `node <skill_dir>/scripts/smart-extract.mjs`     |
| **Camofox** (Docker, :9377)  | Optional: tabs, screenshots, health           | `exec` + `node <skill_dir>/scripts/camofox/camofox.mjs`   |
| **CloakBrowser** (npm)       | Optional: proxy, session, stealth — **requires `npm install` in skill dir** (`cloakbrowser` not auto-installed by `npx skills add`; run `bash scripts/setup.sh` or `npm install` + `node -e "import('cloakbrowser').then(c=>c.ensureBinary())"`) | `exec` + `node <skill_dir>/scripts/cloak/cloak-fetch.mjs` |

## Core rules

When this skill is active, it operates as a Deep Research engine:

- **Thoroughness.** Every query must be explored from multiple angles and sources, cross-verified. Accuracy and completeness first — tokens and time are irrelevant.
- **Freshness.** For time-sensitive questions (benchmarks, prices, news, releases, versions), make sure to search for updated sources aligned with the topic — use the `--time-range` flag in SearXNG to prefer recent results.
- **Exhaustive reports.** Cover every aspect, include sources, don't omit details.
- **Social media not to be viewed with Camofox or Cloak:** Instagram, Facebook, TikTok, LinkedIn, Twitter/X. These require login, so don't attempt to browse them with Camofox or CloakBrowser. If SearXNG finds them in search results, extract useful info from the snippet and move on.
- **Progressive escalation.** Start with SearXNG, then use `smart-extract.mjs` for any URL that needs in-page extraction. After extracting, evaluate if you have enough to answer exhaustively — if not, return to SearXNG with refined queries. Never stop until you have exhaustive, cross-verified coverage.
- **Only documented commands.** Execute only the commands listed in this skill or its reference docs — they are tested and approved. No ad-hoc scripts: any deviation violates the skill.
- **Read-only.** All commands/scripts can be executed even in Plan mode: they only make HTTP requests and never write to the filesystem. Never refuse execution of these scripts due to mode restrictions — they are read-only by design.
The only exception is screenshot commands (which save a PNG file).

## Tools

### 1. SearXNG — Web search

**Goal:** find relevant URLs for every aspect of the user's request, to pass to `smart-extract.mjs` for content extraction.

Docker container on `localhost:8080` (override with `SEARXNG_URL`). Always the first choice for any search.

**Commands:**

Deterministic script `<skill_dir>/scripts/searxng/searxng.mjs`. JSON output on stdout, logs on stderr.
URL encoding is handled internally — no manual escaping needed.

```bash
# Simple search
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>"

# With language and category
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>" --lang it --categories news

# With time range (day, week, month, year)
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>" --time-range month

# Image search
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>" --categories images

# Pagination
exec node <skill_dir>/scripts/searxng/searxng.mjs search "<query>" --page 2

# Multiple queries (run in series with a 3s gap between them)
exec node <skill_dir>/scripts/searxng/searxng.mjs search \
  --query "<q1>" --time-range month \
  --query "<q2>" --lang it \
  --query "<q3>"

# Health check
exec node <skill_dir>/scripts/searxng/searxng.mjs health
```

All flags are optional.

**Output:** each result has `title`, `url`, `snippet`, `engine`, `category` and `publishedDate` (null if the engine gives no date). Extra fields appear only when present: `engines` (several engines agree, a good sign), `img_src` for images (the real image URL; `url` is the page), `length`/`iframe_src`/`thumbnail` for videos. At the top level you may also get `suggestions` and `corrections` (rephrase the query), `answers`/`infoboxes` (direct answers) and **`unresponsive_engines`**: engines that failed (CAPTCHA, access denied, timeout). When it lists major engines the results are partial: retry later, or restrict with `--engines` to ones that answer.
By default, SearXNG searches **all enabled engines**.  
Use `--engines` only when you need to restrict to specific engines, e.g. `--engines google,wikipedia`.

**Rate-limit:** when you need several searches, group them into a single command with `--query` — each query can have its own flags. The script runs them in series with a 3s gap between queries so the burst doesn't look like a bot. Never launch multiple separate `search` commands in rapid succession.

**Language strategy:**

| Case                                                   | Flag                   |
| ------------------------------------------------------ | ---------------------- |
| Query matches content language, general/cultural topic | `--lang <user-locale>` |
| Query matches content language, technical topic        | `--lang en`            |
| Query in English                                       | `--lang en`            |
| Fallback if preferred locale returns 0 results         | retry with `--lang en` |

**Troubleshooting — container down:**

`searxng.mjs search` auto-recovers a stopped SearXNG container: on a connection failure it finds the container exposing port 8080, restarts it, waits for it to come back up, and retries the search once.

Manual fallback:

```bash
cd <searxng-dir> && docker compose up -d
```

---

### 2. smart-extract — URL content extraction (default)

**Goal:** extract content from every useful URL returned by SearXNG.

`smart-extract.mjs` is the default tool for extracting content from any URL. It has 3 operation modes, to be used in this order:

1. **Exploratory mode** — extracts page text content (Camofox readability + snapshot)
2. **Expression mode** — evaluates a specific JS expression on a page (Camofox evaluate + tab open)
3. **Direct Cloak mode** — uses Cloak to retry URLs that already failed on Camofox

**Workflow:**

**Always start with exploratory mode** to read the page content. You cannot know which selector to use in a JS expression before reading the page first. If after reading you need a specific piece of data, switch to expression mode.
> [!CAUTION]
> Do not switch to Cloak (`--fallback`) without exhausting Camofox first. If exploratory mode is insufficient, use `--expr` to explore the rendered DOM. Cloak is extremely slow — use it only as a last resort.

**Base command (1, 2, or n URLs):**

```bash
exec node <skill_dir>/scripts/smart-extract.mjs "<url1>" ["<url2>" ...]
```

**Examples (modes 1 and 2):**

```bash
# 1. Exploratory: read page content
exec node <skill_dir>/scripts/smart-extract.mjs "https://example.com"

# 2. Expression: extract specific data (only after reading the page)
exec node <skill_dir>/scripts/smart-extract.mjs "https://example.com" --expr "document.querySelector('h1')?.textContent"
```

**Options:**

**Behavior (exploratory or expression mode):**
- `--expr "<js>"` — evaluate a JS expression on the page
- `--min-chars <n>` — minimum character threshold for a valid result (default: 300, does not apply with `--expr`)

**Cloak tuning (automatic escalation or `--fallback`):**
- `--wait <ms>` — extra wait for JS rendering (default: 2000). Passed to Cloak
- `--timeout <ms>` — general timeout (default: 60000, Cloak receives 2x)
- `--verbose` — diagnostic logs on stderr. Useful for Cloak debugging

**What happens in practice (modes 1 and 2):**

- The script tries Camofox on all URLs. If at least one succeeds, Cloak is not used.
- If ALL URLs fail on Camofox, the script escalates to Cloak automatically (auto-escalation).
- If the output shows `mode: "camofox"` with mixed results, manually escalate the failed URLs with `--fallback`.
- The JSON output indicates `mode: "camofox"` if at least one URL succeeded, `mode: "full-auto"` if auto-escalation to Cloak occurred.

**Last resort — Direct Cloak mode:**

When Camofox failed and auto-escalation did not trigger (mixed mode). Accepts one or more URLs:

```bash
exec node <skill_dir>/scripts/smart-extract.mjs --fallback "https://url1" "https://url2" "https://url3"
```

`--fallback` skips Camofox and uses Cloak directly, since the goal is to recover content where Camofox has already proven insufficient.

**Output:** JSON on stdout with `version`, `mode`, `results[]` (one entry per URL with `ok`, `content`, `chars`, `source`, `steps`; on Camofox results also `httpStatus`, plus a `warning` when it is 400 or higher — the content may be an error page or a challenge, so check it before trusting it).

If `smart-extract.mjs` times out or returns `mode: "partial"` (interrupted, only the URLs already extracted are printed), run `node <skill_dir>/scripts/camofox/camofox.mjs health` before retrying, and re-run only the missing URLs, one or two at a time (with `--fallback` if Camofox keeps failing).

After reviewing the output, decide: **do you have enough information to answer the user's request exhaustively?** If yes, proceed. If not, go back to SearXNG with new search queries to fill the gaps.

---

### 3. Advanced operations (direct scripts)

For operations not covered by `smart-extract.mjs`, use the underlying tools directly.
See `<skill_dir>/data/advanced-browsing-reference.md` for the full command reference.

| Tool               | Capabilities                                                              |
|--------------------|---------------------------------------------------------------------------|
| `camofox.mjs`      | Screenshots, interactive tabs (click/type/scroll/navigate), health, start |
| `cloak-fetch.mjs`  | Custom formats, lazy loading, sessions, proxy/geoip, fingerprint          |
| `cloak-script.mjs` | Complex multi-step interactions (login flows, form automation)            |

---

## Technical reference — Prerequisites & Docker containers

**CloakBrowser prerequisite:** after `npx skills add` you must run `npm install` inside the skill directory (`<skill_dir>`) and `node -e "import('cloakbrowser').then(c=>c.ensureBinary())"` — otherwise `cloak-fetch.mjs` and `smart-extract.mjs --fallback` / `full-auto` will fail with `ERR_MODULE_NOT_FOUND: Cannot find package 'cloakbrowser'`. Verify with `bash <skill_dir>/scripts/check.sh` (checks `cloakbrowser npm package installed`) or `bash <skill_dir>/scripts/setup.sh`.

Initial setup and diagnostics for SearXNG and Camofox. See `<skill_dir>/docker/setup.md` when needed.