Skip to main content
skill CX Codex

gallery-scraper — Claude Code skill: bulk-download gated gallery images

gallery-scraper — a Claude skill that collects full-size image URLs from a gallery behind a login and downloads them in bulk through an attached browser tab.

MIT ★ 239 updated 2026-08-30

Install into Claude Code · Skills CLI (Vercel) All methods and platforms ↓
npx skills add jdrhyne/agent-skills --skill gallery-scraper -a claude-code
Open on GitHub

What gallery-scraper does

You attach the tab yourself: sign in to the gallery in Chrome, open the page you want, then click the OpenClaw Browser Relay button until the badge reads ON. The agent then looks for where full-size files live: data-max, data-src, data-full, lightbox markup. Snippets built on querySelectorAll come with the skill, so you are not writing them from scratch.

Pages are walked one at a time, and several galleries can be pulled through a hidden iframe with a load timeout. Downloading goes either as a curl loop with a Referer header and at most eight parallel jobs, or as a Python script on requests and ThreadPoolExecutor. Before the batch starts, the skill advises testing the CDN with curl -I, with and without Referer.

No Chrome with the OpenClaw Browser Relay extension and an attached tab with the badge on means no run at all, because that session is the only place the login exists. A 403 usually calls for Referer or cookies from document.cookie; rate limiting calls for fewer parallel downloads. Cookies and tokens never land in the output files, and the skill stays inside the gallery you picked.

What it helps with

  • pull every image from a gallery that only opens after sign-in
  • recover full-size URLs hidden behind thumbnails and lightboxes
  • walk a multi-page gallery and gather the links into one list
  • download a batch in parallel without tripping the rate limit
  • work out why the CDN returns 403 and add the right Referer

Example prompts for the agent

  • Scrape every image on this gallery page, I've attached the tab
  • Gather the full-size links from all pages and write them to urls.txt
  • Check whether the CDN serves files without a Referer, then download the gallery into ~/Downloads/gallery_name

How to install gallery-scraper

1Skills CLI (Vercel)

npx skills add jdrhyne/agent-skills --skill gallery-scraper -a claude-code

2Manually: put the files in place

~/.claude/skills/gallery-scraper/SKILL.md

Copy the whole skill folder; for one project use .claude/skills/gallery-scraper/

Step by step, with commands for each platform: How to install skills in Claude Code: five ways

Questions about gallery-scraper

How do I install gallery-scraper in Claude Code?

Run npx skills add jdrhyne/agent-skills --skill gallery-scraper -a claude-code. The skill then sits at ~/.claude/skills/gallery-scraper/SKILL.md.

What does gallery-scraper do?

It walks gallery pages in an attached tab, finds the URLs of full-size images, and saves them as files in a local folder.

What do I need before it works?

Chrome with the OpenClaw Browser Relay extension, a signed-in session on the gallery site, and the tab attached so the badge shows ON.

Source text

SKILL.md · MIT
---
name: gallery-scraper
description: Bulk download images from login-protected gallery websites using an attached browser session. Use when asked to scrape, download, or save images from authenticated gallery pages, extract full-size images from thumbnails, or batch download from multi-page galleries.
permissions:
  - exec: "Runs local download commands for the URL list gathered from the attached browser session."
  - file_write: "Creates URL lists and downloaded image files in the user-approved output directory."
  - network: "Uses the attached browser session and direct image downloads against the user-approved gallery domain."
---

# Gallery Scraper

Bulk download images from authenticated gallery websites via browser relay.

## Safety Boundaries

- Do not access gallery sites or user accounts that the user has not explicitly attached and authorized.
- Do not download beyond the selected gallery, profile, or page range without confirmation.
- Do not store cookies, tokens, or hidden form values in local output files.
- Do not keep retrying blocked downloads indefinitely; surface rate limits or auth failures instead.

## Prerequisites

- User must have Chrome with OpenClaw Browser Relay extension
- User must be logged into the target site
- User must attach the browser tab (click relay toolbar button, badge ON)

## Workflow

### 1. Attach Browser Tab

Ask user to:
1. Log into the gallery site in Chrome
2. Navigate to the target gallery/profile page
3. Click the OpenClaw Browser Relay toolbar button (badge shows ON)

### 2. Discover Image URL Pattern

Most gallery sites store full-size URLs in data attributes. Common patterns:

```javascript
// Extract via browser evaluate
() => {
  // Try common patterns
  const patterns = [
    'img[data-max]',           // data-max attribute
    'img[data-src]',           // lazy-load pattern
    'img[data-full]',          // full-size pattern
    'a[data-lightbox] img',    // lightbox galleries
    '.gallery-item img'        // generic gallery
  ];
  
  for (const sel of patterns) {
    const imgs = document.querySelectorAll(sel);
    if (imgs.length > 0) {
      return {
        selector: sel,
        count: imgs.length,
        sample: imgs[0].outerHTML.substring(0, 200)
      };
    }
  }
  return null;
}
```

### 3. Extract Full-Size URLs

Once pattern identified, extract all URLs:

```javascript
// For data-max pattern (common)
() => Array.from(document.querySelectorAll('img[data-max]'))
  .map(img => img.dataset.max)

// For thumbnail→full conversion (replace path segment)
() => Array.from(document.querySelectorAll('.gallery img'))
  .map(img => img.src.replace('/thumb/', '/full/'))
```

### 4. Handle Pagination

Check for multiple pages:

```javascript
() => {
  const pagination = document.querySelectorAll('.pagination a, [class*="page"] a');
  return Array.from(pagination).map(a => ({text: a.textContent, href: a.href}));
}
```

Navigate to each page and collect URLs.

### 4b. Batch scrape multiple galleries (iframe trick)

When you need multiple galleries quickly and can’t automate CDP, you can load each gallery in a hidden iframe and extract `data-max` URLs:

```javascript
async () => {
  const urls = [
    'https://site.example/galleries/view/123',
    'https://site.example/galleries/view/456'
  ];
  const results = [];
  for (const url of urls) {
    const iframe = document.createElement('iframe');
    iframe.style.position = 'fixed';
    iframe.style.left = '-9999px';
    iframe.style.width = '800px';
    iframe.style.height = '600px';
    iframe.src = url;
    document.body.appendChild(iframe);
    await new Promise((resolve, reject) => {
      const t = setTimeout(() => reject(new Error('timeout load')), 20000);
      iframe.onload = () => { clearTimeout(t); resolve(); };
    });
    const doc = iframe.contentDocument;
    const start = Date.now();
    let imgs = [];
    while (Date.now() - start < 20000) {
      imgs = Array.from(doc.querySelectorAll('img[data-max]')).map(i => i.dataset.max);
      if (imgs.length) break;
      await new Promise(r => setTimeout(r, 500));
    }
    results.push({ id: url.split('/').pop(), urls: imgs });
    iframe.remove();
  }
  return results;
}
```

### 5. Check CDN Access

Test if CDN requires authentication or just Referer:

```bash
# Test direct access
curl -I "CDN_URL" 2>/dev/null | head -3

# Test with Referer
curl -I -H "Referer: https://SITE_DOMAIN/" "CDN_URL" 2>/dev/null | head -3
```

### 6. Bulk Download

Collect the URLs into a text file, then parallel download:

```bash
# Create output directory
mkdir -p ~/Downloads/gallery_name

# Download with Referer header (parallel)
cd ~/Downloads/gallery_name
while IFS= read -r url; do
  filename=$(basename "$url")
  curl -s -H "Referer: https://SITE_DOMAIN/" -o "$filename" "$url" &
  [ $(jobs -r | wc -l) -ge 8 ] && wait -n
done < urls.txt
wait
```

**Python ThreadPool fallback (avoids shell quoting + wait -n issues):**

```python
import os
import requests
from concurrent.futures import ThreadPoolExecutor

outdir = os.path.expanduser('~/Downloads/gallery_name')
os.makedirs(outdir, exist_ok=True)
headers = {'Referer': 'https://SITE_DOMAIN/', 'User-Agent': 'Mozilla/5.0'}

with open('urls.txt') as f:
    urls = [line.strip() for line in f if line.strip()]

def download(url):
    filename = os.path.join(outdir, os.path.basename(url))
    if os.path.exists(filename) and os.path.getsize(filename) > 0:
        return
    r = requests.get(url, headers=headers, timeout=60)
    r.raise_for_status()
    with open(filename, 'wb') as f:
        f.write(r.content)

with ThreadPoolExecutor(max_workers=8) as ex:
    for url in urls:
        ex.submit(download, url)
```

## Handling Lock Buttons

Some galleries have "lock" buttons to reveal hidden content. Look for:

```javascript
// Find lock/unlock buttons
() => {
  const locks = document.querySelectorAll(
    '[class*="lock"], [class*="unlock"], ' +
    'button[title*="lock"], .premium-unlock'
  );
  return Array.from(locks).map(el => ({
    tag: el.tagName,
    class: el.className,
    text: el.innerText?.substring(0, 30)
  }));
}
```

Click each lock button before extracting URLs.

## Output Organization

Optionally organize by gallery:

```bash
# Derive a gallery-specific folder name from the selected URL
mkdir -p "gallery_<id>"
```

## Troubleshooting

- **403 Forbidden**: Add Referer header or extract cookies from browser
- **Rate limited**: Reduce parallel downloads, add delays
- **Missing images**: Check for JavaScript-loaded content, may need scroll injection
- **Login required for CDN**: Extract session cookies via `document.cookie`

© jdrhyne · MIT · jdrhyne/agent-skills

Similar skills

All skills for Browser and scraping →

From the same set27

The whole set →
todo-tracker todo-tracker is a Claude Code skill that tracks workspace tasks in a local Markdown file with stable IDs for adding, listing, completing, removing, and summarizing tasks. CX Codex +1 ★ 239repo stars frontend-design frontend-design — a skill that sets the order of UI work: wireframe, theme, animation, then code. Useful for landing pages, dashboards and single components. CX Codex +1 ★ 239repo stars nudocs Connects your agent to Nudocs.ai to upload files, generate collaborative editing links, and pull edited documents back to disk. CX Codex +1 ★ 239repo stars jira This skill searches and reads Jira work, prepares drafts, and makes approved changes. It activates for explicit Jira or Atlassian context, a Jira URL, or an issue key such as PROJ-123. CX Codex +1 ★ 239repo stars gong Fetches call records, transcript snippets, and team stats via Gong API. Useful when inspecting customer conversations and sales activity directly in chat. CX Codex +1 ★ 239repo stars munger-observer The munger-observer skill reviews a concrete decision, plan, or artifact, separating evidence from interpretation and suggesting a check. Use it for a premortem or a blind-spot review. CX Codex +1 ★ 239repo stars context-recovery context-recovery rebuilds the state of your work after compaction or truncation, and shows where each recovered fact came from. CX Codex +1 ★ 239repo stars autoresearch-loop The autoresearch-loop skill helps an agent choose project metrics and test changes in a keep-or-revert cycle. Use it when progress can be measured against an objective. CX Codex +1 ★ 239repo stars

Project sponsors

Companies that keep this analytics open