# harvard-artifacts-collection-analytics-pipeline

A skill that walks your agent through a full Harvard Art Museums API pipeline: ETL, SQL storage and a Streamlit dashboard.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 1900 installs · ★ 5
- Author: reason-machines
- License: LicenseRef-Unknown · policy=link-only · description only; the file stays with the author
- Source: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-artifacts-collection-analytics-pipeline/SKILL.md
- Updated: 2026-10-09
- HTML: https://affpapa.org/en/nevibecode/skills/harvard-artifacts-collection-analytics-pipeline

## What it does

The skill drives a project along one path: API → ETL → SQL → analytics → visualization. You supply a Harvard Art Museums API key, and the agent writes the extraction code, reshapes the JSON into metadata, media and color tables, then loads them into MySQL or TiDB Cloud. Three tables carry the data — `artifactmetadata`, `artifactmedia`, `artifactcolors` — joined on `artifact_id`.

Inside are Python building blocks: `requests` with a pause between pages so the API rate limit does not bite, `pandas` for the transform, `mysql-connector-python` for the connection, plus `plotly` and `streamlit` for the dashboard. Fetching is paginated and limited to objects that have images.

Before anything runs you need a `.env` file with `HARVARD_API_KEY` and the database settings: `DB_HOST`, `DB_PORT`, `DB_USER`, `DB_PASSWORD`, `DB_NAME`. No working MySQL or TiDB Cloud connection means the pipeline stalls at the load step, because there is nowhere to write the tables. Dependencies come from `requirements.txt`.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-collection-analytics-pipeline -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/harvard-artifacts-collection-analytics-pipeline/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/harvard-artifacts-collection-analytics-pipeline/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-collection-analytics-pipeline -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/harvard-artifacts-collection-analytics-pipeline/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-collection-analytics-pipeline -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/harvard-artifacts-collection-analytics-pipeline/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install reason-machines/data-skills harvard-artifacts-collection-analytics-pipeline
```

2. **Manually: put the files in place**
- File: `.github/skills/harvard-artifacts-collection-analytics-pipeline/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-collection-analytics-pipeline -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/reason-machines/data-skills --path skills/harvard-artifacts-collection-analytics-pipeline
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/harvard-artifacts-collection-analytics-pipeline/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-collection-analytics-pipeline -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-artifacts-collection-analytics-pipeline/SKILL.md

## When to use it
- pull artifact metadata from the Harvard Art Museums API page by page
- split nested JSON into metadata, media and color tables
- load the transformed rows into MySQL or TiDB Cloud
- write analytical SQL over the collection
- build an interactive Streamlit dashboard with Plotly charts

## Example prompts
- build an ETL pipeline for the Harvard Art Museums API: extract, transform and load into MySQL
- add SQL queries to analyze the collection by culture and technique
- create a Streamlit dashboard visualizing colors and departments

## Similar
- [polars](https://affpapa.org/en/nevibecode/skills/polars.md) — skill · 2000 installs
- [data-pipeline](https://affpapa.org/en/nevibecode/skills/data-pipeline.md) — skill · 4900 installs
- [ETL Pipeline](https://affpapa.org/en/nevibecode/skills/etl-pipeline.md) — skill · 4800 installs
- [sql-pro](https://affpapa.org/en/nevibecode/skills/httpsgithubcomjeffallan-sql-pro.md) — skill · 6400 installs
- [database-optimizer](https://affpapa.org/en/nevibecode/skills/httpsgithubcomjeffallan-database-optimizer.md) — skill · 6000 installs
- [mysql-patterns](https://affpapa.org/en/nevibecode/skills/mysql-patterns.md) — skill · 4700 installs

From the same set: [reason-machines/data-skills](https://affpapa.org/en/nevibecode/by/reason-machines/data-skills.md) — 22 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md