# harvard-art-museum-data-engineering

A skill that builds an ETL pipeline for the Harvard Art Museums API: pull artifacts, shape them into tables, load into SQL, and chart them in Streamlit.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 1300 installs · ★ 5
- Author: reason-machines
- License: LicenseRef-Unknown · policy=link-only · description only; the file stays with the author
- Source: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-art-museum-data-engineering/SKILL.md
- Updated: 2026-10-09
- HTML: https://affpapa.org/en/nevibecode/skills/harvard-art-museum-data-engineering

## What it does

The skill lays out the whole path: API → ETL → SQL → analytics → visualization. Artifacts come from the Harvard Art Museums API and split into three tables — metadata, media and colors. From there the data lands in MySQL or TiDB, with a Streamlit dashboard on top.

It ships working code: paged fetching with `requests`, JSON-to-DataFrame conversion with `pandas`, batch inserts through `mysql-connector-python` using `REPLACE INTO` to handle duplicates. The three tables link by `artifact_id` through foreign keys. `plotly` is listed for charts.

You need a Harvard API key before anything runs — register and get one, then put it in `.env` next to the database settings: host, port, user, password, database name. No key means no extraction, so check that first. Dependencies come from `requirements.txt` or the package list.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museum-data-engineering -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/harvard-art-museum-data-engineering/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/harvard-art-museum-data-engineering/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museum-data-engineering -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/harvard-art-museum-data-engineering/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museum-data-engineering -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/harvard-art-museum-data-engineering/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install reason-machines/data-skills harvard-art-museum-data-engineering
```

2. **Manually: put the files in place**
- File: `.github/skills/harvard-art-museum-data-engineering/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museum-data-engineering -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/reason-machines/data-skills --path skills/harvard-art-museum-data-engineering
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/harvard-art-museum-data-engineering/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museum-data-engineering -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-art-museum-data-engineering/SKILL.md

## When to use it
- pull artifacts from the Harvard Art Museums API page by page
- turn API JSON responses into related tables
- load metadata, images and colors into MySQL or TiDB
- build a Streamlit dashboard over a museum collection
- avoid duplicate rows on repeated loads

## Example prompts
- build an ETL pipeline for the Harvard Art Museums API
- extract artifacts with images and colors into MySQL
- create a Streamlit dashboard for the museum collection data

## Similar
- [data-engineering](https://affpapa.org/en/nevibecode/plugins/data-engineering.md) — plugin · ★ 41729
- [analyzing-data](https://affpapa.org/en/nevibecode/skills/analyzing-data.md) — skill · 1400 installs
- [polars](https://affpapa.org/en/nevibecode/skills/polars.md) — skill · 2000 installs
- [data-analysis-jupyter](https://affpapa.org/en/nevibecode/skills/data-analysis-jupyter.md) — skill · 1500 installs
- [analytics-data-analysis](https://affpapa.org/en/nevibecode/skills/analytics-data-analysis.md) — skill · 1100 installs
- [Data Visualization](https://affpapa.org/en/nevibecode/skills/aj-geddes-data-visualization.md) — skill · 568 installs

From the same set: [reason-machines/data-skills](https://affpapa.org/en/nevibecode/by/reason-machines/data-skills.md) — 22 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md