# harvard-art-museums-data-engineering

A skill for building ETL pipelines on Harvard Art Museums API data: normalise into MySQL, run SQL analytics, ship a Streamlit dashboard.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 843 installs · ★ 5
- Author: reason-machines
- License: LicenseRef-Unknown · policy=link-only · description only; the file stays with the author
- Source: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-art-museums-data-engineering/SKILL.md
- Updated: 2026-10-09
- HTML: https://affpapa.org/en/nevibecode/skills/harvard-art-museums-data-engineering

## What it does

The skill walks an agent through the whole path from an API request to an interactive dashboard. It covers paged fetching of objects with rate limits in mind, splitting nested JSON into metadata, image and colour tables, then querying the result.

Inside you get `requests` and `pandas` code, a MySQL/TiDB schema with foreign keys, a set of analytical queries, and a Streamlit app with Plotly charts. You need a Harvard Art Museums API key, a MySQL database, and environment variables in `.env`: `HARVARD_API_KEY`, `DB_HOST`, `DB_PORT`, `DB_USER`, `DB_PASSWORD`, `DB_NAME`. Without the key the API calls fail outright, so get one before the first run.

The pipeline assumes MySQL or TiDB. On Postgres you rewrite the schema and the connection code. Dependencies come from `requirements.txt` plus `streamlit`, `pandas`, `requests`, `mysql-connector-python`, `plotly`, `python-dotenv`.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museums-data-engineering -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/harvard-art-museums-data-engineering/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/harvard-art-museums-data-engineering/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museums-data-engineering -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/harvard-art-museums-data-engineering/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museums-data-engineering -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/harvard-art-museums-data-engineering/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install reason-machines/data-skills harvard-art-museums-data-engineering
```

2. **Manually: put the files in place**
- File: `.github/skills/harvard-art-museums-data-engineering/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museums-data-engineering -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/reason-machines/data-skills --path skills/harvard-art-museums-data-engineering
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/harvard-art-museums-data-engineering/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-art-museums-data-engineering -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-art-museums-data-engineering/SKILL.md

## When to use it
- pull museum objects from the Harvard Art Museums API page by page
- flatten nested JSON into normalised tables
- design a MySQL schema with foreign keys for artifact metadata
- run SQL analytics over a collection
- build a Streamlit dashboard with Plotly charts

## Example prompts
- Build an ETL pipeline for Harvard Art Museums API artifacts and load them into MySQL
- Write SQL queries for collection analytics and show the results in Streamlit
- Design tables for artifact metadata, images and colors

## Similar
- [data-engineering](https://affpapa.org/en/nevibecode/plugins/data-engineering.md) — plugin · ★ 41729
- [analyzing-data](https://affpapa.org/en/nevibecode/skills/analyzing-data.md) — skill · 1400 installs
- [polars](https://affpapa.org/en/nevibecode/skills/polars.md) — skill · 2000 installs
- [sql-pro](https://affpapa.org/en/nevibecode/skills/httpsgithubcomjeffallan-sql-pro.md) — skill · 6400 installs
- [database-optimizer](https://affpapa.org/en/nevibecode/skills/httpsgithubcomjeffallan-database-optimizer.md) — skill · 6000 installs
- [mysql-patterns](https://affpapa.org/en/nevibecode/skills/mysql-patterns.md) — skill · 4700 installs

From the same set: [reason-machines/data-skills](https://affpapa.org/en/nevibecode/by/reason-machines/data-skills.md) — 31 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md