# harvard-artifacts-data-engineering-pipeline

Pulls artifact records from the Harvard Art Museums API, loads them into SQL tables, and renders a Streamlit dashboard on top.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 2000 installs · ★ 5
- Author: reason-machines
- License: LicenseRef-Unknown · policy=link-only · description only; the file stays with the author
- Source: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-artifacts-data-engineering-pipeline/SKILL.md
- Updated: 2026-10-09
- HTML: https://affpapa.org/en/nevibecode/skills/harvard-artifacts-data-engineering-pipeline

## What it does

The skill fetches museum artifacts from the Harvard Art Museums API with paged requests and rate limiting, then splits the nested JSON into three relational tables covering item metadata, media files, and color palettes. Storage runs on MySQL or TiDB Cloud, with a schema of three linked tables: `artifactmetadata`, `artifactmedia`, `artifactcolors`.

Running it requires an API key from Harvard Art Museums and a `.env` file holding the database credentials. On the Python side it relies on requests for the API calls, pandas for reshaping records into dataframes, and mysql-connector-python for writing rows into the tables.

Once the data lands in SQL, the skill runs more than 20 prebuilt analytical queries and draws interactive Plotly charts inside a Streamlit app. Without that API key and a configured database, the pipeline has nothing to connect to.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-pipeline -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/harvard-artifacts-data-engineering-pipeline/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/harvard-artifacts-data-engineering-pipeline/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-pipeline -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/harvard-artifacts-data-engineering-pipeline/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-pipeline -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/harvard-artifacts-data-engineering-pipeline/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install reason-machines/data-skills harvard-artifacts-data-engineering-pipeline
```

2. **Manually: put the files in place**
- File: `.github/skills/harvard-artifacts-data-engineering-pipeline/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-pipeline -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/reason-machines/data-skills --path skills/harvard-artifacts-data-engineering-pipeline
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/harvard-artifacts-data-engineering-pipeline/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-pipeline -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-artifacts-data-engineering-pipeline/SKILL.md

## When to use it
- set up an ETL pipeline for the Harvard Art Museums API
- split nested artifact JSON into normalized SQL tables
- load a museum collection into MySQL or TiDB Cloud
- build a Streamlit dashboard with Plotly charts over artifact data
- write SQL analytics queries on culture, period, and technique fields

## Example prompts
- Build an ETL pipeline for the Harvard Art Museums API with MySQL storage
- Split artifact JSON into metadata, media, and colors tables
- Set up a Streamlit dashboard with charts over the Harvard artifacts collection

## Similar
- [xlsx](https://affpapa.org/en/nevibecode/skills/xlsx.md) — skill · 187223 installs
- [google-agents-cli-observability](https://affpapa.org/en/nevibecode/skills/google-agents-cli-observability.md) — skill · 511300 installs
- [caveman-evidence-review](https://affpapa.org/en/nevibecode/skills/caveman-evidence-review.md) — skill · 104500 installs
- [migration](https://affpapa.org/en/nevibecode/skills/migration.md) — skill · 103900 installs
- [performance-optimization](https://affpapa.org/en/nevibecode/skills/performance-optimization.md) — skill · 65200 installs
- [supabase](https://affpapa.org/en/nevibecode/skills/supabase.md) — skill · 8 installs

From the same set: [reason-machines/data-skills](https://affpapa.org/en/nevibecode/by/reason-machines/data-skills.md) — 31 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md