# harvard-artifacts-data-engineering-app

A skill that walks an agent through an ETL pipeline on Harvard Art Museums API data, SQL storage and a Streamlit dashboard with Plotly charts.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 2000 installs · ★ 5
- Author: reason-machines
- License: LicenseRef-Unknown · policy=link-only · description only; the file stays with the author
- Source: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-artifacts-data-engineering-app/SKILL.md
- Updated: 2026-10-09
- HTML: https://affpapa.org/en/nevibecode/skills/harvard-artifacts-data-engineering-app

## What it does

The skill covers the whole chain. It pulls museum objects from the Harvard Art Museums API page by page, unpacks the nested JSON, and spreads it across three tables: `artifactmetadata`, `artifactmedia` and `artifactcolors`. From there the rows go into MySQL or TiDB, and a Streamlit dashboard with Plotly charts sits on top.

You get working pieces to start from: a `pymysql` connection helper, table definitions with foreign keys, extract and transform functions, and inserts that refresh existing rows with `ON DUPLICATE KEY UPDATE`. For analysis there are more than 20 ready SQL queries over the collection.

Nothing runs without `HARVARD_API_KEY` and a reachable database, since the app reads `DB_HOST`, `DB_USER`, `DB_PASSWORD` and `DB_NAME` from the environment. The schema targets MySQL or TiDB — the upsert statements will not work on SQLite. Dependencies come from `requirements.txt`: `streamlit`, `pandas`, `requests`, `pymysql`, `plotly` and `python-dotenv`.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-app -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/harvard-artifacts-data-engineering-app/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/harvard-artifacts-data-engineering-app/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-app -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/harvard-artifacts-data-engineering-app/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-app -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/harvard-artifacts-data-engineering-app/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install reason-machines/data-skills harvard-artifacts-data-engineering-app
```

2. **Manually: put the files in place**
- File: `.github/skills/harvard-artifacts-data-engineering-app/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-app -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/reason-machines/data-skills --path skills/harvard-artifacts-data-engineering-app
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/harvard-artifacts-data-engineering-app/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add reason-machines/data-skills --skill harvard-artifacts-data-engineering-app -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/reason-machines/data-skills/tree/HEAD/skills/harvard-artifacts-data-engineering-app/SKILL.md

## When to use it
- build an ETL pipeline from the Harvard Art Museums API into a relational database
- flatten nested object JSON into normalized tables
- stand up a dashboard with charts for a museum collection
- run analytical SQL over artifact metadata and colors
- load rows into MySQL or TiDB while updating existing records

## Example prompts
- Build the pipeline: fetch objects from the Harvard Art Museums API and load them into `artifactmetadata`, `artifactmedia` and `artifactcolors`.
- Create a Streamlit dashboard with Plotly charts for artifact colors and techniques.
- Write SQL queries to analyze the collection by culture and period.

## Similar
- [xlsx](https://affpapa.org/en/nevibecode/skills/xlsx.md) — skill · 187223 installs
- [google-agents-cli-observability](https://affpapa.org/en/nevibecode/skills/google-agents-cli-observability.md) — skill · 511300 installs
- [caveman-evidence-review](https://affpapa.org/en/nevibecode/skills/caveman-evidence-review.md) — skill · 104500 installs
- [migration](https://affpapa.org/en/nevibecode/skills/migration.md) — skill · 103900 installs
- [performance-optimization](https://affpapa.org/en/nevibecode/skills/performance-optimization.md) — skill · 65200 installs
- [supabase](https://affpapa.org/en/nevibecode/skills/supabase.md) — skill · 8 installs

From the same set: [reason-machines/data-skills](https://affpapa.org/en/nevibecode/by/reason-machines/data-skills.md) — 31 cards

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md