hand-gestures — Claude Code skill for webcam controls
The hand-gestures Claude Code skill adds hand-gesture controls to browser games and apps. The agent uses it for webcam tracking and gesture recognition on a web page.
MIT ★ 220 updated 2026-11-03
npx skills add abouchedidxcode/open-source-hands --skill hand-gestures -a claude-codeWhat hand-gestures does
The skill guides an agent through building a browser game controlled by hand gestures or adding hand tracking to an existing app. It uses hand-gesture-kit on top of MediaPipe to detect poses, motions, and two-hand gestures. Examples include a fist, a swipe, a pinch, a clap, and zooming.
Camera access requires HTTPS or localhost, plus a user click. An app integration needs @mediapipe/[email protected], the hand_landmarker.task model, and WASM files; pin the library to 1.1.0 to match the copied WASM. The skill also covers mouse fallback controls and testing gesture handlers with synthetic input, without a webcam.
What it helps with
- add hand controls to a browser game
- recognize swipes and pinches in a web app
- map a clap or two-hand gesture to an action
- test gesture handlers without a webcam
Example prompts for the agent
- Add webcam hand controls to this browser game.
- Connect hand-gesture recognition to my web app.
- Test the swipe and pinch handlers without a webcam.
How to install hand-gestures
1Skills CLI (Vercel)
npx skills add abouchedidxcode/open-source-hands --skill hand-gestures -a claude-code2Manually: put the files in place
~/.claude/skills/hand-gestures/SKILL.mdCopy the whole skill folder; for one project use .claude/skills/hand-gestures/
1Skills CLI (Vercel)
npx skills add abouchedidxcode/open-source-hands --skill hand-gestures -a codex2Manually: put the files in place
.agents/skills/hand-gestures/SKILL.mdFor all projects use ~/.agents/skills/
1Skills CLI (Vercel)
npx skills add abouchedidxcode/open-source-hands --skill hand-gestures -a cursor2Manually: put the files in place
.cursor/skills/hand-gestures/SKILL.mdCursor also reads .agents/skills/ and .claude/skills/
1GitHub CLI
gh skill install abouchedidxcode/open-source-hands skills/hand-gestures2Manually: put the files in place
.github/skills/hand-gestures/SKILL.md1Skills CLI (Vercel)
npx skills add abouchedidxcode/open-source-hands --skill hand-gestures -a gemini-cli2Platform CLI
gemini skills install https://github.com/abouchedidxcode/open-source-hands --path skills/hand-gesturesManually: put the files in place
.devin/skills/hand-gestures/SKILL.mdThe legacy .windsurf/skills/ path still works
Claude.ai
Zip the folder and upload it in Settings → Capabilities → Skills
Step by step, with commands for each platform: How to install skills in Claude Code: five ways
Questions about hand-gestures
How do I install hand-gestures in Claude Code?
Run npx skills add abouchedidxcode/open-source-hands --skill hand-gestures -a claude-code. The skill file is installed at ~/.claude/skills/hand-gestures/SKILL.md.
What does hand-gestures do?
It guides browser games and apps that use webcam hand controls. Gesture recognition uses hand-gesture-kit on top of MediaPipe.
What does hand-gestures need to work?
Camera access requires HTTPS or localhost and a user click. An app integration needs @mediapipe/[email protected], the hand_landmarker.task model, and WASM files.
Source text
SKILL.md · MIT
--- name: hand-gestures description: Build webcam hand games and hand-gesture controls for browser apps (poses like fist, point, peace sign, thumbs up; motions like swipes, circles, pinch taps; two-hand clap/zoom) with hand-gesture-kit on top of MediaPipe. Use when the user wants something controlled by their hands through a webcam, hand tracking in the browser, or gesture recognition in a web page. --- # Hand gestures with hand-gesture-kit Repo: https://github.com/abouchedidxcode/open-source-hands (MIT). Its `AGENTS.md` is the full guide. Read it before you design anything: https://raw.githubusercontent.com/abouchedidxcode/open-source-hands/main/AGENTS.md ## Making a game Start from the starter game. It already has the camera flow, a mouse and touch fallback, a `?bot=1` player, tests, and screenshot, fps and loudness tools: ```bash npx degit abouchedidxcode/open-source-hands/templates/game my-game cd my-game && npm install && npm run dev ``` Then follow the playbook: https://raw.githubusercontent.com/abouchedidxcode/open-source-hands/main/docs/MAKING-A-GAME.md ## Adding hand gestures to an existing app 1. **Add the kit** (TypeScript source, no dependencies besides MediaPipe): ```bash npx degit abouchedidxcode/open-source-hands/src/kit src/hand-gesture-kit npm install @mediapipe/[email protected] mkdir -p public/models public/wasm curl -L -o public/models/hand_landmarker.task https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task cp node_modules/@mediapipe/tasks-vision/wasm/* public/wasm/ ``` Add the wasm copy to the build script so it survives reinstalls. 2. **Wire it up.** Start the camera from a click (browsers require one) and offer a mouse fallback: ```ts import { createHands, toScreen } from './hand-gesture-kit'; const hands = createHands({ config: { pose: { experimental: false } }, onFrame: (f) => draw(f) }); hands.engine.on('fist', (e) => (e.phase === 'start' ? grab(e.hand!) : release(e.hand!))); hands.engine.on('swipe-left', () => back()); startButton.onclick = () => hands.startCamera().catch(showCameraError); ``` Map positions with `toScreen(hand.features.center)` (0..1). Show the `<video>` mirrored with `transform: scaleX(-1)`. The landmarks are already mirrored. 3. **Pick the gestures** from the events: poses `open fist claw point middle victory three four thumbs-up thumbs-down rock shaka gun pinch ok pinch-middle` (phase start/end); motions `swipe-left/right/up/down push pull circle-cw/ccw wave shake twist-cw/ccw tap double-tap throw hold`; two hands `clap zoom-in zoom-out steer-left steer-right`. Give each action one distinct gesture. 4. **Verify without a webcam.** Use synthetic hands: `hands.inject([syntheticHand(0.5, 0.5, SYNTHETIC_POSES.fist)])` or `hands.feed(clip)`. Unit-test with `new GestureEngine()` and expose a `window.__app` hook to drive the app from Playwright. Don't claim it works until a scripted run shows the gestures firing the right actions. ## Pitfalls - The camera needs https or localhost, and a click. - Don't add extra smoothing. The engine already smooths. - `tap` fires when the fingers part. Record where the pinch happened while `hand.pinch > 0.75`. - Back-of-hand poses are unreliable. Prefer palm-facing gestures. - Keep `@mediapipe/tasks-vision` pinned to 1.1.0 so the JS matches the copied wasm.
© abouchedidxcode · MIT · abouchedidxcode/open-source-hands