Jiggle Wiggle
Jiggle Wiggle is a real-time AI movement coach that works with three input modes: YouTube videos, live Zoom calls, and AI-generated references.
Core trick
Runs MediaPipe pose detection in-browser on webcam and reference video frames, scoring per-limb angles and voicing LLM corrections.
Description
Jiggle Wiggle is a real-time AI movement coach that works with three input modes: YouTube videos, live Zoom calls, and AI-generated references. Paste any dance or fitness video URL and the app downloads it, extracts reference poses frame-by-frame, and compares your webcam feed against the dancer in real-time with split-screen skeleton overlays and per-body-part scoring (arms, legs, torso). In Zoom mode, it captures an instructor's video feed and runs pose detection on both streams simultaneously, coaching you through matching their moves. An LLM-powered voice coach provides adaptive spoken feedback in real-time with specific limb-level corrections. Hand gesture controls let you interact without touching the keyboard. All pose detection runs entirely in-browser via MediaPipe WASM with no server-side GPU needed for the core experience, while SAM2 runs on Modal's A100 GPUs for video segmentation. Built with Next.js 16, React 19, TypeScript, OpenAI GPT-4o-mini, Groq, Zoom Meeting SDK.
README excerpt
# Jiggle Wiggle 💃 devpost: https://devpost.com/software/jiggle-wiggle **Real-time AI dance & fitness coaching from any YouTube video.** <img width="1568" height="890" alt="image" src="https://github.com/user-attachments/assets/ee59ebe5-8bf5-477a-b759-f3ea83ae9024" /> Built at TreeHacks 2026. ## What it does Paste any YouTube dance or workout video. Your webcam tracks your body in real-time, scores every move against the reference, and an AI coach gives live audio feedback. When the video ends, you get a full Spotify Wrapped-style performance report. ### Core features - **Auto mode detection** — classifies videos as dance or gym and re-themes the entire UI - **Real-time pose scoring** — geometric comparison against reference frames blended with Groq vision scoring, EMA-smoothed at 30fps - **Score popups** — PERFECT (+25), GREAT (+20), OK (+15), ALMOST (+10), MISS (0) flash on screen with particle effects, accumulating to a total points counter - **Combo streaks** — every 5th consecutive non-miss hit triggers a gold streak celebration - **AI coach** — OpenAI LLM watches your pose summary every few seconds and gives personality-driven audio feedback via TTS - **Gesture controls** — raise both hands to pause/play, no keyboard needed - **Person segmentation** — isolates the dancer from the background using Modal (SAM2) - **AI video generation** — describe a workout or dance and it generates a video (Perplexity research → GPT-4o synthesis → Grok video gen) - **Performance r