It
It turns any camera into a live music instrument.
Core trick
Gemini Vision turns each camera frame into mood, tempo and key parameters that steer a live Lyria music stream.
Description
It turns any camera into a live music instrument. Point it at something and plays music that matches what it sees — dark room gets ambient drone, bright party gets upbeat pop, nature gets acoustic folk. Generate a creative experience: bridging sight and sound in real-time. Filler music to prevent awkward silence. Like giving someone music on demand. You don't pick the music — the world around you does.
README excerpt
# Vision Music Real-time synesthesia app that watches through your camera and generates music based on what it sees. Gemini Vision analyzes camera frames to understand the scene, mood, and visual characteristics, then Lyria RealTime generates a continuous music stream that evolves as the scene changes.  ## How it works 1. Camera captures frames every 5 seconds 2. **Gemini 2.5 Flash** analyzes each frame — extracts scene description, mood, BPM, brightness, density, and musical scale 3. **Lyria RealTime** generates a live audio stream based on the analysis 4. As the scene changes, the music adapts — new prompts, tempo shifts, key changes ## Setup ```bash npm install ``` Create a `.env` file with your Gemini API key: ``` GEMINI_API_KEY=your_api_key_here ``` ## Run ```bash npm start ``` Open `https://localhost:3000`, click **Start Experience**, and grant camera access. For development with auto-reload: ```bash npm run dev ``` ## Advanced Settings Click **Settings** in the HUD to manually override AI-controlled parameters: - **BPM** — tempo (60–200) - **Brightness** — tonal brightness (0–1) - **Density** — musical density/complexity (0–1) - **Temperature** — generation randomness - **Guidance** — how closely to follow the prompt - **Scale** — musical key (12 options from C major to B major) - **Mode** — Quality, Diversity, or Vocalization - **Mute Bass / Mute Drums / Only Bass & Drums** — stem toggles Each parameter has an **Auto** ch