Cinemind
● Cinemind is a tool YouTubers drop into their live streams as an OBS Browser Source: their viewers type choices in chat, Gemini 3.5 Flash reads thousands of natural-language messages every ~15 seconds — handling synonym
Core trick
Gemini reads thousands of chat messages every 15 seconds to pick the next pre-rendered scene, which FFmpeg concatenates into a live RTMP stream.
Description
● Cinemind is a tool YouTubers drop into their live streams as an OBS Browser Source: their viewers type choices in
chat, Gemini 3.5 Flash reads thousands of natural-language messages every ~15 seconds — handling synonyms, negations,
and trolls — and decides which pre-rendered scene plays next from a Google Cloud Storage vault, with three coordinated
sub-agents making it work (a Director that picks the next clip with story-aware reasoning, a Live Tally classifier
that updates vote bars every 3 seconds, and a Narrative Summarizer that maintains continuity across a branching DAG
with reconvergence). FFmpeg seamlessly concatenates the chosen clips and pushes a continuous 1080p stream to YouTube
Live's RTMP ingest; a Next.js dashboard renders the live tally, branch map, and Director's reasoning panel so the
audience watches the AI direct in real time. Gemini also intelligently matches image fragments to comments — the
predicted-pane composite layers a different per-story-node reaction overlay (cartoon meme generated by Higgsfield
Kling 3.0, AI-cinematic b-roll generated by Seedance 2.0, or stock footage) that maps chat energy to animation, so
"LETS GO" gets a triumphant cartoon, "💀" gets a Home-Alone-style scared meme, and "WAIT WHAT" gets shocked wide-eyes.
Built on Gemini 3.5 Flash + Google Cloud Storage + YouTube Data API + Veo 3 + Higgsfield + FFmpeg, with three deploy
modes (own broadcast, local HLS demo, embedded as a guest in another creator's stream via scripts/guest_mode.py), the
system turns thousands of strangers into a single collective director — a broadcast-quality experience whose narrative
bends to whoever's watching, achievable for the first time because Flash can classify ~5,000 messages and decide in
under 2 seconds at fractions of a cent per call (~$0.06 per 10-minute segment), and shippable as a creator integration
because the entire chat-read-and-decide pipeline runs on free-tier YouTube APIs against any public live broadcast
without the creator installing anything.
README excerpt
# Cinemind **A YouTuber tool for AI-directed interactive live segments.** A creator goes live, an audience types choices in chat, **Gemini 3.5 Flash reads thousands of natural-language messages every ~15 seconds — handling synonyms, negations, and trolls — and decides which pre-rendered scene plays next** from a Google Cloud Storage vault. Three coordinated sub-agents make it work: a **Director** that picks the next clip, a **Live Tally classifier** that updates vote bars every 3 seconds, and a **Narrative Summarizer** that maintains continuity across a branching DAG with reconvergence. FFmpeg seamlessly concatenates the chosen clips and pushes a continuous 1080p stream to YouTube Live's RTMP ingest. A Next.js dashboard renders the live tally, branch map, and Director's reasoning panel — judges (or viewers) watch the AI direct in real time. **Gemini also intelligently matches image fragments to comments at runtime.** The predicted-pane composite layers a different per-story-node "reaction" video — cartoon meme, AI-generated cinematic b-roll, or stock footage — that maps to the energy of the chat. "LETS GO 🚀" gets a triumphant cartoon, "💀" gets a Home-Alone-style scared meme, "WAIT WHAT 😱" gets shocked wide-eyes. Comment-vibe -> story-node -> animation overlay, all in real time. > **Cinemind is a tool YouTubers drop into their live streams**, not a single static film. Embed it as an OBS Browser Source in any creator's broadcast (see `scripts/guest_mode.py` and `CREATOR_PIT