Skip to content
3rd Place

Chess Commentary AI

Chess Commentary AI is a real-time AI-powered chess commentator that uses Google Gemini Flash to watch your screen and provide exciting play-by-play commentary.

Code Video Cerebral Valley

Core trick

Sends periodic screen-capture frames to Gemini Flash to read the board state, then speaks the commentary through ElevenLabs TTS.

Tags

visionvoice

Description

Chess Commentary AI is a real-time AI-powered chess commentator that uses Google Gemini Flash to watch your screen and provide exciting play-by-play commentary. It captures your chess game via screen sharing, analyzes the board state using AI vision, and streams live text commentary.
Shreyas-Yadav/Live-Caster★ 01 forksPythonCreated Dec 6, 2025 · last push Apr 4, 2026

README excerpt

# Live Caster

Live Caster is an AI-powered real-time game commentary system that watches your screen, understands the current gameplay frame, and generates short hype-caster style commentary with optional voice narration.

This project won 3rd prize at the Google DeepMind x Cerebral Valley Gemini Hackathon.

## What It Does

- Captures frames from a live shared screen in the browser
- Sends those frames to Gemini for vision-based game understanding
- Generates ultra-short dramatic commentary
- Refines the line into a more theatrical shoutcaster style
- Optionally converts the final commentary into speech using ElevenLabs

The current frontend copy is chess-focused, but the prompt pipeline is already written to support multiple games.

## Demo Flow

1. Start screen sharing from the frontend
2. The app captures a frame every 5 seconds
3. The backend sends the frame to Gemini
4. Gemini returns a short line of commentary
5. A second prompt sharpens the line into dramatic caster-style delivery
6. The app optionally plays back narration using ElevenLabs TTS

## Tech Stack

- Frontend: React + Vite + Tailwind CSS
- Backend: FastAPI + Uvicorn
- Vision + LLM: Gemini 2.5 Flash
- Text to Speech: ElevenLabs

## Project Structure

```text
Live-Caster/
├── backend/
│   ├── main.py
│   ├── gemini_service.py
│   ├── prompts.py
│   └── requirements.txt
├── frontend/
│   ├── src/
│   ├── package.json
│   └── vite.config.js
└── README.md
```

## Requirements

- Python 3.10+
- Node.js 18+
- A G

Team