NewBalance
You ask a question out loud.
Core trick
Continuous speech recognition triggers image and animation generation, and interruptions re-prompt the existing scene with the changed physical rule.
Description
You ask a question out loud. ChalkTalk listens, understands what's being asked, and draws it — images in seconds, animation moments later. Nothing is typed, nothing is prepared, and no slides exist. When someone asks a follow up question, for example by interrupting with "but what if the sun wasn't there?", ChalkTalk stops mid-sentence, works out what removing the sun physically does, and redraws the scene under the new rule. The class sees the consequence along with being told about it.
README excerpt
# ChalkTalk — explain it out loud, watch it appear
A hands-free, speech-driven classroom assistant. A teacher explains a concept out loud and
ChalkTalk draws it — a bright cartoon **image** (Nano Banana) or a short **animation** (Omni) — on
the board in real time. Anyone in the room can speak up ("what if the sun disappears?") and it
refines the media, reasoning about the real physics rather than just editing the picture. The mic
is always listening: no button to push per turn, and you can barge in mid-sentence.
## Repository layout
| Path | What it is |
| --- | --- |
| `app/` | FastAPI backend — the Gemini Live session, tool-calling loop, image/video generation |
| `static/` | The interactive app (mic capture, board, transcript) served by FastAPI |
| `web/` | Separate **Next.js** marketing landing page |
## Prerequisites
- [**uv**](https://docs.astral.sh/uv/) (manages Python 3.13 and dependencies)
- **ffmpeg** — the video path stitches clips with it (`brew install ffmpeg` / `apt install ffmpeg`)
- **Node 18+** — only to run the `web/` landing page
- A **Gemini API key** with access to the hackathon preview models
## Run the backend locally (the actual app)
```bash
uv sync # install Python deps (fetches Python 3.13 if needed)
cp .env.example .env # then edit .env and set GEMINI_API_KEY=... (GOOGLE_API_KEY also works)
uv run main.py # starts uvicorn on http://localhost:8000 (auto-reload)
```
Open **http://localhost:8000**,