Skip to content

Gemini Clarity

I built Clairity around the managed agents pattern.

Code Demo Cerebral Valley

Core trick

Sub-agents watch camera frames and narrate only when scene changes matter, while a live voice session handles questions.

Tags

accessibilityvisionvoice

Description

I built Clairity around the managed agents pattern. Three Gemini 3.5 Flash sub-agents and gemini-3.1-flash-live-preview (for live visual) run in the background while Gemini Live handles the voice loop. One watches the camera and narrates only when something matters. One captions and embeds moments the wearer wants to remember. One classifies every wake word utterance in 150 ms. Together they feel like one assistant.
AryanFP/Project-V★ 00 forksTypeScriptMITCreated May 23, 2026 · last push May 23, 2026

README excerpt

airity

**A vision assistant for blind and low-vision wearers, running on Mentra Live smart glasses.**

Built at the [Google I/O Hackathon](https://cerebralvalley.ai/e/google-io-hackathon/) — Shack15, San Francisco — on **May 23, 2026**, against the prompt:

> *Build something that's never been built before with Gemini 3.5 Flash.*

---

## What it does

Clairity is the wearer's eyes. The glasses see what they see, and a small team of Gemini models work together to:

1. **Narrate the world proactively** — when something meaningful changes in front of the wearer (a person approaches, a hazard appears, they enter a new room), Clairity speaks up on its own. When nothing has changed, it stays silent.
2. **Answer questions on demand** — *"Hey Gemini, what am I looking at?"* / *"Hey Gemini, what time is it?"* triggers a wake-word Q&A turn with the live camera frame attached.
3. **Remember moments** — *"Hey Gemini, remember this"* captures the current scene plus the surrounding speech, captions it with Gemini 3.5 Flash, extracts structured entities, and embeds the result into a long-term visual memory.
4. **Recall on natural-language query** — *"Hey Gemini, where did I put my walkie-talkie?"* runs semantic vector search across stored memories and answers with the time and place.

All four behaviors share one user-facing surface: the glasses speaker. The wearer never sees a screen.

---

## The Gemini 3.5 Flash story

Clairity uses **three distinct Gemini surfaces, each chosen for wha

Team