Skip to content

Setu

Setu - Never letting love get lost in translation.

Core trick

A native-audio Live API session translates each utterance aloud while a separate function call returns speaker identity and emotional tone for on-screen display.

Tags

Description

Setu - Never letting love get lost in translation. Every multigenerational Indian family lives this gap: grandparent speaks Kannada ( any native language) , grandchild speaks English, and love keeps getting lost - not because the words are wrong, but because the feeling behind them never makes it across. Worry sounds like nagging. Exhaustion sounds like coldness. Nobody meant what the other person heard. Setu is built to close that gap. It listens live to two people speaking different languages and, for every single thing either person says, translates it aloud in real time, reads the emotional tone behind the words - not just what was said, but how - and reports it plainly above that person's side of the conversation: "Setu feels: she sounds angry, but she's just concerned - talk to her gently." It runs on Gemini 2.5's native-audio model over the Live API with Affective Dialog enabled, so the tone-reading is genuinely native to the model's audio understanding, not a bolted-on classifier. Speaker identity and emotional tone are surfaced through a mandatory function call kept separate from the spoken translation, so Setu's own read never gets spoken aloud over the conversation - it just sits quietly on screen, exactly where a listener needs it. This isn't a translator. It's an emotional interpreter - the thing that lets two people who love each other, and speak different languages, actually hear each other. Because at the end of the day - love is what matters, and with SETU ( Sanskrit ~ Bridge ) we never let love get lost in translation :)
MadhumithaKolkar/SETU★ 20 forksPythonCreated Jul 11, 2026 · last push Jul 12, 2026

README excerpt

# सेतु - Setu

**An emotionally-aware intergenerational language bridge.**

*Never Letting love get lost in translation :)*

[![Watch the demo](https://img.youtube.com/vi/OATfyO08xvc/maxresdefault.jpg)](https://youtu.be/OATfyO08xvc)

Built for the Google DeepMind Bangalore Hackathon — Track 2: Real-Time Multimodal Interaction (Gemini Live API).

---

## What It Does

Setu listens live to two people speaking different languages — Person A and Person B, each with their own column — and for every single thing either person says:

1. **Translates** it naturally and warmly into the other person's language, spoken aloud in real time
2. **Reads the emotional tone** behind the words — not just what was said, but how
3. **Reports it plainly**, every time, as *"Setu feels: ..."* above that person's column

*"Setu feels: She sounds angry, but she's just concerned — talk to her gently."*
*"Setu feels: He's asking because he's worried, not judging you — reassure him."*

This is the gap that exists in every Indian multigenerational family — grandparent and grandchild, different languages, and the emotional register that gets lost in translation between them. Setu bridges both the words and the feeling behind them.

---

## Tech Stack

- **Gemini 2.5 native-audio (Live API, `v1alpha`)** — continuous, bidirectional audio streaming with **Affective Dialog** (native vocal-tone understanding)
- **Function calling** — the model reports speaker + emotional tone via a mandatory `report_tone` tool 

Team