Sahayak-AI
In India, filling a form assumes you can read and write.
Core trick
Live voice session reads a form aloud and fills fields via tool calls, reading Aadhaar/PAN from camera frames and confirming values back.
Description
In India, filling a form assumes you can read and write. Hundreds of millions can't — not the form's language, sometimes not any script — so a pension, a bank account, or a ration card depends on trusting a stranger to fill it for them.
Sahayak changes who does the reading and writing. Prop a phone on the table, tap Start, and talk. A live Gemini session greets you in your language, reads the form aloud, and fills it field by field: for a name or date of birth, it asks you to show your Aadhaar/PAN to the camera and reads it off the card (printed text, far more reliable than handwriting); for a nominee, it simply asks and takes your voice. Every value appears on screen and the agent confirms it back in your language.
README excerpt
# Sahayak-AI **An Indian-native, voice-first agentic layer for official paperwork.** A person who can't read or write completes a real government/finance form by *speaking* and *showing their documents* to a camera — and watches it fill itself. Built for the Google DeepMind Bangalore Hackathon (PS1 — Live API / Live Translate). Sahayak is not a chatbot that answers questions. It **completes a task**. An agent built on the Gemini Live API: - **Perceives** — reads your Aadhaar / PAN through the camera *and* hears your voice. - **Acts** — writes structured form fields via tool-calls, live on screen. - **Adapts** — follows you across six Indian languages mid-conversation, no toggle. - **Confirms** — reads every value back in your language before committing it, so a misread is caught by *you*, not discovered at rejection. - **Generalizes** — an admin uploads *any* blank form; Gemini vision parses it into a field-map; anyone can then fill it by talking. Forms are the "apps". Design thesis: **perception is probabilistic; the record is deterministic.** The Live session sees / hears / speaks; the proxy owns an append-only, hash-chained **capture log** recording every confirmed value with its source (`document` | `voice`). Languages today: Hindi, Tamil, Telugu, Bengali, Marathi, Kannada (`hi ta te bn mr kn`). ## The loop 1. Agent greets in the applicant's language and asks for the first field. 2. Per template field: **document** field → agent asks for the card, reads it via