Skip to content
Finalist

SaralPhone

My grandfather recently walked 2km to a hospital because he couldn't book a cab—the ads and clutter on the Ola app made him feel helpless.

Code Demo Cerebral Valley

Core trick

Android overlay drives real apps via an agent that reads the live screen and replaces cluttered UI with a few large buttons.

Tags

accessibilityvoiceagents

Description

My grandfather recently walked 2km to a hospital because he couldn't book a cab—the ads and clutter on the Ola app made him feel helpless. This "digital paralysis" affects millions of elderly Indians who own smartphones but fear the complexity of modern apps like Zomato and UPI. We built a "Dignity Layer"—an intelligent Android overlay powered by Gemini. Acting as a digital "Beta" (son), it bridges the gap between intent and action. The user simply speaks in their local language (e.g., "Beta, hospital jana hai"). Our system launches the real app in the background, interprets the live screen using Gemini, filters out the ads and noise, and renders a clean, high-contrast overlay with just the essential buttons needed for that task. This restores control to the user, allowing them to navigate the complex modern web through a simplified, safe lens.
SinghCoder/gemini3-hackathon-bengaluru★ 00 forksKotlinCreated Feb 14, 2026 · last push Feb 14, 2026

README excerpt

# 🙏 SaralPhone — AI Phone Assistant for Elderly Users, in Their Language

## Overview
SaralPhone is an AI-powered Android overlay that makes smartphones accessible to elderly users across regions and languages. Users speak naturally in their preferred language, and the AI agent autonomously navigates apps (Swiggy, Zomato, Ola, Uber, GPay, etc.) on their behalf, presenting simplified choices at decision points with large, clear local-language buttons.

## Key Features
- **Voice-first interaction**: Speak naturally in your preferred language
- **Autonomous app navigation**: AI agent taps, types, scrolls through apps automatically
- **Simplified overlay UI**: At decision points, shows 2-5 big local-language buttons with emoji
- **Custom input**: Users can type or speak a custom choice via the "Other" option
- **Locale-aware adaptation**: UI and assistant responses adapt to user language and local context
- **Multi-app support**: Food ordering (Swiggy/Zomato), cab booking (Ola/Uber), payments (GPay/PhonePe), messaging (WhatsApp), and more
- **Stop anytime**: Red stop button to abort agent execution instantly

## Architecture

```
User speaks in their preferred language
    → Android SpeechRecognizer (on-device STT)
    → LLM classifies intent (Gemini)
    → Launches target app via AccessibilityService
    → Agentic loop:
        → Reads screen state (accessibility tree)
        → LLM decides next action (tap/type/scroll/show_ui)
        → Executes action via AccessibilityService

Team