Skip to content

The Dining Philosophers

**AURA (Phone Saathi)** is a hybrid, autonomous Android agent that navigates your phone for you.

Code Demo Cerebral Valley

Core trick

Gemini Live drives a sense-decide-act-check loop over Android accessibility nodes, halting before payment screens and falling back to local Gemma.

Tags

accessibilityvoice

Description

**AURA (Phone Saathi)** is a hybrid, autonomous Android agent that navigates your phone for you. You speak naturally to it using the **Gemini Live API**, and it executes complex app workflows step-by-step. **Why we chose this problem:** In regions like India, millions face a steep digital literacy curve and struggle with complex app interfaces, yet internet connectivity remains spotty and privacy around tasks like payments is critical. We built AURA to solve this dual challenge by targeting two tracks: 1. **Real-Time Multimodal Interaction:** We use Gemini Live to break the rigid, turn-based chatbot model, giving users a fluid, interruptible voice assistant that truly understands intent. 2. **Local-First Agents on Gemma 4:** We brought the "sense-decide-act-check" loop directly to the edge. When privacy or connectivity demands it, AURA falls back to a local Gemma 4 model (via LiteRT-LM) to execute tasks entirely offline, halting automatically before sensitive payment boundaries.
perfect7613/project-aura-deepmind★ 00 forksHTMLCreated Jul 11, 2026 · last push Jul 14, 2026

README excerpt

# Saarthi (सारथी)

The primary product is now a native Android 10+ application in `android-app/`. It runs on the phone, uses Android Accessibility APIs for semantic observation and safe actions, and calls Gemini directly from the APK. A laptop, Python, ADB, and USB are not needed after the APK is installed. The older Python/ADB Phone Saathi prototype remains in `saathi/` as a reference and deterministic test harness.

## Native Android APK

Requirements: Android Studio with JDK 17 and Android SDK 35. Open the `android-app` directory as an Android Studio project, let Gradle sync, then choose **Build → Build APK(s)**. Install the resulting debug APK from:

```text
android-app/app/build/outputs/apk/debug/app-debug.apk
```

You can also build and install from a terminal after Android Studio has supplied the Gradle wrapper/SDK:

```bash
cd android-app
./gradlew test assembleDebug
adb install -r app/build/outputs/apk/debug/app-debug.apk
```

On the phone:

1. Open **Saarthi → Configure Gemini API**, enter a newly issued hackathon key, and save it.
2. Open **Saarthi → Open accessibility settings** and enable **Saarthi phone control**.
3. Return to Saarthi. The visible status must say that Gemini is configured.
4. Use the large Start button. Saarthi is restricted to WhatsApp and Google Maps in this MVP.
5. Review every send/share/call/delete confirmation. Silence never grants permission.

### Example: Zomato preparation flow

Say: **“Open Zomato, search for biryani, and add one suita

Team