Skip to content

NaviSwarm

NaviSwarm is an AI-powered spatial intelligence assistant that runs directly in your mobile browser.

Code Video Cerebral Valley

Core trick

Parallel Gemini agents read the live camera feed to build a spatial map and overlay AR direction arrows without GPS.

Tags

accessibilityvisionagents

Description

NaviSwarm is an AI-powered spatial intelligence assistant that runs directly in your mobile browser. It uses the Gemini Multimodal API to autonomously map your physical environment in real-time, providing vision-impaired or lost users with augmented reality (AR) guidance, natural conversational directions, and live hazard detection—all without requiring GPS or dedicated hardware.
Jimmy291/Gemini3Hackathon★ 00 forksJavaScriptCreated Mar 7, 2026 · last push Mar 7, 2026

README excerpt

# 🧭 NaviSwarm — AI-Powered Indoor AR Navigation

> **Real-time indoor wayfinding using a multi-agent AI swarm, powered by the Gemini Multimodal Live API.**

![NaviSwarm Demo](naviswarm/Demo.gif)

NaviSwarm transforms your phone's camera into an intelligent indoor navigation system. Instead of relying on GPS (which fails indoors), NaviSwarm uses a coordinated swarm of AI agents — built on Google's Gemini 2.5 Flash and the Gemini Multimodal Live API — to continuously map your surroundings, understand spatial context, and guide you to your destination with real-time AR directional overlays.

---

## ✨ Key Features

### 🤖 Multi-Agent Swarm Architecture
NaviSwarm deploys specialized AI agents that work in parallel:
- **Scout Agent** — Continuously scans the live camera feed to identify and memorize architectural landmarks (e.g., *"Red Double Doors"*, *"Restroom Sign"*) and hazards (*"Wet Floor"*, *"Broken Elevator"*), building a persistent spatial memory database in real-time.
- **Coordinator Agent** — When the user asks a question (text or voice), this agent cross-references the live camera view with the spatial memory to determine the best action, providing concrete directional commands and AR overlays.

### 🎙️ Gemini Multimodal Live API Integration
- **Real-time voice conversation** — Speak naturally to NaviSwarm and receive instant audio responses via the Gemini Live API with low-latency bi-directional audio streaming.
- **Continuous vision uplink** — Camera frames are streame

Team