Magellan
*DUE TO TIME RESTRCITIONS ONLY THE LIVE NAVIGATION and assitant features have been demoed on the video.
Core trick
Live camera and voice stream feed Gemini, and image-to-image translation rewrites signs into a translated replica keeping the original layout.
Description
*DUE TO TIME RESTRCITIONS ONLY THE LIVE NAVIGATION and assitant features have been demoed on the video. ALL THE MENTIONED FEATURES ARE THERE IN THE READY LIVE PRODUCT*
Executive Summary
The Core Problem Solved
Traditional AI chatbots and assistants operate in a situational vacuum—they have no continuous, real-time context of your physical environment, ongoing travel timeline, or ambient actions.
The Solution: Project Magellan
Magellan is an advanced agentic spatial-AI travel companion that continuously observes, listens, and remembers. It bridges digital intelligence with physical environments to offer hands-free voice guides, real-time spatial translations, navigation routing, historical context, and localized weather.
Key Technical Specs & Core Features
Model Orchestration:
Gemini 3.5 Flash: Orchestrates transcription, text/voice queries, and ambient reasoning.
Imagen 3 (Omni Image Gen): Recreates photorealistic landmarks and objects based on user interactions.
Nano Banana 2 (Modular Image-to-Image translation): Translates physical signs, maps, and menus on camera while recreating the original layout, font style, and art design as a 16:9 translated replica.
Persistent Trip Memory: Magellan slots individual trips in and out, creating memory ledgers and summaries of your route history, recommendations, and local context wherever you go.
Dynamic Spatial Utilities: Seamlessly displays navigation cards, local weather forecast feeds, landmark history summaries, and restaurant recommendations inside a fluid, dual-orientation (vertical/horizontal) UI.
README excerpt
# Magellan Lens Magellan Lens is a phone-first AI travel companion built for a hackathon. It combines Magellan's travel-chat interface with Lens's server-side Gemini Live backend. The product has two intentional modes: - **Portrait chat** is a normal Gemini Flash travel chat. It does not open the camera. - **Landscape Lens** is an explicit Gemini Live camera and voice session, with GPS-aware maps, weather, memory, translated visuals, city finds, and TimeLens historical reconstructions. ## What it does - Keeps durable trip/profile facts in Supermemory and curates new explicit facts from final Live transcript turns. - Starts with a local default trip memory: vegetarian, Rs 2,500/day, July 11–13, and Indiranagar cafe exploration. - Uses Google Places and Routes tools for location, nearby recommendations, and budget/mobility-aware directions. - Uses weather and current-time tools for practical travel advice. - Captures menus, signs, notices, placards, timetables, storefronts, and packaging; Gemini Flash reads the text and Nano Banana 2 renders a translated visual copy. - Runs safe, city-specific visual scavenger hunts. Quest templates are cached per city for 24 hours while progress stays session-specific. - Generates short, clearly labeled TimeLens historical reconstructions through Gemini Omni Flash when explicitly requested. These are illustrative, non-graphic reconstructions with an educational voiceover, never archival footage. ## Architecture ``` Android Chrome / desk