Overview PRISM

Overview PRISM is an always-on, ambient AI assistant designed to give users particularly the visually impaired a persistent spatial memory of their surroundings. By continuously monitoring the user's environment through a camera, PRISM acts as a "second brain" that remembers where everyday objects were left, tracks dynamic environmental changes, and guides the user back to specific items using natural, conversational voice commands and real-time spatial navigation. Key Features Continuous Spatial Memory & Tracking: PRISM continuously scans the environment, identifying objects and logging their exact geographic coordinates into a local SQLite database. It builds a persistent "World Memory" that remembers what it saw, where it saw it, and when. Dynamic Distance & Bearing Navigation: When a user asks for a previously seen object (e.g., "Where did I leave my backpack?"), the system calculates the real-time Haversine distance and compass bearing between the user's current location and the object's tagged location. It then provides dynamic auditory guidance (e.g., "Your backpack is 140 meters behind you to the South"). Zero-Latency Multimodal Reasoning: To ensure lightning-fast interactions, PRISM uses a "Zero-STT" (Speech-to-Text) architecture. Raw audio bytes from the user's microphone are streamed directly into a Gemini Multimodal model alongside the historical database context and the current camera feed. This eliminates the latency of traditional transcription services, allowing the AI to reason over audio, visuals, and memory simultaneously. Ambient "Live Memory Catch" UI: PRISM features a sleek, dark-mode CustomTkinter interface. It silently tracks environmental context in the background (Live Memory Catch) without overwhelming the user, only speaking up when directly queried or when critical environmental hazards are detected. Privacy-First Local Fallback: Designed with privacy in mind, PRISM incorporates local offline fallback mechanisms to ensure basic environmental awareness is maintained even when internet connectivity drops. Technical Stack AI & Vision: Google Gemini Multimodal API (for simultaneous audio/visual reasoning) and OpenCV (for efficient background frame buffering). Location & Math: IP-based/Mock Geolocation tracking, Haversine formula for spherical distance calculation, and Bearing mathematics for compass direction. Backend: Python, SQLite (World Memory storage), and threading for non-blocking UI and camera loops. Frontend: CustomTkinter (Python) for a sleek, highly responsive desktop hub. Use Case While useful for anyone prone to misplacing items, PRISM is purpose-built as an accessibility tool for the visually impaired. By combining computer vision, spatial mapping, and conversational AI, it allows users to navigate the physical world with greater independence, turning a standard camera into an intelligent, memory-augmented guide.