Cloe
Cloe is an AI desktop companion powered by Gemini Live API that sees your screen and talks to you in real-time.
Core trick
Gemini Vision analyzes a screenshot, then animated overlays highlight the UI elements being explained as streaming voice responds.
Description
Cloe is an AI desktop companion powered by Gemini Live API that sees your screen and talks to you in real-time. Ask a question she analyzes your screen via Gemini Vision, highlights the exact UI elements she's explaining with animated overlays, and responds via streaming voice with full barge-in support. She auto-opens a visual scratchboard with markdown/diagrams when needed, executes system commands on request. No typing, no copy-pasting screenshots just talk
README excerpt
# Chloi — Desktop Assistant A privacy-first, highly intelligent Mac menu-bar AI companion, powered by Google Gemini. Chloi is designed to act as your personal AI assistant — sitting quietly in your menu bar and assisting you on demand. It features deep screen awareness, stunning visual annotations, duplex voice conversations, and an autonomous background agent capable of executing actions on your Mac. ## Features - **Advanced Screen Smartboard**: Chloi can draw visual overlays directly on your screen to guide, debug, or teach you: - **Glowing Underlines**: Color-coded severity lines under text/code (red for errors, yellow for warnings, green for OK). - **Numbered Callouts**: Badges with leader lines pointing exactly to UI elements or text. - **Visual HUD Steps**: For math solutions or multi-step explanations, Chloi renders a clean, persistent left-aligned HUD column of step-by-step text. - **Geometric Highlights**: Glowing arrows, translucent bounding boxes, and custom shapes. - **Autonomous Background Agent**: Say `"Chloi agent, [task]"` to spawn an asynchronous background researcher. The agent can: - Browse the web for live information. - Execute bash commands on your local system. - Read local files and configurations. - Execute AppleScript to control native macOS apps (Mail, Calendar, Finder). - *Chloi will verbally announce the results to you once the background task is complete.* - **Always-Visible Trust Indicator**: A sleek floating pill indicato