Desktop Agent

README excerpt

# 🎤 Voice-Controlled Computer Assistant (test3.py)

An AI-powered voice assistant that controls your computer through natural language commands using Google's Gemini 2.5 Flash with Live API.

---

## ✨ Features

### 🖱️ **Mouse Control**
- Move mouse to specific coordinates (absolute or relative)
- Click (left/right, single/double/triple)
- Hold and release mouse buttons
- Smooth animated mouse movements
- Scroll in any direction
- AI-powered coordinate detection from screen descriptions

### ⌨️ **Keyboard Control**
- Press individual keys (including special keys)
- Type text with optional select-all-first
- Execute keyboard shortcuts (Cmd+C, Cmd+V, etc.)
- Replace text in fields

### 🎯 **AI-Powered Screen Understanding**
- **Smart Coordinate Detection**: Tell the AI "click on Discord" and it finds it!
- **Quiz Generator**: Generates fun quizzes based on what's on your screen
- **Visual Recognition**: Analyzes screen content with Gemini 2.5 Pro

### 🎨 **User Interface**
- Real-time voice interaction with audio feedback
- Translucent modal windows for quiz display
- Screen sharing with cursor overlay
- Visual grid system for precise coordinate detection

---

## 📋 Prerequisites

### System Requirements
- **OS**: macOS (tested on macOS 14+)
- **Python**: 3.12 or higher
- **Microphone & Speakers**: Required for voice interaction
- **Screen Recording Permission**: For screen capture
- **Accessibility Permission**: For mouse/keyboard control

### API Key
- Google AI API Key from [G