Coffee Station
Coffee Station is a desktop agentic robotics app that lets a Gemini Flash-powered agent control a USB Hugging Face LeRobot arm.
Core trick
Gemini tool calls map to joint-pose or inverse-kinematics targets on a USB robot arm, with time-offset bundles scheduled for execution.
Description
Coffee Station is a desktop agentic robotics app that lets a Gemini Flash-powered agent control a USB Hugging Face LeRobot arm. It provides direct joint-pose tool calls, IK-based world-space targeting, scheduled/bundled tool execution, webcam feedback, multi-camera configuration, persistent local agent sessions, and a desktop UI with live camera view plus autonomous chat/ session controls. The project solves the problem of safely connecting vision, tool-calling agents, and physical robot-arm control in one local operator-friendly application
README excerpt
# Coffee Station ## Repository status This repository state is frozen. Future active development has moved to [JacobFV/phys-0](https://github.com/JacobFV/phys-0); use that repository for new work, fixes, and current project direction. Coffee Station is a native desktop app for driving a Hugging Face LeRobot follower arm with a Gemini agent. It runs as a single window on macOS, Linux, and Windows — no browser, no server to manage. ## What it does - Gemini tool calls for direct joint-pose control. - IK-based world-space target and relative-offset commands. - Scheduled bundles of tool calls with per-call time offsets. - Queue inspection and cancellation for scheduled calls. - Multi-camera OpenCV capture with configurable automatic frame injection into the agent loop. - Camera scanning, enable/disable, auto-feed frequency, and latest-frame requests. - Live camera display in the desktop window, capped by `CAMERA_FPS`. - Local session storage in SQLite. - Markerless monocular self-calibration for fitting arm geometry and camera extrinsics from synchronized joint/camera samples. - A glass desktop UI with the live camera feed, conversation, queue chips, emergency stop, and a settings drawer that exposes camera, skills, queue, and developer tools. The current default model is `gemini-flash-latest`. Override it with `GEMINI_MODEL` to pin a stable model. ## Install ```bash python -m venv .venv source .venv/bin/activate pip install -e ".[dev]" ``` ## Run Set your Gemini API key