Watch & Learn
Watch & Learn transforms screen recordings into executable "skills" for AI agents.
Core trick
Extracts step-by-step instructions and automation scripts from narrated screen recordings and packages them as agent skills.
Description
Watch & Learn transforms screen recordings into executable "skills" for AI agents. Instead of writing tedious documentation, simply record yourself performing a task with narration. W&L automatically extracts step-by-step instructions, automation scripts, templates, and reference assets—converting tacit knowledge into structured formats AI agents can follow. Deploy skills as downloadable ZIP files, MCP servers, or run via Gemini computer-use models in-app.
The core insight: "showing" captures nuance that written instructions miss, making on-the-job agentic training faster and more reliable.
README excerpt
# Watch & Learn **Teach AI agents new skills by showing, not writing.** *Started at the Gemini x TED AI Hackathon (Oct 2025)* Watch & Learn transforms screen recordings into "skills" for AI agents. Instead of writing tedious documentation, simply record yourself performing a task with narration. The system automatically extracts step-by-step instructions, automation scripts, templates, and reference assets—converting tacit knowledge into structured formats AI agents can follow. Deploy skills as downloadable ZIP files, MCP servers, or run computer-use automations. The core insight: demonstration captures nuance that written instructions miss, making on-the-job agentic training faster and more reliable. ## The Problem Writing detailed prompts and instructions for AI agents is time-consuming and tedious. Complex workflows require lengthy documentation that's hard to maintain and often misses crucial details that are obvious when you actually perform the task. ## Our Solution Watch & Learn converts screen recordings into executable skill packages. Simply record yourself performing a task with light narration, and our system automatically generates: - **SKILL.md** - Step-by-step instructions AI agents can follow - **Scripts** - Automation code extracted from your demonstration - **Templates** - Configuration files and boilerplate - **Assets** - Reference screenshots and outputs These skill packages can be downloaded as zip files, converted into MCP servers for AI integra