Bernstein-Lens

Bernstein-Lens is a gesture + voice control layer for AI-native developers. Named after Leonard Bernstein — who conducted a full orchestra without ever touching an instrument — it lets you control Cursor IDE and your entire OS using only your hands and voice. Phase 1 is a MediaPipe hand-tracking pipeline running at 60fps with <12ms latency. Gestures map directly to OS actions: point to move the cursor, pinch to click, peace sign to scroll, palm for right-click, and a 🤙 invoke gesture to summon the semantic layer. Phase 2 is the semantic control layer. Invoke triggers a liquid-glass circular HUD at the cursor position. Dwell on the mic zone to issue a voice command — transcribed locally via whisper.cpp + Apple Neural Engine CoreML in ~150ms. The screenshot and voice command are analyzed by a local vision model (Moondream via Ollama) to determine intent, returning a structured action: keystroke, click, type, or managed_agent. Complex developer tasks ("run the tests and fix failures") route to Gemini Managed Agents in a persistent cloud sandbox.