Team Alpha
AI agent for evaluating Physical AI systems.
Core trick
Plays a spoken prompt through a laptop speaker, records the robot's reply with a microphone, and has Gemini judge it against the spec.
Description
AI agent for evaluating Physical AI systems.
Describe the behavior you expect out of your physical AI system in plain English. BotProbe speaks to it, listens to its response, and returns a PASS or FAIL verdict with observations — no manual testing required.
Its LLM-as-a-judge but for physical AI systems!
README excerpt
# BotProbe **AI agent for evaluating Physical AI systems.** Describe the behavior you expect out of your physical AI system in plain English. BotProbe speaks to it, listens to its response, and returns a PASS or FAIL verdict with observations — no manual testing required. --- ## What it does BotProbe is a behavioral diagnostic agent designed for physical AI robots. You give it a natural language specification, and it runs an autonomous observe-evaluate loop: 1. **Speaks** a prompt to the bot via the laptop speaker 2. **Listens** to the bot's response via the laptop microphone 3. **Analyzes** the audio against your specification using Gemini 4. **Returns** a PASS/FAIL verdict with a description of what it actually observed The agent decides how many observations are needed before issuing a verdict — simple tests complete in one round; complex behavioral tests may require more. --- ## Use cases - **Responsiveness** — Does the bot respond when spoken to? - **Interruption handling** — When a user interrupts mid-sentence, does the bot stop within 1 second? - **Audio quality** — Is the bot's speech clear and audible? - **Command recognition** — Does the bot correctly execute a given voice command? - **Conversation flow** — Does the bot handle multi-turn exchanges correctly? - **Edge case robustness** — How does the bot behave under background noise or simultaneous speech? --- ## Architecture ``` Browser | | HTTP (UI assets) v Next.js :3000 (Frontend Server)