Skip to content

Black Box

Black Box is a forensic copilot for robots and embodied systems.

Code Video Cerebral Valley

Core trick

A grounding gate cross-checks ranked root-cause hypotheses against telemetry and video, dropping any the evidence does not support.

Tags

robotics/hardwaredev tools

Description

Black Box is a forensic copilot for robots and embodied systems. You hand it a field recording — whatever the platform produces: ROS1/ROS2 bags, raw telemetry logs, multi-camera video, LiDAR/IMU traces, GPS/RTK data, controller logs, or repo context — and it returns a grounded post-mortem: ranked root-cause hypotheses, timestamped multimodal evidence, an NTSB-style PDF, and a scoped code patch. The core insight is that robotics failures are rarely obvious from one log or one operator's account. In our hero case, the operator filed "tunnel caused the anomaly." Black Box's grounding gate cross-checked telemetry, video, and source code and refuted the operator: RTK heading was already degraded 43 minutes before tunnel entry, drive-by-wire never engaged, so the tunnel could not plausibly be the cause. The refutation ships as a ranked hypothesis with its own confidence and patch hint — not as agreement. We built around long-horizon agentic investigation rather than chat. The system includes a FastAPI/HTMX UI with streaming reasoning, a deterministic grounding gate that drops weakly-supported hypotheses, an append-only 4-layer memory stack (case → platform → taxonomy → eval), a frozen 7-class bug taxonomy enforced at parse time by Pydantic, an HITL approve/reject gate before any patch is applied, and Claude Managed Agents for persistent forensic sessions with tool use, checkpoints, time-travel rollback, and operator steering.
LucasErcolano/BlackBox★ 10 forksPythonMITCreated Apr 22, 2026 · last push Apr 26, 2026

README excerpt

# Black Box — forensic copilot for robots

[![ci](https://github.com/LucasErcolano/BlackBox/actions/workflows/ci.yml/badge.svg)](https://github.com/LucasErcolano/BlackBox/actions/workflows/ci.yml)
**Built with Claude Opus 4.7** · `claude-opus-4-7` · vision + reasoning + Managed Agents · no downgrades.

Feed it a robot recording — ROS1/ROS2 bag, telemetry, multi-camera video, LiDAR/IMU, plus repo context — and get back a grounded post-mortem: ranked root-cause hypotheses, timestamped multimodal evidence, an NTSB-style PDF, and a scoped code patch.

> When a robot crashes, the flight data recorder tells you *what* happened. Black Box tells you *why*, and hands you the diff.

---

## Table of contents

1. [Demo](#demo)
2. [Hero case — operator said "tunnel," telemetry said otherwise](#hero-case--operator-said-tunnel-telemetry-said-otherwise)
3. [Why this is hard](#why-this-is-hard)
4. [What we shipped](#what-we-shipped)
5. [How it works](#how-it-works)
6. [Grounding gate](#grounding-gate)
7. [Bug taxonomy — frozen 7](#bug-taxonomy--frozen-7)
8. [Memory stack](#memory-stack)
9. [Verification ledger — honest forensics](#verification-ledger--honest-forensics)
10. [Token discipline](#token-discipline)
11. [Benchmark](#benchmark)
12. [UI](#ui)
13. [Modes and trust tags](#modes-and-trust-tags)
14. [Capability matrix](#capability-matrix)
15. [Package layout](#package-layout)
16. [Demo asset catalog](#demo-asset-catalog)
17. [Bonus — NAO6 humanoid adapter](#bonus--nao6-humanoid-adapter)

Team