Skip to content

Hindsight

Robot fleets record thousands of hours of video, but real failures are rare and no one has time to find them.

Code Video Cerebral Valley

Core trick

Argon finds failure moments in robot footage, rebuilds each as a physics twin, trains an RL fix, and checks reward hacking before shipping.

Tags

robotics/hardwarevision

Description

Robot fleets record thousands of hours of video, but real failures are rare and no one has time to find them. Gemini 4 Argon finds them, rebuilds each one as a digital twin, writes the reward, trains a fix with RL, catches reward hacking, and approves the fix only after checking every site.
bouncypitch/hindsight★ 00 forksHTMLCreated Oct 25, 2026 · last push Oct 26, 2026

README excerpt

# Federated Robot Learning - Enabled by Argon
**Demo site:** https://bouncypitch.github.io/hindsight/ · **All failures Argon found:** https://bouncypitch.github.io/hindsight/review/ · **1-minute video:** https://youtu.be/ZgMhqIWLpNs

https://github.com/user-attachments/assets/f1342774-0385-4251-81a7-e0a83d48397b

**Hindsight.** Every robot failure makes the fleet smarter. Federated robot learning, enabled by Gemini 4 Argon.

**Robots that learn from their own failures.** Hindsight watches real robot footage (indoors and outdoors), finds the moments a robot went wrong, rebuilds each one as a physics digital twin that reproduces the failure, retrains the robot with reinforcement learning, and only ships the fix once **Gemini 4 Argon** verifies it. Robots then share what they learned as tiny policy updates instead of hours of video.


Built in one day at the Gemini 4 Argon SF Hackathon (Google DeepMind x Cerebral Valley, 2026-10-25).

## The proof
Two real failures from public FrodoBots-Mini-4K footage, one indoor and one outdoor, both fixed and approved by Argon:
- **Indoor:** a robot clipped a barstool leg and flipped onto its side.
- **Outdoor:** a robot drove into another robot parked among bikes on a crowded sidewalk.

For the barstool case, Hindsight:

1. **Confirmed** the failure from video + IMU (Argon, with visual evidence and timestamps).
2. **Rebuilt** it as a MuJoCo twin that reproduces the crash (original behavior: 3.5 cm clearance, collision).
3. **Retrained** the

Team