Skip to content

ArgonTwin

I built ArgonTwin to test robot plans in a simulation of your real workspace before running them on physical hardware.

Core trick

Reconstructs a desk from phone video with Gaussian splatting, runs ALOHA arms in MuJoCo, and sends failure frames and forces to Argon for patches.

Tags

Description

I built ArgonTwin to test robot plans in a simulation of your real workspace before running them on physical hardware. The first user is a robotics developer setting up a task in a new workspace. Every table, obstacle and handover position changes what will work. As more capable models start acting through robot arms and humanoids, we need a way to test those actions in the spaces where they will happen. I started with a ten-second phone video of my desk at SHACK15. COLMAP and Gaussian splatting reconstruct its appearance. Two simulated ALOHA arms then try to pass my Celsius can in MuJoCo. The first plan hits a screen and drops it. Gemini 4 Argon reviews twenty rendered frames and the failure measurements, explains the cause, and proposes a plan patch. The application applies the patch and reruns the task. MuJoCo's fixed checks decide whether it passes. In the recorded demo, obstacle contact falls from 216.2 N to zero and the can lands 5.1 mm from target. I also fine-tuned Gemma for a separate local repair comparison. On twelve held-out cases, the base model passed four and the tuned model passed eleven. The stack is Python, FastAPI, MuJoCo, mink, Pydantic, plain JavaScript and Three.js. Public replay: https://argon-twin.vercel.app This is a simulation prototype with fitted collision boxes and assisted grasps. Fully automatic video-to-physics generation and transfer to real hardware remain unverified. The public site replays recorded results; new model calls and simulation run locally.
benikigai/argon-twin★ 00 forksPythonCreated Oct 25, 2026 · last push Oct 25, 2026

README excerpt

# ArgonTwin

**Your desk, rehearsed by robots before hardware.**

Film a table with your phone. ArgonTwin turns it into a physics twin, two ALOHA robot arms try a hand-over on it in MuJoCo, and when an attempt fails, Gemini 4 Argon reads the rendered frames and the measured numbers, explains why, and patches the plan. MuJoCo runs it again and decides.

Built at the [Gemini 4 Argon SF Hackathon](https://cerebralvalley.ai/events/~/e/gemini-4-argon-sf), Google DeepMind × Cerebral Valley, Oct 25, 2026. Problem Statement Two, *More Dimensions*: render what it made, watch the result, critique it against the goal, fix it in a loop. Everything in this repo was written during the event.

**[Try the verified Celsius replay](https://argon-twin.vercel.app)**: watch the screen collision, read Argon's repair, and replay the passing hand-over. The public demo is a recorded run; new model calls and physics runs need the local server (see [Run it](#run-it)).

**[Watch the 60-second tour](https://argon-twin.vercel.app/watch)** · 1920×1080, no audio. Phone capture, Gaussian splat, collision, Argon repair, and the passing hand-over.

![Screenshot of the local viewer in Photoreal mode: the Gaussian splat of Ben's desk with two simulated ALOHA arms, the Celsius can splat riding on the simulated can, a MuJoCo Pass verdict (can placed 0.5 cm from target, 0.0 N obstacle contact), two wrist camera views, and the attempt strip: attempt 1 failed, Argon explains and fixes, attempt 2 passed.](docs/proof-c

Team