Skip to content

SimForge

Building simulation environments for robotics is painfully manual.

Code Video Cerebral Valley

Core trick

A vision agent estimates object dimensions from a photo, then a validation loop checks the generated MJCF physics for stability.

Tags

robotics/hardwarevision

Description

Building simulation environments for robotics is painfully manual. SimForge converts a single photo into a physics-ready robotics simulation using 6 AI agents orchestrated by Claude. Each agent handles a stage of the pipeline, from object detection and dimension estimation to MuJoCo XML generation, physics validation, and visual verification through an iterative feedback loop. The result is a sim2real-ready training environment built from a snapshot in a few minutes.
jacquekang/claude-code-hackathon★ 21 forksPythonMITCreated Feb 11, 2026 · last push Feb 16, 2026

README excerpt

# SimForge

Photo-to-MuJoCo simulation pipeline orchestrated by a Claude multi-agent system.

> Upload a photo, 5 AI agents analyze it and generate a physics simulation, and the result is displayed in a 3-panel UI with an interactive MJCF viewer.

## How It Works

1. **Upload** a photo of a tabletop or workspace scene
2. **5 AI agents** process the image in sequence — analyzing objects, selecting assets, generating physics XML, validating stability, and defining manipulation tasks
3. **Results** are displayed in a three-panel UI: uploaded image, agent pipeline status, and simulation output with scene analysis, MJCF XML, and validation screenshots

## Prerequisites

- **Python 3.12+**
- **Node.js 18+**
- **pnpm** (`npm install -g pnpm`)
- **MuJoCo** (>= 3.5.0, installed via pip)
- **MuJoCo Menagerie** robot models (see [Asset Preparation](#asset-preparation))

## Setup

### Backend

```bash
# Create and activate virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Clone robot models (see Asset Preparation below)
git clone --depth 1 https://github.com/google-deepmind/mujoco_menagerie.git vendor/menagerie

# Verify MuJoCo installation
python3 scripts/verify_mujoco.py
python3 scripts/verify_menagerie.py

# Copy and configure environment variables
cp .env.example .env
# Edit .env and add your ANTHROPIC_API_KEY

# Start the dev server
uvicorn backend.main:app --reload --port 8000
```

### Frontend

```bash
cd

Team