Skip to content
3rd Place

Custom Universe

A realtime synthetic-data engine for world models.

Code Demo Video Cerebral Valley

Core trick

Captures the WebGL viewport on each edit and sends it as a reference image to a 4-step distilled image model for near-realtime restyling.

Tags

creative/musicvision

Description

A realtime synthetic-data engine for world models. Make a photorealistic scene from crude 3D objects online or brought by you. 1. Take a pictures of an object and upload it. AI converts it into a 3D object. Drag and drop it into the scene builder. Give the scene builder a stylistic text prompt instruction. Watch the scene get created! 2. The scene is fully controllable in realtime with full support for scaling, rotating, translating, and editing the text-based style. Photo on phone → 3D object → AI-stylized editable scene.
jss8649/image-edit-realtime-hackathon★ 318 forksHTMLCreated Jun 13, 2026 · last push Jun 15, 2026

README excerpt

# Realtime 3D Canvas → AI Image Editing

A browser-based 3D canvas where you import, position, rotate, and scale 3D objects,
then watch an AI re-render the viewport **in ~realtime** as you edit. By default,
generation runs on a **self-hosted FLUX.2 Klein 9B** model (distilled, 4-step) loaded
in-process on a local H100 — no remote API, no submit-then-poll latency floor. It can
also be **hosted remotely** via FAL or Fireworks (see *Model hosting* below).

Every time you move the scene, the frontend captures the WebGL viewport and sends it to
the backend as a *reference image*; the model edits it according to your prompt and
the result appears side-by-side.

## Requirements

- **GPU:** 1× NVIDIA H100 80GB (or any ~40GB+ CUDA GPU) for the default local model.
  Runs bf16, no quantization — the ~9B flow model plus the Qwen3 text encoder are ~34GB
  resident in VRAM. *Not needed if you use a remote provider (FAL/Fireworks).*
- **Python:** 3.10+ (with a CUDA build of PyTorch for the local model).
- For the local model: a Hugging Face account that has accepted the FLUX.2 license
  (weights are gated). For remote: a FAL or Fireworks API key.

## Run with Docker (recommended)

The whole app — FastAPI + in-process FLUX.2 Klein + the UI + Blender USDZ conversion —
runs as a single GPU container. (The TRELLIS image→3D sidecar is intentionally *not*
containerized; in the container the "Create 3D from image" button returns a placeholder
asset. To run TRELLIS too, see the manual setup + `tre

Team