Skip to content
2nd Place

Sim Sketch Artist

Sim Sketch Artist turns your selfie into a portrait drawn by a robot in simulation.

Code Demo Cerebral Valley

Core trick

Converts a selfie into ordered pen-stroke paths that are sent to a simulated SO-101 arm in Isaac Sim.

Tags

robotics/hardwarevision

Description

Sim Sketch Artist turns your selfie into a portrait drawn by a robot in simulation. Capture a photo through our Lovable app, and GPT-6 Astra transforms it into structured pen strokes. An SO-101 robot arm in NVIDIA Isaac Sim executes the drawing, while an interactive browser viewer lets you watch, orbit and zoom around the scene. It connects AI vision with robot motion, making creative robotics accessible from a laptop without physical hardware.
Kazybekkh/sim-sketch-artist★ 11 forksPythonCreated Oct 6, 2026 · last push Oct 8, 2026

README excerpt

# Sim Sketch Artist


Take a webcam portrait on your Mac. Astra turns it into simple pen strokes, and
an SO-101 in Isaac Sim draws them on a virtual sheet of paper.

<img width="1112" height="720" alt="Built-in Retina Display" src="https://github.com/user-attachments/assets/088303a5-4141-4b9d-af3f-f0af30d59212" />

Demo video: https://screen.studio/share/MAH3vp7T


**Connect your own setup:** this public repository supplies the code, not a shared
GPU or model-credit service. Each operator supplies their own Isaac Sim machine, OpenAI API credentials and
model access. Follow the [self-hosting handoff](docs/self-hosting.md).

Run the reference studio at `http://localhost:8000` on your simulator host, or
connect a browser frontend to your own protected backend address using Connection
settings. The interactive viewer is available at your backend's `/sim/view` path.
Codex helped build the project; it is not required while the app runs.

```text
Mac webcam / upload in Lovable
              |
         HTTPS tunnel
              |
Ubuntu FastAPI ---> OpenAI Responses API (ASTRA_MODEL)
              |
       atomic jobs/<uuid>/ files
              |
Isaac Sim 5.1 Python ---> SO-101 joint motion + measured pen trail
              |
       live viewport + result.png ---> Mac browser
```

The browser's **Sketch me** action creates the portrait and queues the drawing.
Astra receives the photo and returns only bounded line coordinates. No model
output is executed as code. The backend uses

Team