Skip to content

Openrescue

In real disasters, you can’t rely on maps or connectivity.

Code Demo Video Cerebral Valley

Core trick

A vision-language model reasons in think tags about rendered scenes and picks the next grid move, trained with a custom GRPO loop.

Tags

visionrobotics/hardware

Description

In real disasters, you can’t rely on maps or connectivity. A drone needs to see, understand, and decide on its own where to go to reach people who need help. OpenRescue, built for OpenEnv, trains vision-language models to navigate just by looking at images. We built a custom training loop that lets the model see a scene, think step by step, and choose the next move.
ltejedor/openrescue★ 00 forksPythonCreated Mar 8, 2026 · last push Mar 8, 2026

README excerpt

# Visual Reasoning RL with OpenEnv

Teaching vision-language models to navigate by looking at images — trained with GRPO (Group Relative Policy Optimization), following the [Groundlight r1_vlm](https://www.groundlight.ai/blog/visual-reasoning-models) approach adapted for spatial navigation tasks.

Built during the OpenEnv Hackathon (March 2026) using the [OpenEnv](https://github.com/meta-pytorch/OpenEnv) framework.

![GRPO Grid Training](assets/grpo-grid.png)

## What We Built

### 1. Grid World with Visual Reasoning (`train_gridworld_grpo.py`)

A custom multi-step GRPO training loop that teaches **Qwen2.5-VL-3B-Instruct** to navigate a grid by looking at rendered images. The model must reason in `<think>` tags before choosing a direction.

**Key design decisions:**
- **Custom GRPO loop** — TRL's `GRPOTrainer` can't handle multi-step VLM episodes (each step has a different image). We wrote the GRPO algorithm from scratch: run episodes, compute group-relative advantages, backprop step-by-step to stay in memory.
- **Layered rewards** following Groundlight's pattern: format reward (must use `<think>/<answer>` tags) + game reward (-0.1/step, +1.0 at goal, -0.2 bad format)
- **No coordinates in the prompt** — the model must use the image to determine where to go
- **Runs on M1 Mac** (16GB) — LoRA + step-by-step gradient accumulation to fit 3.8B parameters

```
<think>
The goal (red cell) is below and to the right of me (blue cell).
I should move DOWN to get closer.
</think>
<answe

Team