Skip to content

expertoncall

**Statement 1: Multi-Agent Interactions** Dynamic Expert-in-the-Loop GRPO Training on Agent World Model What Is This?

Code Demo Video Cerebral Valley

Core trick

GRPO teaches a 4B model when to call a stronger expert advisor during API tool tasks, versus solving alone.

Tags

agents

Description

**Statement 1: Multi-Agent Interactions** Dynamic Expert-in-the-Loop GRPO Training on Agent World Model What Is This? Imagine teaching a new employee. You wouldn't just hand them a manual and walk away. You also wouldn't stand behind them dictating every keystroke. The best approach? Let them try, and tell them an expert is available if they get stuck. We give a small language model (Qwen3-4B) a set of ~35 API tools, a task description, and access to a brilliant advisor (GPT-5.1). Then we use reinforcement learning (GRPO) to teach it when calling the expert leads to better outcomes — and ultimately, when it can fly solo. Detailed description is in this readme: https://github.com/sfc-gh-mhidayetoglu/OpenEnv/blob/add-agent-world-model/envs/agent_world_model_env/EXPERT_ENHANCEMENT.md
sfc-gh-mhidayetoglu/OpenEnv★ 00 forksBSD-3-ClauseCreated Mar 8, 2026 · last push Mar 10, 2026

README excerpt

# <img width="35" height="35" alt="image" src="https://github.com/user-attachments/assets/2700a971-e5d6-4036-b03f-2f89c9791609" /> OpenEnv: Agentic Execution Environments

An e2e framework for creating, deploying and using isolated execution environments for agentic RL training, built using Gymnasium style simple APIs.

[![PyPI](https://img.shields.io/pypi/v/openenv?color=blue)](https://pypi.org/project/openenv/)
[![Discord](https://img.shields.io/badge/Discord-OpenEnv-7289da?style=flat&logo=discord&logoColor=white)](https://discord.gg/YsTYBh6PD9)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/meta-pytorch/OpenEnv/blob/main/examples/OpenEnv_Tutorial.ipynb)
[![Docs](https://img.shields.io/badge/Docs-Explore-blue?logo=readthedocs&logoColor=white)](https://meta-pytorch.org/OpenEnv/)

---

**🚀 Featured Example:** Train LLMs to play BlackJack using [torchforge](https://github.com/meta-pytorch/torchforge) (PyTorch's agentic RL framework): [`examples/grpo_blackjack/`](examples/grpo_blackjack/)

**🔥 GPU Mode Tutorial:** End to end tutorial from [GPU Mode](gpu-mode-tutorial/README.md) blog post.

## Quick Start

Install the OpenEnv core package:

```bash
pip install openenv-core
```

Install an environment client (e.g., Echo):

```bash
pip install git+https://huggingface.co/spaces/openenv/echo_env
```

Then use the environment:

```python
import asyncio
from echo_env import EchoAction, EchoEnv

async def main():

Team