Skip to content

SuperGeneral

Frontier models are surprisingly bad at using tools.

Code Demo Video Cerebral Valley

Core trick

Agents get per-step rubric correctness rewards and are trained with GRPO over multi-turn rollouts, penalizing responses that talk instead of act.

Tags

agentsdev tools

Description

Frontier models are surprisingly bad at using tools. They cheat by memorizing knowledge instead of learning to act. But tool use is the key to adapting to new domains and long horizon tasks. 3 main focuses: Tool use — can the agent call tools correctly? Tool composition — can it chain tools into multi-step workflows? Tool creation — can it create new tools for unfamiliar tasks? I evaluated Claude Sonnet, GPT-4o, Qwen, and DeepSeek across SVG illustration, law, consulting, and investment banking. The environment provides building blocks but forces the agent to discover them, gives per-step correctness feedback without hints, lets the agent decide how to decompose problems, and rewards tool composition and creation over single tool use. Rewards combine a rubric correctness score with behavior signals. Agents that talk instead of act get penalized. Training uses GRPO with multi-turn rollouts. Reward curves go up. Agents learn to explore first, compose second, act efficiently.
lilyzhng/OpenEnv★ 10 forksPythonBSD-3-ClauseCreated Jan 17, 2026 · last push Jul 13, 2026

README excerpt

# <img width="35" height="35" alt="image" src="https://github.com/user-attachments/assets/2700a971-e5d6-4036-b03f-2f89c9791609" /> OpenEnv: Agentic Execution Environments

An e2e framework for creating, deploying and using isolated execution environments for agentic RL training, built using Gymnasium style simple APIs.

[![PyPI](https://img.shields.io/pypi/v/openenv?color=blue)](https://pypi.org/project/openenv/)
[![Discord](https://img.shields.io/badge/Discord-OpenEnv-7289da?style=flat&logo=discord&logoColor=white)](https://discord.gg/YsTYBh6PD9)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/meta-pytorch/OpenEnv/blob/main/examples/OpenEnv_Tutorial.ipynb)
[![Docs](https://img.shields.io/badge/Docs-Explore-blue?logo=readthedocs&logoColor=white)](https://meta-pytorch.org/OpenEnv/)

---

**🚀 Featured Example:** Train LLMs to play BlackJack using [torchforge](https://github.com/meta-pytorch/torchforge) (PyTorch's agentic RL framework): [`examples/grpo_blackjack/`](examples/grpo_blackjack/)

**🔥 GPU Mode Tutorial:** End to end tutorial from [GPU Mode](gpu-mode-tutorial/README.md) blog post.

## Quick Start

Install the OpenEnv core package:

```bash
pip install openenv-core
```

Install an environment client (e.g., Echo):

```bash
pip install git+https://huggingface.co/spaces/openenv/echo_env
```

Then use the environment:

```python
import asyncio
from echo_env import EchoAction, EchoEnv

async def main():

Team