Bears
Our project is an environment designed for an agent to oversee 7 agents playing a popular strategy game Diplomacy.
Core trick
A monitor model trained with RL to infer hidden Diplomacy strategies from agent actions, rewarded by an LLM judge's verdict.
Description
Our project is an environment designed for an agent to oversee 7 agents playing a popular strategy game Diplomacy. Players in this game can have discussions, collude, and betray each other using complex, long-horizon strategies: reasoning that is hard to interpret from the observation space of an outside agent.
The overseer learns via RL, rewarded whenever it correctly infers a player's hidden strategy, determined by an LLM acting as a judge. Our agent powers alternative interpretability of LLMs: Instead of focusing on reading model weights and activations, we learn intent from the agent's actions.
As multi-agent systems become standard infrastructure, we need an oversight mechanism that can monitor many agents simultaneously and detect nuanced and hidden misalignment. Our overseer is exactly this, a dedicated monitoring model trained specifically to infer strategy in a complex multi-agent environment.
README excerpt
# Overseer [Winner 🏆 @ Meta's OpenEnv Hackathon] Recently, an AI agent secretly mined cryptocurrency. Not because it was told to, but because it could. ROME, an Alibaba-affiliated research agent, bypassed its sandbox, diverted GPU resources to mine crypto, and opened a hidden tunnel to an external machine, **all without being told to do so**. `80%` of organizations that have deployed AI agents report encountering risky or unexpected behavior. As LLMs grow more capable and autonomous, **understanding** what they are actually doing becomes **critical.** The Overseer trains a model to infer hidden strategies from the public behavior of LLM agents, *without access to their internal reasoning.* If we can predict what an agent is planning before it acts, **we can catch misalignment before it becomes a problem.** ## Why Diplomacy? Real-world AI oversight faces a fundamental challenge: agents operate in complex environments, pursue hidden goals, and communicate strategically while humans only see public outputs. Diplomacy is a compact version of that problem. Seven powers negotiate, deceive, form coalitions, and betray, while true intent is never stated directly. - Autonomous trading agents concealing strategies from regulators - Multi-agent pipelines where one agent manipulates another without human visibility - Agentic enterprise systems pursuing misaligned subgoals behind legitimate-looking actions If an overseer can learn to detect betrayal before it happens in Diplomacy,