Skip to content

AlphaWolf

AlphaWolf is a self-play pipeline that teaches LLMs social deception through the game of Werewolf.

Code Demo Video Cerebral Valley

Core trick

A wolf agent replays from a critical decision point after losing, and winning versus losing transcripts become DPO preference pairs.

Tags

agents

Description

AlphaWolf is a self-play pipeline that teaches LLMs social deception through the game of Werewolf. A wolf agent plays, reflects on losses, rewinds to the critical decision point, and replays with an improved strategy. This approach resulted in Qwen3-8B-Thinking going from a 0.3 win rate to a 0.7 win rate over the span of only 45 games and 9 total gradient steps. The winning and losing transcripts become Direct Preference Optimization training pairs, while real-time consensus polling (every player votes on "who's suspicious?" after each speech) provides a novel auxiliary signal trained via ZIP-RC (ICLR 2026) to bootstrap a value function to the model. The result: a recursive self-improvement loop where the wolf gets better at deception, forcing villagers to adapt, driving emergent theory-of-mind.
lexhacke/AlphaWolf★ 00 forksPythonCreated Mar 8, 2026 · last push Mar 8, 2026

README excerpt

# AlphaWolf

Werewolf (Mafia) game environment for training deceptive LLM agents via self-play, reflection, and DPO.

**5 players: 1 Werewolf, 1 Seer, 3 Villagers.**

## How It Works

AlphaWolf has three layers:

1. **Game Engine** (`werewolf_engine.py`) -- Pure game logic. No LLM, no network. Manages roles, phases, turns, and win conditions.
2. **OpenEnv Wrapper** (`werewolf_environment.py`, `app.py`, `client.py`) -- Wraps the engine as a REST/WebSocket server using the OpenEnv framework. Pydantic models define the Action/Observation types.
3. **LLM Runner** (`runner.py`) -- The orchestrator. Connects 5 LLM-powered players to the engine. Handles prompting, JSON parsing, parallel API calls, reflection, rewind, and DPO pair collection.

## Architecture

```
alphawolf_env/
    server/
        werewolf_engine.py       # Pure game logic (WerewolfGame, GameConfig)
        werewolf_environment.py  # OpenEnv wrapper
        app.py                   # FastAPI server (uvicorn)
    models.py                    # Pydantic Action/Observation types
    runner.py                    # LLM orchestrator, prompts, DPO, reflection
    client.py                    # OpenEnv WebSocket client
    tests/
        test_engine.py           # 39 tests covering all mechanics
```

## Game Rules

- **Night**: The Werewolf picks someone to kill. The Seer investigates one player (learns their role).
- **Day**: Players debate via a bidding system, then vote to exile someone. Majority required to exile.
- **V

Team