SmashBot
We created a simulator to simulate Smash Bros Melee and using a reward system we create the play style of our favorite smash players from scratch.
Core trick
A PPO agent controls Fox through libmelee on Slippi Dolphin, with reward shaping that reproduces specific human players' play styles.
Tags
Description
We created a simulator to simulate Smash Bros Melee and using a reward system we create the play style of our favorite smash players from scratch. Then forwarding that model data to an emulator in order to use OpenENV for further training, creating bots that can competently defeat high-range CPUs.
README excerpt
# Melee RL — Fox vs Jigglypuff via Dolphin + OpenEnv A reinforcement learning system that trains a **Fox** AI agent to play **Super Smash Bros. Melee** in real-time. The agent connects to [Slippi Dolphin](https://slippi.gg/) via [libmelee](https://github.com/altf4/libmelee) and learns through self-play using PPO, served over HTTP with [Meta's OpenEnv](https://github.com/meta-pytorch/OpenEnv) framework. ## How It Works ``` Training Script (PPO) OpenEnv Server Slippi Dolphin ┌──────────────────┐ HTTP/WS ┌──────────────┐ libmelee ┌──────────────┐ │ dolphin_train.py │ ──SmashAction──> │ FastAPI app │ ──controller───> │ Melee game │ │ │ <─SmashObs───── │ (port 8000) │ <─gamestate──── │ (60 FPS) │ │ CompetitiveMelee│ │ EmulatorEnv │ │ Fox vs Puff │ │ Reward shaping │ │ Server │ │ on FD │ └──────────────────┘ └──────────────┘ └──────────────┘ ``` 1. **OpenEnv server** launches Dolphin, navigates menus, and exposes `reset()` / `step()` over HTTP 2. **Training client** sends controller inputs (stick, buttons) each frame and receives game state back 3. **Reward shaping** (`CompetitiveMeleeReward`) computes per-frame reward on the client side 4. **PPO** updates the policy every 2048 frames ## Training Pipeline ### Phase 1 — Physics Simulator (fast, offline) Train base policies in a custom Melee