Skip to content

PolicyGrader

PolicyGrader is an agentic system that helps make robots safer to operate alongside humans.

Code Video Cerebral Valley

Core trick

Runs parallel vision-model judges on simulated robot rollouts and attaches measured per-label precision from a human-labeled calibration set.

Tags

robotics/hardwarevision

Description

PolicyGrader is an agentic system that helps make robots safer to operate alongside humans. It automatically stress-test and analyse robot control policies, end-to-end. It compresses a process that costs robotics teams weeks of frame-by-frame video review into minutes. You describe an evaluation goal in plain English; then a swarm of specialized Claude Opus 4.7 Managed Agents — a planner, a rollout worker, K-parallel vision judges, and a reporter — design the test suite, execute it in simulation, point at each failure with pixel accuracy (or honestly abstain), and cluster the deployment findings into actionable patterns using Opus 4.7's 1M-context window. The differentiator is trust: every run includes a calibration cohort whose ground truth comes from a human labeling a sampled subset, and the judge's measured per-label precision is attached as a confidence chip to every deployment finding. No vibes with safety. The judge is auditable from the dashboard's runtime.json and findings.jsonl. Submitted to the Anthropic Opus 4.7 Hackathon.
mgsa1/PolicyGrader★ 11 forksPythonCreated Apr 24, 2026 · last push May 3, 2026

README excerpt

# PolicyGrader

> **One prompt in. A full control policy stress test out.**
> An agentic system that designs, runs, judges, and reports on robot policy evaluations end-to-end, while measuring its own judge against human ground truth.

<p>
  <a href="https://platform.claude.com/docs/en/about-claude/models/whats-new-claude-4-7"><img alt="Claude Opus 4.7" src="https://img.shields.io/badge/Claude-Opus%204.7-0b5fff?style=flat-square&labelColor=1f1f1f"></a>
  <a href="https://platform.claude.com/docs/en/managed-agents/overview"><img alt="Managed Agents" src="https://img.shields.io/badge/Managed%20Agents-2026--04--01-1967d2?style=flat-square&labelColor=1f1f1f"></a>
  <a href="https://robosuite.ai"><img alt="robosuite 1.4.1" src="https://img.shields.io/badge/robosuite-1.4.1-b06000?style=flat-square&labelColor=1f1f1f"></a>
  <a href="https://www.python.org/downloads/release/python-3120/"><img alt="Python 3.12" src="https://img.shields.io/badge/Python-3.12-3776ab?style=flat-square&labelColor=1f1f1f"></a>
</p>

<!-- HERO ANIMATION — two panoramic loops (cards + data-flow), click-through to the full explainer.
     Rendered by `robotics_pitch/` (Remotion). Re-render with:
       cd robotics_pitch && npm run render:pipeline-cards-gif   # → docs/media/pipeline-cards.gif
       cd robotics_pitch && npm run render:pipeline-flow-gif    # → docs/media/pipeline-flow.gif
       cd robotics_pitch && npm run render                      # → docs/media/hero.mp4 -->

<p align="center">
  <a href="doc

Team