Helios
Helios is a multi-agent system that catches silent bugs in scientific Python: the kind that don't crash but produce wrong-looking-right results (off-by-one errors in numerical integration, unit mismatches, broken boundar
Core trick
Verifier runs original and patched functions on synthesized inputs in isolated sandboxes and rejects any fix that disagrees.
Description
Helios is a multi-agent system that catches silent bugs in scientific Python: the kind that don't crash but produce wrong-looking-right results (off-by-one errors in numerical integration, unit mismatches, broken boundary conditions, subtractive cancellation).
Built on Gemini 3.5 Flash, Helios deploys four specialized sub-agents in a long-horizon workflow with a researchers-in-the-loop flywheel:
- Auditor: combines static analysis (ast/libcst) with Gemini-driven semantic review to catch numerical instability, unit mismatches, and shape assumption errors
- Fixer: regenerates corrected code per accepted issue
- Verifier: synthesizes test inputs (random within type-inferred bounds, edge cases, large-N stress), runs original vs. fix in isolated sandboxes, and rejects fixes that don't pass, triggering regeneration
- Router: flags hot loops (Monte Carlo, FFTs, matmuls, nested numeric loops) for GPU or quantum dispatch with estimated speedup
Researcher review of accepted/rejected fixes feeds back into the system, creating a continuous improvement loop.
What's never been built before: other AI coding tools generate fixes; Helios is the first multi-agent system that verifies its own fixes against synthesized tests before showing them to you, and the first to combine correctness verification with hardware-aware routing in a single agent pipeline. The target users are scientists, not software engineers, the audience whose bugs end up in published research.
README excerpt
# Helios
> A multi-agent system that audits, rewrites, and verifies its own fixes
> against synthesized tests before showing them to you.
Scientific Python is full of **silent bugs** — numerical cancellation,
off-by-one in loops, unit mismatches, broken boundary conditions. They
don't crash. They produce results that look right and are wrong. The
Reinhart-Rogoff Excel error influenced global austerity policy. Helios
catches bugs like that one.
```
[10:23:01] auditor · scanning mc_2deg_thermo_init.py (196 LOC)…
[10:23:03] auditor ! LLM pass: 3 silent bugs (numerical cancellation,
hardcoded factor, no tests)
[10:23:06] fixer · fix #1 · extracting MetropolisSampler class…
[10:23:09] verifier · synthesizing 12 test inputs from issue specs…
[10:23:12] verifier ✗ 8/12 passed → 4 boundary failures
[10:23:13] verifier ! rejecting fix #1 · regenerating with hint
[10:23:14] fixer · fix #2 · applying Kahan compensated summation
[10:23:17] verifier ✓ 12/12 passed (rtol=1e-9) · analytical match
[10:23:19] router ✓ GPU candidate: inner Metropolis loop (60-180×)
```
## The agents
| Agent | Role |
|---|---|
| **Auditor** | Static pass (`ast` / `libcst`) + Gemini LLM pass for silent scientific bugs. Output: ranked issue list. |
| **Fixer** | Regenerates the file applying one bug fix at a time. Returns full new source + diff summary. |
| **Verifier** | Synthesizes 12 test inputs from each issue spec, runs original vs. fix in a sandbox (or via the managed-ag