Skip to content

SVBench AI

SVBench AI is a Claude verdict layer for structural-variant callers.

Code Video Cerebral Valley

Core trick

Sends each structural-variant call's alignment image and genomic context to Claude for a cited verdict, corroborated against gnomAD-SV, dbVar, and HGSVC catalogs.

Tags

sciencedev tools

Description

SVBench AI is a Claude verdict layer for structural-variant callers. One reviewer — Claude Opus 4.8 — reads each call's alignment image and genomic context and returns an evidence-cited verdict, with or without a truth set, corroborated against independent population catalogs (gnomAD-SV, dbVar, HGSVC). Against HG002 it showed 93% of Sniffles' and 71% of SVIM's "false positives" are benchmark artifacts, not caller errors (raw 0.886→0.992; 0.694→0.886) — and SVIM's corrected precision equals Sniffles' raw, exposing a gap the benchmark hid. On genomes with no truth, the same reviewer still delivers grounded verdicts. Trustworthy SV evaluation, everywhere.
Ramanandan/svbench-ai★ 00 forksHTMLApache-2.0Created Jul 13, 2026 · last push Jul 14, 2026

README excerpt

# SVBench AI

**Every SV call gets a verdict.** The **Claude verdict layer** helping Structural Variant scientists separate real variants from artifacts — with or without a truth set.
Turns a benchmark's wall of disagreements into an auditable, human-in-the-loop review queue — with a reason for every call.

## The user

**Meet Maya, an SV-caller developer at a genome center.** Every release, she benchmarks her caller against the [GIAB](https://www.nist.gov/programs-projects/genome-bottle) HG002 truth set and gets back *hundreds* of FP/FN loci. Today she opens IGV and eyeballs them one by one, asking the same question each time: **"is my caller wrong, or is the benchmark wrong?"** That triage takes hours and lives only in her head.

SVBench AI is the tool she's missing. She points it at any GIAB-benchmarked VCF and gets a confidence-sorted review queue: each discordant locus shows the read-alignment image beside **Claude Opus 4.8**'s verdict — TP / FP / FN / **benchmark_artifact** — with the evidence it used and a plain-language reason. She accepts or overrides each call from the keyboard, and exports a curated decision set (JSON/CSV) back into her pipeline. She runs it next release without us in the room.

> Built for the *Built with Claude: Life Sciences* hackathon (Builder Track). Uses only public GIAB/UCSC data; makes no clinical or diagnostic claims; does not implement a new SV caller or annotation algorithm. Apache-2.0.

## Why this is not "just another benchmark"

Metri

Team