Anuj Dev Singh
I tested a specific, high-stakes question for disease genetics: can today's best sequence-to-function deep-learning models (AlphaGenome, Borzoi, and the DeepSEA/Basset lineage) predict the DIRECTION — up or down — of a n
Core trick
Scores each noncoding variant's predicted effect direction against measured effect signs, using chromosome-held-out splits.
Description
I tested a specific, high-stakes question for disease genetics: can today's best sequence-to-function deep-learning models (AlphaGenome, Borzoi, and the DeepSEA/Basset lineage) predict the DIRECTION — up or down — of a noncoding variant's regulatory effect in a defined cell type? This is the quantity a variant-interpretation pipeline actually needs, yet it is rarely benchmarked directly.
I assembled a chromosome-split benchmark of noncoding variants with laboratory-measured effect direction across nine cell contexts (6,377 variant×context measurements), plus a native human-microglia chromatin-accessibility QTL map from 95 donors as an on-target test. On the fairest possible test — accessibility direction in the native cell type — AlphaGenome (0.537), Borzoi (0.551), motif-PWM floor baselines, and models trained in-distribution on the caQTL data itself were ALL statistically indistinguishable from chance (majority-class 0.565).
I then found the mechanism. In the same 95 donors, a variant's effect on chromatin accessibility and its effect on gene expression are statistically INDEPENDENT (sign concordance 0.506, 95% CI 0.455–0.556, p=0.87). A model that reads one functional layer cannot recover the direction of a layer it does not read. The Alzheimer's risk variant rs6733839 (upstream of BIN1) anchors it: the risk allele opens chromatin AND raises BIN1 expression, yet represses episomal enhancer activity — three "directions" at once. A Boltz-2 co-fold shows a tighter MEF2A–DNA interface at the risk allele (+12.5% contacts).
Why it matters: this is a well-powered NEGATIVE result with a mechanism. It tells the field that directional predictions from single-layer sequence models should not be trusted for variant interpretation — and explains precisely why. I release the harmonized benchmark, the paired-layer independence statistic, and all per-variant scores as a reusable community resource.
anujdevsingh/regulatory-direction-independence★ 10 forksPythonMITCreated Jul 7, 2026 · last push Jul 13, 2026
README excerpt
# regulatory-direction-independence **Can sequence-to-function models predict the _direction_ of a noncoding variant's regulatory effect? A benchmark, a well-powered negative result, and the mechanism behind it — in human microglia.** Built with **Claude Science** for the *Built with Claude: Life Sciences* hackathon (Research track, Jul 2026). [](https://doi.org/10.5281/zenodo.21332017) 📄 **Preprint (Zenodo, CC-BY 4.0):** https://doi.org/10.5281/zenodo.21332017 · local copy [`MANUSCRIPT_bioRxiv.pdf`](MANUSCRIPT_bioRxiv.pdf) · [`MANUSCRIPT_bioRxiv.md`](MANUSCRIPT_bioRxiv.md) 📈 **Provenance:** [`ROADMAP.md`](ROADMAP.md) — every stage with config, seed, and split manifest --- ## TL;DR Deep-learning models that read DNA (Borzoi, AlphaGenome, and the DeepSEA/Basset lineage) predict genomic assay *tracks* very well. But the question a disease-variant pipeline actually needs — **does this variant push accessibility / expression UP or DOWN in the relevant cell type?** — they answer at chance. We show this on a fair, chromosome-split benchmark in the microglia/brain/immune context, and then we show **why**: in the same donors, a variant's effect on chromatin accessibility and its effect on gene expression are **statistically independent**. A model that reads one layer cannot recover the direction of a layer it does not read. > **The whole study in one figure:** ![Study summary — a well-powered negative, and why