VERITAS
VERITAS is an independent verifier for AI-generated single-cell findings.
Core trick
Re-runs an AI-generated single-cell finding across alternative analysis choices, including donor-level pseudobulk, and returns SURVIVES, DOES NOT SURVIVE, or UNVERIFIABLE.
Description
VERITAS is an independent verifier for AI-generated single-cell findings. A copilot says "gene X is up in Alzheimer's." VERITAS re-runs that claim across every defensible analysis choice the analyst didn't try, checks the experimental design for structural confounds, and returns a signed verdict: SURVIVES, DOES NOT SURVIVE, or UNVERIFIABLE. Every flag carries a re-derivable statistic, never a model opinion.
The methods are real, not heuristics: donor-level DESeq2 pseudobulk, design-matrix batch models, Leiden, Harmony, Scrublet. Where a real method wasn't possible (ambient contamination needs empty-droplet data we never receive), we deleted the check rather than ship a proxy.
What we found: pointing VERITAS at real published Alzheimer's cortex data exposed a false-green in VERITAS itself. A finding that broke only at the pseudobulk axis still scored 0.94 and was certified GREEN — the exact failure the tool exists to catch. We made pseudobulk a critical axis and wrote it up.
Why it matters: Mathys et al. 2019 reported 14,274 Alzheimer's DEGs computed at the cell level. A donor-level reanalysis (eLife 2023) found 26. AI now generates findings faster than anyone can check them. Claude Science ships a reviewer, but it's the same model checking itself. VERITAS is the independent check: it re-runs the analysis and computes the number.
README excerpt
# VERITAS **CI/CD for scientific conclusions — the independent trust layer that checks whether an AI-generated finding actually survives re-analysis.** Every AI copilot in science generates findings. None of them tells you which ones are real. Only ~6% of published biomedical notebooks reproduce, and copilots now generate results faster than anyone can check them. VERITAS is the layer between *"the AI said so"* and *"we published it."* > **⭐ The headline result — [VERITAS caught a false-green in *itself*](docs/false_green_result.md).** We pointed it at real published Alzheimer's snRNA-seq, graded it against an *independent reanalysis* (Mathys 2019 → eLife 90214, ~549× fewer DEGs), and it correctly refused the published cell-level findings — exposing a false-green bug in its own verdict logic on the exact failure mode it exists to catch. **[Read the write-up →](docs/false_green_result.md)** ## The problem Existing research-integrity tools (Ripeta, SciScore) check whether a *manuscript reports* trust markers — data statements, RRIDs, code availability. They read the paper. **VERITAS checks whether the finding itself holds up when you re-run the analysis.** Report-level vs. result-level. ## What it does - **Multiverse engine** — re-runs a finding across the defensible choices the analyst didn't try, each a **real method**: QC thresholds, normalization, a **design-matrix batch model** (`~ condition + batch`), DE test, Benjamini-Hochberg correction, **real Leiden re-cluster