Dr. Alexander King
sc-referee automatically checks single-cell analyses for statistical mistakes that commonly slip through analysis and review.
Core trick
Deterministic verifiers re-test single-cell analyses for pseudoreplication and lost donor pairing, and abstain when a validated check lacks evidence.
Description
sc-referee automatically checks single-cell analyses for statistical mistakes that commonly slip through analysis and review. It targets known failure modes—using the wrong model for count data, losing donor pairing, testing clusters selected from the same data, or claiming false-discovery control that the results do not reproduce—that can turn technical artifacts into convincing discoveries. Referee reads the analysis files, Claude maps unfamiliar folders into an explicit scientific design, and the scientist confirms what was tested. Deterministic verifiers measure the problem and abstain whenever evidence or a validated check is missing.
These failures are not hypothetical. In a published human melanoma analysis, Referee found that 95.3% of 16,289 reported cell-level discoveries did not survive patient-level inference; only 770 remained. GeneBench-Pro GB-P07 asked Claude Science to estimate how a genetic variant changes CXCL10 expression in activated monocytes from cell, donor, and empty-droplet data. Claude reported a positive effect (+0.484) when the benchmark reference was negative (-0.600), and its reviewer approved the analysis. Referee reconstructed an ambient-RNA contamination axis from the supplied data and conditionally proved the submitted model omitted it. Referee replaces plausibility judgments with encoded checks capable of catching specific mistakes that both bioinformaticians and frontier agents miss.
README excerpt
# sc-referee `sc-referee` checks a biology analysis for two of the most common mistakes in the field: pseudoreplication, which is counting repeated measurements from the same animal as if they came from different animals, and a rate reported over the wrong denominator, where a headline percentage is worked out from only the samples that survived filtering but presented as if it covered all of them. It reads the code and data files in your project folder. It never runs your code, it never asks a language model for an opinion, and it reports an error only when it can point at the exact lines that prove it. It has a false positive rate of 0%. It ships as a Claude Code skill and a Codex skill, with the command-line tool underneath. The tool asks you a few things it cannot work out from the files, such as which column identifies an animal; the agent relays those questions to you and shows you the report. The actual arbitration is independent of the agent and is determined by code, which means that it can catch mistakes LLMs cannot. ## What it catches today - **Pseudoreplication.** The analysis fed six measurements from one mouse into a test that treated them as six mice, so the p-value is far too small. You tell `sc-referee` which column identifies the independent unit. It reads the data file and proves that the test received more than one row per unit, and it cites the lines of code that fed the test. - **A rate reported over the wrong denominator.** Your write-up says