Glassbox
Glassbox is a self-improving forecasting engine.
Core trick
A diagnostician agent reads graded forecast errors and rewrites the forecasting rulebook each generation, with web access disabled to isolate reasoning gains.
Description
Glassbox is a self-improving forecasting engine. Hand it any set of already-answered yes/no questions and it teaches itself to forecast them better, with no new information, by grading its own predictions, finding its own mistakes, and rewriting its own rules, generation after generation.
Why forecasting? On prediction markets like Kalshi, a crowd prices the future and often beats the experts. That is the wisdom of crowds, and once a market resolves you are left with a forecasting question that has a verified answer. Forecasting is really two skills: getting information, and reasoning well about what you already have. Tetlock's research found the best forecasters win on the reasoning. So we built a machine to sharpen that second skill on its own, with the internet switched off.
It runs as one repeatable Claude Code workflow with three autonomous agents: a forecaster, an isolated grader, and a diagnostician. They loop unattended for five generations. The whole run is leak-proof, zero web calls, so any gain is provably the model reasoning over its own mistakes, not looking anything up. On resolved Kalshi markets it measurably improved its accuracy on questions it had never trained on, and twice it judged there was no honest gain left and deliberately changed nothing.
The deliverable is the engine, not a forecaster. The same workflow, same git commit, taught itself a second and unrelated skill, "will this GitHub issue close within 30 days?", with no code changes. Point it at any history of verified outcomes, like tickets, claims, trades, or SLAs, and it bootstraps a calibrated, self-documenting forecaster for that domain, where every rule it writes cites the evidence that created it.
README excerpt
# Glassbox — a model that finds its own mistakes and rewrites its own rules **Live demo → https://raghavrajsah.github.io/glassbox/** *An autonomous loop: the model forecasts, an isolated grader scores it, a diagnostician finds its own systematic errors and rewrites its own rulebook — five generations, no internet access, and it stops when there's no honest improvement left.* **Why it's interesting:** getting better at a skill normally takes new information. Glassbox gets measurably better with **none** — purely by auditing the pattern in its own past mistakes. We prove the gain is real by running the whole loop leak-proof (zero web calls) and scoring on held-out questions it never trained on. Forecasting resolved prediction markets is just the testbed; the deliverable is the self-improving **engine** — and it transfers to a brand-new domain with no code changes. --- ## The idea A crowd guessing the number of jellybeans in a jar averages out close to the truth — "the wisdom of the crowd." Prediction markets like **Kalshi** and **Polymarket** turn that into a price: a contract that pays $1 if an event happens trades at the crowd's probability that it will. Once the event resolves, you're left with something rare — a forecasting question with a *verified* answer. Thousands of resolved markets are, in effect, a giant graded exam for forecasters. And forecasting is really *two* skills: **getting information**, and **reasoning well about the information you already have** —