What a 64% backtest actually means
The model called 64% of 2025 games correctly with a Brier score of 0.222. Here is why that is good, why it is not impressive, and which number to watch instead.
Replayed across the 2025 season, the model picked the straight-up winner in 64.0% of games, with a Brier score of 0.222 and a log loss of 0.635. Those are the headline numbers on the accuracy page. They are worth understanding properly, because straight-up accuracy is the least useful of the three.
The bar is higher than 50%
Picking the home team every week gets you well past a coin flip on its own, and picking the Vegas favourite lands in the mid-60s most seasons. A model at 64% is therefore in the neighbourhood of the market, not beating it by a distance. Anyone quoting a 70%+ season-long hit rate is either counting a small sample, grading their own picks after the fact, or selling something.
Brier score is the honest one
Brier score measures the squared distance between a stated probability and what happened, averaged over every game. Always saying 50% scores 0.25. Perfect confident calls score 0. The model's 0.222 says its probabilities carry real information — and that the margin over a shrug is meaningful but modest.
| Metric | 2025 backtest | Reference point |
|---|---|---|
| Straight-up accuracy | 64.0% | Coin flip: 50% |
| Brier score | 0.222 | Always 50%: 0.250 (lower is better) |
| Log loss | 0.635 | Always 50%: 0.693 (lower is better) |
| Games replayed | 272 | Full 2025 regular season |
Calibration matters more than accuracy
The question that actually matters for a probability model is not how often it is right — it is whether a 70% call wins about 70% of the time. That is calibration, and it is what the confidence buckets on the accuracy page are for. A model that says 90% and wins 70% of those games is dangerous even if its raw hit rate looks fine, because the number is lying about how sure it is.
Why the split test is in the results
Any set of constants can be tuned until it looks brilliant on the data used to tune it. To guard against that, the constants were fitted on Weeks 1-13 and graded on Weeks 14-18. The fitting half scored 0.619 accuracy; the held-out half scored 0.692. The held-out number being the higher of the two is a good sign — it means the improvement was not an artefact of the fit — but five weeks is a small sample and should be read as reassurance, not proof.
A backtest tells you the model was not built backwards. Only the live tracker tells you whether it works.
That is why every prediction on this site is graded against the final score as games finish, and the running record is published whether it flatters the model or not.