The Brier score measures how good probability forecasts are: square the gap between each forecast and what happened (1 if it happened, 0 if not), then average the squares. Zero is perfect and lower is better, and saying 50% on every two-way event scores exactly 0.25, a baseline any model worth testing should score below.
Glenn Brier devised it at the US Weather Bureau to score weather forecasts. In betting it tests a model's chances before money is at stake, one of the checks in a betting strategy.
How to calculate a Brier score
Brier score = the average of (forecast - result) x (forecast - result), with the result 1 for a win and 0 for a loss.
Example: Six illustrative AFL games, none drawn, scoring a model's chance of the home side winning and the market's margin-free chance.
| Game | Model's chance | Market's chance | Result | Model's score | Market's score |
|---|---|---|---|---|---|
| 1 | 0.75 | 0.72 | 1 | 0.0625 | 0.0784 |
| 2 | 0.60 | 0.62 | 0 | 0.3600 | 0.3844 |
| 3 | 0.40 | 0.45 | 0 | 0.1600 | 0.2025 |
| 4 | 0.55 | 0.52 | 1 | 0.2025 | 0.2304 |
| 5 | 0.30 | 0.26 | 0 | 0.0900 | 0.0676 |
| 6 | 0.65 | 0.70 | 1 | 0.1225 | 0.0900 |
| Brier score | 0.9975 / 6 = 0.16625 | 1.0533 / 6 = 0.17555 |
Game 1's model score is (0.75 - 1) x (0.75 - 1) = 0.0625. Both beat the flat 0.25, and the Brier skill score against the market, 1 - 0.16625 / 0.17555 = 5.3%, puts the model 5.3% ahead on these games.
For a race, score each runner as a yes or no and average them, or sum the squares across the field, Brier's original form, which runs from 0 to 2.
Why accuracy is not the same as profit
Standard footy tipping counts correct tips and ignores confidence: the model and the market above both favour the winner in five games of six, yet score differently. With illustrative picks, saying 90% on four picks that win three times scores (3 x 0.01 + 0.81) / 4 = 0.21. Saying 75% on the same picks scores (3 x 0.0625 + 0.5625) / 4 = 0.1875, so the honest 75% does better.
Profit also needs prices: your chance has to beat the price on offer, after the margin, on the games you bet. A model can beat the market's score and still lose money if its gains sit in games where it agreed with the market and placed no bet.
Using it to test a betting model
- Record each forecast before the event, with the market's margin-free chance at that time.
- Score both on the same events, never on games the model was built from.
- Check calibration: forecasts near 60% should win about 60% of the time, as machine learning in betting shows.
- Score the bets you would have placed on their own.
A good model's lead over the market is small next to the noise in a few hundred games, so read it over a season, as sample size in betting explains. How to build a betting model and backtesting betting strategies set out the full test.
Market odds for scoring your own model
B337 has no model and makes no selections. To record the market side as you go, API Odds sends live racing and sports odds from Australian bookmakers and Betfair into your own models. It places no bets and needs coding; coverage and access are agreed at setup with the team, and the price, including any charges on top of the plan price, is confirmed before you pay. Bets placed through B337 use credits. Betfair is a trade mark of its owner, and B337 is not affiliated with it.
For free and confidential support call 1800 858 858 or visit gamblinghelponline.org.au.
Risk: Betting involves risk. A good Brier score describes past forecasts, not the next bets, and a bet priced from a sound model can still lose. See responsible gambling for limits and support.