Brier score: how to grade a probability forecast
How to score a probability forecast with the Brier score: the binary formula, a five-question worked example, and what the number can't tell you.
Updated November 10, 2026 · Lumina Markets
On this page
"It'll probably happen" can't be checked against an outcome. "68%" can. The Brier score is one of the standard measures for doing that: it takes how far the stated probabilities landed from what happened, across a set of forecasts.
Lower is better. Zero is a perfect score and 1 is the maximum on the standard binary scale. Saying 50% to everything scores 0.25, which is the figure a set of forecasts is usually compared against.
The formula
BS (1/N) Σ (fₜ − oₜ)²
- fₜ: your forecast for question t, as a decimal between 0 and 1
- oₜ: what happened: 1 if the event occurred, 0 if it didn't
- N: how many forecasts you're scoring
In words: for each question, take the gap between your probability and the outcome, square it, then average the squares. Squaring is what makes a confident miss cost more than a cautious one: 95% on something that doesn't happen contributes 0.9025, while 60% on the same miss contributes 0.36.
- Penalty for a 95% forecast
- 0.9025
- Penalty for a 60% forecast
- 0.36
- The same miss, stated with less confidence
Worked example: a five-question slate
The slate below is illustrative, chosen to show how the arithmetic works rather than to describe any real market or any real forecaster's record.
| Question | Forecast f | Outcome o | (f − o)² |
|---|---|---|---|
| One party holds Senate control | 0.62 | 1 | 0.1444 |
| CPI prints above 3.0% | 0.30 | 0 | 0.0900 |
| The film opens above $100M domestic | 0.75 | 1 | 0.0625 |
| The city sets a new record high | 0.45 | 0 | 0.2025 |
| The model ships before the deadline | 0.85 | 1 | 0.0225 |
| Total | 0.5219 |
Brier score 0.52195 0.104
Against the reference points, 0.104 sits well below the 0.25 that 50%-on-everything would score, and well above 0.
- Brier score on the slate
- 0.104
- Saying 50% to everything
- 0.25
- A perfect score
- 0
Reading the number
| Score | What it means |
|---|---|
| 0.00 | Every forecast was 0 or 1, and every one was right |
| ~0.10 | Confident and mostly correct |
| 0.25 | What 50%-on-everything scores |
| >0.25 | Typically confident calls in the wrong direction |
| 1.00 | The maximum: 100% certain every time, and wrong every time |
The comparison that carries information is against 0.25 rather than against zero.
Why a yes/no hit rate misleads
Take the same five questions and imagine a forecaster who never commits: they say 51% on whichever side they expect, and they get the direction right all five times.
Their hit rate is 100%. Their Brier score is 0.240, barely distinguishable from the 50%-on-everything baseline.
- Hit rate at 51% on every call
- 100%
- Right direction all five times
- Brier score at 51% on every call
- 0.240
- Next to the 0.25 baseline
- Brier score for the slate's forecasts
- 0.104
A hit rate records only which side of 50% each forecast landed on, so it reads the same for a confident, precise forecaster and for someone hedging at 51% who leaned the right way. The Brier score uses the stated number, which is why "I called it" tells you less than the number behind it: 51% and 85% on the same question are different forecasts, and the score separates them.
Brier vs calibration: different questions
They're often mentioned together and they don't measure the same thing.
- Brier score answers: how far off were you, overall? One number, combining confidence and correctness.
- Calibration answers: when you said 70%, did it happen about 70% of the time? A pattern across buckets, not a single score.
The two can diverge. A forecaster who says 55% on everything, and is right that 55% of them happen, is well calibrated with a middling Brier score, because the forecasts never committed. In the other direction, a reasonable Brier score can sit on top of one badly-off bucket that the average absorbs. Brier is generally used to compare forecasters on the same question set; calibration tends to be more useful for locating which stated probabilities diverged from outcomes.
Scoring a market instead of yourself
The same arithmetic works on prices. Take a market's quoted probability at a fixed point (a week before resolution, say), treat it as f, and score it against the outcome the same way.
Two choices determine whether the resulting figure is comparable to anything:
- Choose the timestamp before you start
A price a day before resolution will typically score better than a price a month out, on any market, for reasons unrelated to forecasting quality. Settle the horizon first, then score.
- Score the whole set, not the memorable ones
Choosing which markets to include after seeing how they resolved will tend to produce a better score than the same method applied to the full set.
Four things a Brier score can't do
- Compare across different question sets. A 0.08 on near-certain questions represents less than a 0.18 on genuinely uncertain ones. The figure is generally only comparable when the question sets are.
- Survive a small sample. Five forecasts is a teaching example rather than evidence. On a set that size, a single surprising resolution typically moves the score more than a real difference in forecasting quality would.
- Tell you where the error sits. A single figure combines confidence and correctness, so it can't distinguish probabilities that were systematically too extreme from ones that were reasonable but wrong on a particular class of question. Splitting the same forecasts by stated probability — a calibration check — is what separates those.
- Account for base rates. Forecasting rare events well is harder than the score suggests; see base rate.
FAQ
How do you calculate a Brier score?
Square the gap between each forecast and its 0/1 outcome, then average the squares: BS = (1/N) Σ (fₜ − oₜ)². Lower is better.
What is a good Brier score?
The usual comparison is against 0.25, which is what saying 50% to everything scores. A score meaningfully below that is generally interpreted as skill on that particular question set; scores aren't comparable across different sets of questions.
Is a lower Brier score better?
Yes. It's an error measure: 0 is a perfect score and 1 is the maximum on the binary scale.
Can you score a prediction market with it?
Yes. Treat the quoted probability at a chosen horizon as the forecast and score it against the outcome. Settle the horizon before you look, and score every market in the set rather than the memorable ones.