What Is a Brier Score and Why It Matters for Your Decisions

The Simple Explanation

A Brier score measures how close your probabilistic predictions are to what actually happened. Think of it as the distance between what you believed and what turned out to be true.

Imagine you say there is an 80% chance of rain tomorrow. If it rains, your prediction was close — you get a low (good) score. If it stays sunny, your prediction was far off — you get a high (bad) score. The Brier score captures this gap with a single number between 0 and 1.

The Formula

For a single prediction, the Brier score is straightforward:

The prediction is your probability estimate (e.g., 0.80 for 80%), and the outcome is what actually happened (1 if it occurred, 0 if it did not). When you have multiple predictions, the Brier score is the average of all the individual squared errors.

Let us walk through an example. You make three predictions:

• 1."90% chance this deal closes" — it closes. Score: (0.90 − 1)² = 0.01
• 2."60% chance the product ships on time" — it does not. Score: (0.60 − 0)² = 0.36
• 3."30% chance the candidate accepts" — they do not. Score: (0.30 − 0)² = 0.09

Average Brier score: (0.01 + 0.36 + 0.09) / 3 = 0.153. That is a solid result — better than random guessing.

What the Numbers Mean

The Brier score ranges from 0 to 1, and lower is always better:

A score of 0.25 is the critical benchmark. It is what you would get by flipping a coin and always assigning 50% confidence. Anything below 0.25 means you are adding real information with your predictions. Anything above it means you would be better off guessing.

Where the Brier Score Is Used in Practice

The Brier score is not an academic curiosity — it is used in some of the highest-stakes prediction environments in the world:

• Weather forecasting. The National Weather Service uses the Brier score to evaluate precipitation forecasts. It is the primary metric for measuring whether forecasters are improving over time.
• Intelligence analysis. The Intelligence Advanced Research Projects Activity (IARPA) used Brier scores in the Good Judgment Project — the largest forecasting tournament ever run. Superforecasters were identified by consistently achieving Brier scores around 0.15.
• Medicine. Clinical prediction models for patient outcomes are evaluated using the Brier score. A surgeon who says there is a 20% chance of complications should be right about 20% of the time across many cases.
• Venture capital. Some funds have started tracking Brier scores for their investment committee predictions. When a partner says a deal has 70% chance of returning 3x, that claim should be falsifiable and trackable.

Why It Beats Simpler Metrics

You might wonder: why not just count how many predictions you got right? The problem with binary accuracy (right/wrong) is that it throws away the most valuable information — your confidence level.

Suppose two analysts both predict that a company will beat earnings. Analyst A says 95% confident, Analyst B says 55% confident. The company beats earnings. Binary accuracy gives both analysts the same score: correct. But clearly Analyst A made a much stronger, more useful prediction.

The Brier score captures this. Analyst A gets 0.0025 (excellent), while Analyst B gets 0.2025 (mediocre). Over hundreds of predictions, the Brier score separates people who truly understand the world from those who are just lucky.

How to Improve Your Calibration

The good news is that calibration is a skill, not a talent. Research from the Good Judgment Project shows that people can meaningfully improve their Brier scores with deliberate practice. Here is what works:

1. Track your predictions. You cannot improve what you do not measure. Start logging predictions with explicit probability estimates, then check back when outcomes are known.
2. Use base rates. Before estimating a probability, ask: how often does this type of thing happen in general? Anchoring to the base rate and adjusting from there dramatically reduces overconfidence.
3. Update when you get new information. Good forecasters revise their estimates as new evidence arrives. Stubbornly sticking to an initial estimate is the hallmark of poor calibration.
4. Think in ranges, not points. Instead of saying "I think this will work," say "I think there is a 70% chance this will work." Forcing yourself to put a number on it exposes hidden uncertainty.
5. Get feedback fast. The tighter the loop between prediction and outcome, the faster you learn. Weekly business decisions resolve faster than decade-long bets, so they are better training ground.