What is a calibration curve?

Group a wallet's entries into confidence bands by entry price (all the roughly-70-cent entries together, all the roughly-30-cent entries together, and so on). For each band, compare the average price paid with the share that actually resolved true. A calibrated wallet's 70-cent entries win about 70% of the time. Each band reports its sample size, realized win rate, expected probability, and the signed gap between them; empty bands report null, never fabricated values.

What is a Brier score and what is a good one?

The Brier score is the mean squared error of entry prices against resolved outcomes: for each position, square the gap between the price paid and what happened (1 or 0), then average. Lower is better: 0 is perfect, 1 is the worst possible. Always predicting 50% scores 0.25, so scores meaningfully below 0.25 beat coin-flip hedging. It is scored per wallet and per market category.

Does good calibration mean high profit?

No. Across the 8,656-wallet reference cohort, the Spearman rank correlation between Brier score and realized profit is only +0.148. Calibration measures whether your entry prices matched reality on average; profit also depends on sizing, concentration, and a fat-tailed outcome distribution. A wallet can be beautifully calibrated on tiny stakes and barely profit.

Calibration and the Brier score

The answer first

An entry price is an implicit forecast: paying 70 cents says 70%. Calibration asks whether those implicit forecasts matched reality on average. The calibration curve shows it band by band; the Brier score compresses it into one number where 0 is perfect, 1 is the worst possible, and lower is better.

The intuition, with one worked example

Take one position entered at 70 cents. If it resolves true, the Brier contribution is the squared gap between 0.70 and 1: that is 0.30 squared, 0.09. If it resolves false, the gap is 0.70 to 0: squared, 0.49. Being wrong from a confident price costs more than five times as much as being right earns, which is what makes the score hard to game: confident nonsense is punished quadratically. Average those contributions over a wallet's resolved record and you get its Brier score. A wallet that always entered at 50 cents would score 0.25 no matter what happened, so scores well below 0.25 mean the wallet's confidence carried real information.

The actual method

Calibration curve. Entries are grouped into confidence bands by entry price. Each band reports its sample size, realized win rate, expected probability, and the signed gap between them. Empty bands report null, never fabricated values.
Brier score. The mean squared error of entry prices against resolved outcomes, per wallet. Lower is better.
Per-category splits. The same diagnostics run per market category (politics, sports, crypto, and so on): resolved count, Brier, a baseline Brier (what the trivial always-predict-the-base-rate strategy would score), the gap between baseline and wallet, and win rate. This surfaces one-trick records: a wallet can be sharp in one category and noise everywhere else.

Where you see this on the site

The analyzer renders the calibration bands and the per-category table for any wallet, and the same fields ship in the API and the enterprise deliverable. The deeper glossary pages are /learn/calibration and /learn/brier-score.

What this does NOT mean

Good calibration is not high profit. Across the 8,656-wallet reference cohort, the rank correlation between Brier score and realized profit is only +0.148: calibration is one ingredient, not the recipe. A calibrated wallet can size timidly and earn little; a poorly calibrated one can get lucky on one big event. And a strong Brier score is not a verdict: the skill read still runs through the realized edge, its interval, and the four-state gates from lesson 4.

Next lesson: concentration

Convexly publishes new methodology research roughly every 6-8 weeks plus the /learn series on a rolling cadence. Get the next paper in your inbox when it ships: