Lesson 3 of 12

Confidence intervals

Every estimate on this site carries its uncertainty in the same sentence. A wide interval means thin evidence, and the interval decides the verdict, not the point estimate.

The answer first

A confidence interval is the honest range around a measurement: the set of values the evidence actually supports. On this site, no edge, delta, or rate is published as a bare number; the 95% interval and the resolved-position count travel with it. The rule exists because a point estimate from 47 positions and a point estimate from 470 positions can look identical while meaning completely different things.

The intuition, with one worked example

Compare two real reads from our own published board. One wallet shows a realized edge of +5.6 points, 95% CI [+1.2, +9.5], on 137 resolved positions. Its interval sits entirely above zero: every edge value consistent with that record is positive. Another wallet, Zarvantis, shows -2.5 points, 95% CI [-7.9, +1.9], on 47 resolved positions. That interval spans zero: the record is consistent with a modest negative edge, no edge, or a small positive one. Same statistic, same method, opposite conclusions about what you can say. The first record supports a read. The second supports honesty about not knowing.

The actual method

The intervals here are BCa bootstrap intervals (bias-corrected and accelerated). In plain words:

  • Take the wallet's own resolved positions and resample them many times with replacement, as if replaying alternate versions of the same record. The registry diagnostic uses 2,000 seeded resamples per wallet, so results are reproducible.
  • Recompute the realized edge on each resample.
  • Keep the middle 95% of those recomputed edges, after two standard corrections (for bias and for skew) that make the band more accurate on lumpy records than the raw percentile method.

The pairing of estimate and interval is enforced in code, not by style: the API schema requires the interval field whenever the edge field is present, and the CSV export emits an edge only when both bounds are present. A wallet with no computable interval reports a null edge, never a naked point estimate.

Where you see this on the site

Everywhere an edge appears: the analyzer, the wallet board, the embeddable skill badge, the API, and the enterprise cohort deliverable all print the interval in the same breath as the estimate. The research pages hold the same line: a published delta without its interval is treated as a bug.

What this does NOT mean

An interval that spans zero is not proof of no skill. It means the record cannot distinguish an edge from chance either way; the wallet may be skilled and under-sampled. An interval clearing zero is not a promise about future trades; it is a property of resolved past positions. And 95% is not a guarantee on any single wallet: across many such intervals, about 1 in 20 will miss. The interval is a discipline, not an oracle.

Convexly publishes new methodology research roughly every 6-8 weeks plus the /learn series on a rolling cadence. Get the next paper in your inbox when it ships:

Frequently asked

What does the 95% interval on a wallet read mean?

It is the range of edge values consistent with the wallet's resolved record. A read of +5.6 points with an interval of [+1.2, +9.5] says the record supports an edge somewhere in that band. A read of -2.5 with an interval of [-7.9, +1.9] spans zero, so the record cannot distinguish a real edge from chance either way. The point estimate is the middle of the story, never the whole story.

What is a BCa bootstrap interval in plain words?

Resample the wallet's own resolved positions many times with replacement, recompute the edge each time, and read off the middle 95% of those recomputed values, with two adjustments (bias correction and acceleration) that fix the raw percentile method's known distortions on skewed records. In the registry diagnostic each wallet gets 2,000 seeded resamples, so the same wallet always yields the same interval.

Why do some wallets have wide intervals?

Fewer resolved positions and lumpier outcomes mean less information, and less information means a wider interval. Wide interval = thin evidence. That is not a flaw in the method; it is the method refusing to sound more certain than the record allows.

Related explainers

Related reading

LearnBrier score

LearnCalibration

LearnCalibration and Brier

LearnCohort audit