What is the multiple-comparisons problem in plain words?

Run a test that is wrong 5% of the time on 3,871 records and you expect about 194 false positives even if nobody has skill. Under that global null we would expect 193.55 wallets past an uncorrected 5% bar; the actual data shows 427, so there is real signal, but an uncorrected list would still be heavily salted with lucky records. The correction exists so the published count is not manufactured by the number of tests.

What exactly does the registry result say?

Of 3,871 Polymarket wallets with at least 30 resolved discretionary positions, 178 clear Benjamini-Hochberg false-discovery control at q = 0.10 for positive realized edge over their entry prices, in sample. Expected false discoveries among those 178 are about 17.8, so even the cleared set is stated with its own error budget. The cleared proportion is 4.6%, with a 95% interval of [3.98%, 5.30%].

Are the 178 cleared wallets a list to follow?

No. The result is in-sample skill-vs-luck separation on the past record, not forward skill, and it is not investment advice. The out-of-sample forward test that would address persistence is filed and maturing, verdict pending. Published signals derived from the registry carry counts, never the member addresses.

Many wallets at once

The answer first

When you evaluate one wallet, a 95% interval is a reasonable bar. When you evaluate 3,871 wallets, that same bar waves through about 194 lucky records on average even if nobody has skill. So before Convexly calls any set of wallets a finding, it applies false-discovery control: a correction that raises the bar with the number of records tested and caps the expected share of false positives in whatever clears.

The intuition, with one worked example

The registry run is the worked example. We tested every Polymarket wallet with at least 30 resolved discretionary positions: 3,871 records. Under a pure global null (no skill anywhere), an uncorrected 5% bar expects about 193.55 wallets through by luck. The actual count past that uncorrected bar was 427, more than double the chance expectation, which says there is real signal in the population. Then the correction: of 3,871 wallets with 30 or more resolved positions, 178 clear the corrected test at q = 0.10, in sample, and expected false discoveries are about 17.8 even among those 178. That is 4.6% of the tested universe, with a 95% interval of [3.98%, 5.30%]. The honest headline carries its own contamination estimate.

The actual method

The correction is the Benjamini-Hochberg procedure. In plain words:

Each wallet gets a p-value: the probability that a record at least this good arises by chance if the entry prices were exactly right (for each position, chance of winning = price paid).
Sort all 3,871 p-values from smallest to largest and compare each to a rising bar that depends on its rank and the chosen error budget q.
At q = 0.10, the procedure caps the expected share of false discoveries among the cleared set at 10%. Clear 178 and the expected number of impostors among them is at most about 17.8. You buy discoveries with a known contamination budget.
In cohort deliverables the same machinery runs at q = 0.05, 0.10, and 0.20 so a client can see how sensitive the survivor count is to the threshold, and a wallet can be reported as a survivor only if its four-state read is skilled.

Where you see this on the site

The registry counts appear in Profit Is Not Proof and across the research surfaces. The same correction is the spine of the enterprise cohort audit, and the frozen scan of our own board at /research/top50-skill-scan shows what it looks like when zero records clear. The glossary page is /learn/false-discovery-rate.

What this does NOT mean

The 178 are not a list to follow, and Convexly does not publish their addresses as a skilled-trader roster; registry-derived signals carry counts, not identities. The result is in-sample: it reads the past record, and a wallet can clear in-sample and revert. Whether the cleared set keeps outperforming is exactly the question the filed out-of-sample forward test exists to answer, and its verdict is pending while the window matures. A null outcome there would be published like any other result.

Next lesson: negative controls and grades

Convexly publishes new methodology research roughly every 6-8 weeks plus the /learn series on a rolling cadence. Get the next paper in your inbox when it ships: