How to Audit a Cohort of Polymarket Wallets

Auditing a cohort of Polymarket wallets is a two-part problem. First score each wallet on its resolved positions: a realized entry edge reported with its 95% confidence interval and the number of resolved positions behind it, plus a concentration read. Second, correct for the fact that you tested many wallets at once with false-discovery-rate control, run a negative control, and report which wallets stay distinguishable from chance and which do not. The single-wallet question (is this trader skilled or lucky) becomes a multiple-testing question the moment you screen a list, and the correction is what separates a genuine cross-sectional read from the tail of the luck distribution.

Step 1: score each wallet on its resolved positions

Every wallet in the cohort is scored the same way, on closed history only. For each one you compute:

• Realized entry edge with its 95% confidence interval and the count of resolved positions behind it. The interval and the position count always travel with the number: an edge estimate from 12 resolved positions is a different object from the same point estimate on 400, and the confidence interval is what makes that visible.
• Resolved-position count as a standalone gate. Below a floor of resolved positions there is not enough closed history to separate skill from noise, and the wallet is reported as insufficient sample rather than scored.
• Concentration, which flags whether the realized edge rests on many independent resolutions or a few large ones. A wallet whose entire edge comes from one or two oversized wins is a different risk object from one whose edge is spread across a long book, even at the same point estimate.

Step 2: correct for multiple testing across the cohort

This is the step a naive leaderboard skips, and it is where most cohort claims fall apart. If you test 50 wallets each at the 5% level, chance alone is expected to produce roughly two or three that clear the bar even if none has any edge. Reporting those as skilled is a false-discovery problem, not a finding.

The fix is false-discovery-rate control (Benjamini-Hochberg), which bounds the expected share of false positives among the wallets you flag. Applied across the whole cohort, it converts a pile of individual p-values into a defensible statement: at this corrected bar, these wallets stay distinguishable from chance and these do not.

Step 3: run a negative control

A cohort audit is only trustworthy if the pipeline finds nothing when there is nothing to find. A negative control is a run where the signal has been removed by construction (shuffled outcome labels, or a placebo cohort): a correct method should return zero wallets distinguishable from chance. If the negative control still lights up wallets, something is leaking (look-ahead bias, a resolution-timing bug, a mis-specified null), and every positive finding is suspect until it is fixed. Running it beside the real cohort is how you tell a genuine read from a plumbing artifact.

Step 4: report skill vs chance, honestly

The deliverable is a per-wallet verdict: distinguishable from chance at the frozen bar, or not, each read alongside its realized edge, 95% confidence interval, resolved-position count, and concentration. What the honest version of this looks like is worth stating plainly. When Convexly ran this exact procedure on its own published Top-50 Edge Score cohort, 35 of the 50 wallets had enough resolved positions to read; exactly one wallet's realized-edge interval cleared zero on the positive side (about what chance predicts across 35 tests), and it did not survive the multiple-comparisons correction, so zero wallets cleared the corrected bar.

A cohort audit that returns “none of these clear the bar” is a valid, publishable result, not a failed one. Read the full per-wallet table and method in the Top-50 skill scan.

What a cohort audit is not

“Distinguishable from chance” is a statement about resolved history, not a forecast. A cohort audit does not:

• Predict which flagged wallets will keep an edge going forward. Convexly's per-wallet temporal holdout did not clear the forward threshold it was filed against, so a distinguishable-from-chance label is descriptive, not validated forward skill.
• Tell you to copy or follow any wallet. Convexly does not take custody, broker trades, or route investment advice; the audit is a skill-vs-luck read, not a recommendation.
• Replace the concentration and resolved-position context. A wallet that clears the bar on a thin, concentrated book is a weaker read than one that clears it across a long, diversified history, and the table keeps both visible.