Where the Swarm Pays

Nothing in the framing of this note is new; the measurements are. The premises come from four places, and it is worth naming them because the vocabulary of the LLM-swarm literature borrows from all four without always saying so.

Juries. Condorcet's jury theorem (1785) says that a majority of independent voters, each right with probability above one half, is right with probability that goes to one as the jury grows. Boland (1989) and Ladha (1992) worked out what happens when the votes are correlated: the majority's accuracy no longer goes to one but to a plateau set by the correlation, and above a threshold correlation adding voters adds nothing. The theorem is about a binary choice; its extension to a plurality over many candidate answers is List and Goodin (2001). Every "N agents beat one" claim is a Condorcet claim, and every vote ceiling in this note is a Boland–Ladha plateau; the coverage ceiling of Section 4 is a different bound.

Surveys. Kish (1965) gave the design effect for a cluster sample: n correlated observations with intraclass correlation ρ carry the information of n/[1+(n−1)ρ] independent ones, which is bounded by 1/ρ however large n grows. The first note on this site applied this to LLM resamples as the effective number of independent voters, n_eff. It is a formula about the variance of a mean; Section 8 shows what it does and does not say about a majority of binary votes.

Latent correlation. Two agents' correctness is a 2×2 table, and the Pearson correlation of two 0/1 variables, φ, cannot reach 1 unless their accuracies are equal. Pearson (1900) defined the tetrachoric correlation as the correlation of two latent normal variables that would produce the table, and gave the cos-π approximation to it; Fisher's intraclass correlation (1925), in the one-way analysis-of-variance form of Shrout and Fleiss (1979), is the panel version of φ on the observed scale, not a latent quantity; it is used here because the debate note used it. The one-factor Gaussian model used in Section 7 is Vasicek's (1987) single-factor model of correlated defaults, in which every borrower loads on one latent economic state; here every model would load on one latent item difficulty.

LLM ensembles. Self-consistency, the plurality over sampled chains of thought, is Wang et al. (2022, arXiv:2203.11171). Multi-agent debate in the form used here, agents that read each other's answers and revise, is Du et al. (2023, arXiv:2305.14325). Mixture-of-agents, a model that reads several answers and writes one, is Wang et al. (2024, arXiv:2406.04692). The LLM-as-a-judge paradigm and its self-preference bias are Zheng et al. (2023, arXiv:2306.05685) and Panickssery et al. (2024, arXiv:2404.13076). The 2025–26 papers whose claims this corpus was built to test are named in Section 17; three of them frame the whole note: the co-failure ceiling (arXiv:2606.27288), which bounds routing, voting and any scheme that returns one member's answer by the rate at which all members fail together; the modal and correlation ceilings of test-time scaling (arXiv:2606.28661), which the previous note tested on the public debate traces of arXiv:2609.35279 (the DebateLedger release); and Nine Judges, Two Effective Votes (arXiv:2605.29800), which found a panel of nine judges to be about two effective votes; Section 13 measures the same quantity for five judges on a different task and finds three.

The three earlier notes on this site¹ established, on public data, that LLM pools are one to two effective voters, that diversity metrics do not predict vote gain once accuracy is controlled, and that debate raises correlation while escaping unanimous error on weak models. This note puts all of that on one corpus with the same items for every operator. It also found, on the way, that the co-failure rate those premises are built around cannot be measured on these benchmarks without first auditing the benchmarks.

The hypotheses in the swarm literature can be sorted into eight questions; this is one such sorting, fixed as axes before the corpus was built, and it is a working taxonomy rather than a claim about the field. The note covers five of them.

A. Parallel answers and aggregation

Does pooling independent answers add intelligence? Sections 7–8.

B. Debate

Is a multi-round exchange a different operator from sampling, and what does it transmit? Sections 10–11, with synthesis in Section 12.

C. Selection and verification

Can a verifier reach the ceiling that voting cannot, and how independent is it from the solvers? Sections 13–15.

D. Sources of diversity

Can independence be bought with families, roles, prompts or temperature? Sections 7–9.

G. Cost at equal budget

What does each operator cost in tokens, and when does it pay against one model given the same budget? The token side is in Section 16; the cost-matched comparison is the next note.

Three axes are not in this note. E, iterated chains: swarms in time, self-training and iterated in-context learning, and whether they converge to the prior. F, organisation with tools: agents that decompose a task, take roles, keep memory and call code or search, which is what industrial agent systems do; the corpus has no decomposition and no tools. H, theory as bound: which of the classical results above is tight on which data; the Kish check in Section 8 is a first instalment, not a treatment.

A literature-based-discovery pipeline over about a thousand agent-swarm papers paired one paper's stated limitation with another paper's result wherever the two did not cite each other, scored each pair with an LLM judge, and produced 947 candidate hypotheses with a score of 4 or more out of 10, 140 of them at 6 or more. A handful were tested the usual way, one bridge at a time, two of them in the previous notes. Reading the rest made the economics obvious: most ask for one of five or six measurements on one kind of data, a correctness matrix across model families, resamples of one model, multi-round debate traces with every intermediate answer kept, a judge panel, and the token count of each call. A bridge test that costs ten dollars is not worth running; a corpus that answers forty bridges for fifty dollars is.

So the corpus was designed from the hypotheses, built in two days, and each of the 140 was then read against its tables by a model instructed to say "supported", "not supported", "mixed" or "the corpus does not measure this". After three passes, each adding the cuts the previous pass had asked for, 27 hypotheses had an answer and 113 asked for things the corpus does not contain. Section 17 re-reads sixteen of the 27 against the audited tables. The rest of this note is the measurements themselves, with the definitions first because half the disagreements in this literature are about what the words mean.

Item, layer, correct

An item is one problem with one gold answer. A layer is a source dataset. A model's answer is correct or not. A truncated answer (the model hit its completion cap, 8,192 tokens for the OpenRouter models, before writing a final line) is missing: it is not a vote, not a dissenter, not a row in a correlation, and an item is not dropped because of it unless the table says so. Where a single agent is the control, its truncated answer scores 0, because a model that does not answer has not answered. The first two versions of this note counted truncated rows as wrong in the debate and judge stages; Section 10 keeps those tables in a sub-section.

Answer identity

Two answers are the same answer when they are equal after normalisation: numbers as numbers to seven significant digits, the grader's own tolerance (24, 24.00, $24, 5,072,530 and 5072529.6 are one answer each), LaTeX fractions and roots in one form. Every plurality, unanimity and candidate count in this note uses this identity. The first version compared raw strings; the second rounded to six decimals, which still split a handful of large GSM-Hard answers by formatting and so counted six items as contested that were not.

Best single

The highest accuracy among the agents on the same items, a truncated answer scoring 0. The standard control in the ensemble literature. It is the floor of the box: any aggregation that lands below it has cost accuracy. Where it matters, the note also gives the best agent's accuracy on the items that agent answered, which is the fairer control for a capped model and the one that decides whether debate adds information rather than tokens. Both are in-sample maxima over six or seven agents and flatter the single model by a point or so on the small layers.

Plurality

The answer given by the largest number of agents, ties broken by first occurrence, as in self-consistency. "Majority" is used for the same thing when the agents are a debate panel. Plurality is computed on answer identity, so two agents who are both wrong in different ways do not vote together.

Ceiling

The share of items on which at least one agent is correct, 1 − P(all wrong); the co-failure ceiling 1 − β of arXiv:2606.27288, the "oracle" or "coverage" of the test-time-scaling literature. No scheme that chooses among the agents' answers can exceed it; a scheme that writes a new answer can. The box between best single and ceiling is where every answer-side result in this note lives. After the label audit the ceiling is 0.998–1.000 on the items all seven models finished and 0.97–0.99 when a truncated answer is a missing vote (Section 7).

Unanimous, unanimous-and-wrong

All agents gave the same answer; that answer is wrong. A unanimously wrong item is invisible to any rule that uses agreement as a confidence signal. Before the audit, every unanimously wrong matrix item turned out to have a wrong or ambiguous gold label.

Fixed, broken

In a debate, an item whose plurality was wrong at round 0 and correct at the last round, and the reverse; arXiv:2609.35279 calls them corrections and collapses.

φ

The Pearson correlation between two agents' 0/1 correctness vectors over items, averaged over the pairs on which it is defined (the phi coefficient, Yule 1912). Each pair is computed on the items both agents answered. It is exact but bounded away from 1 when the two accuracies differ, undefined when either agent is constant, and it understates the correlation of the latent errors.

Tetrachoric ρ

The correlation of two latent normal variables that would produce the observed 2×2 table of correct/incorrect. Pearson (1900); we use his cos-π approximation, which is poor when the margins are skewed and degenerates to ±1 when a cell of the table is empty. Pairs with an empty cell are dropped from the average, and the number of pairs that survive is printed next to it; after the audit few survive on the small layers.

n_eff

Kish's (1965) effective sample size, n/[1+(n−1)φ], as used in Effective Independence. A design-effect formula from survey sampling, used here as a descriptive summary of how much independence n agents bought; Section 8 compares the majority it predicts with the majority observed.

ICC

Fisher's intraclass correlation in the one-way ANOVA form ICC(1) of Shrout and Fleiss (1979), of correctness within an item, the share of the variance of "this agent was correct on this item" that is explained by the item. An observed-scale quantity, close to φ on the debate panel and below it on the matrix; reported because the debate note used it.

Contested item

Introduced here. For the judge panel: the candidate answers disagree by answer identity and at least one of them is correct. On a contested item a perfect selector scores 1 and the plurality scores less; it is the only kind of item on which a verifier can earn anything.

Pick vs certify

Introduced here as names; both tasks exist in the LLM-as-a-judge literature as pairwise/listwise and pointwise judging. Pick: see all distinct candidates with a reasoning excerpt each, choose one or say "none". Certify: see one candidate alone, say correct or incorrect.

AUROC

Area under the receiver operating characteristic curve: the probability that a randomly chosen error receives a higher warning score than a randomly chosen correct answer; 0.5 is no information, 1 is perfect.

Design. The unit of analysis is an item answered by several agents, and every stage of the corpus is a different way of producing several answers to the same item: by different models, by resampling one model, by rounds of debate, by role prompts, by judging. The stages were chosen from the list of measurements the 140 hypotheses asked for, in descending order of how many hypotheses each would serve, and each was given a spend cap before it ran. Nothing was added after seeing a result except the cuts described below.

Controls. For every operator the comparison is to the best single agent on the same items and to the ceiling on the same items; both are computed from the same rows, never from a separate run. Debate has two controls built in: the round-0 plurality, which is what voting would have done, and the self-only arm, in which copies revise without seeing peers, which is what re-elicitation alone does. Judges are compared to the solvers' plurality on the same contested items, to each other, and, over the whole item set, to the best single solver. All comparisons are within-item; no number in this note compares one set of items against another.

Statistics. Accuracies are plain proportions. φ is the mean pairwise Pearson correlation of 0/1 correctness; the tetrachoric estimate is the cos-π approximation; ICC is the one-way ANOVA estimator; n_eff is Kish's formula applied to φ. The Kish-predicted majority is the probability that a majority of n_eff independent votes at the mean accuracy is correct: the exact binomial majority, ties split, at the two integers around n_eff, interpolated linearly. The first version of this note rounded n_eff to an integer, which made the prediction p² whenever it rounded to 2; the second used a normal approximation, which at n < 3 is off by up to four points in a direction set by p, not by φ, and produced a pattern Section 8 of that version mistook for a finding. Intervals, where given, are Wilson 95% intervals on proportions, or percentile bootstrap intervals over items (2,000 resamples) for differences; paired comparisons are exact McNemar tests on discordant items, uncorrected for the four layers. The strata of interest are small enough that the reader can see the counts, and they are reported as counts. Spearman correlations over model triples are over 35 triples per layer, which share models and are not independent.

Labels. After the first version of this note was written, the items on which the models had all been scored wrong were re-solved by Claude Opus 5.5 with explicit arithmetic, eight of them re-solved by hand as a check, the three GPQA Diamond items by hand only. The audit file holds 48 items. Forty-one of them are the items on which all seven models had finished an answer and been scored wrong; six more are items on which every debate agent that finished was wrong while one or two agents had been cut off; one, a GSM-Hard item with a negative count of toys, was added on inspection although four models had matched its gold. The decisions: 21 gold labels were wrong and were relabelled with the demonstrably correct answer (9 MMLU-Pro, 11 GSM-Hard, 1 GPQA Diamond); 22 items were ill-posed or had two defensible options and are excluded from every table (15 MMLU-Pro, 6 GSM-Hard, 1 GPQA Diamond); on 3 the models were wrong and the gold stands; and 2 olympiad items had been scored wrong because the matrix stage had graded expressions with the numeric grader (1/576 against \frac{1}{576}). Every row of every stage was re-graded with the normalising grader, which changed 61 further rows in the matrix, all on HMMT expressions. The first revision reported these as 45 items in a 24/16/2/2 split; that arithmetic did not match the file and is withdrawn. An independent re-check of the file for this revision agreed with all 11 GSM-Hard relabels and 20 of the 21 overall, and would have excluded rather than relabelled one MMLU-Pro item whose two options say the same thing. The audit, with the reason for each decision, is label-audit.json in the companion repository, and the re-grading script makes no model calls. Three things the audit does not cover. Items on which some model matched a wrong gold label were not read, so individual accuracies may still carry label noise in both directions. Items on which every answering model was wrong while another model had been cut off by its token cap were not selected, because the selection rule asked for seven finished answers; there are 18 such items after the audit, plus one all-seven-wrong MMLU-Pro item that sits in an older run file outside the tables, and they are listed in the audit file as unaudited. And the four items on which the six-agent debate is unanimously wrong at round 3 meet the stated rule but were not read either; two of them are GSM-Hard items whose gold has the hallmarks of partial substitution. All of these stay unrelabelled in this version and the note says so where they matter. The auditor is one of the audited models, and the new gold agrees with Opus's own first answer on every relabel; a reader who distrusts that should start from the file.

Truncation. The four OpenRouter models ran under an 8,192-token completion cap; the Claude models, through the CLI, had none. GLM and Qwen hit the cap on 68 and 19 of 573 MMLU-Pro items, 76 and 44 of 197 GPQA Diamond, 8 and 8 of 144 GSM-Hard and 42 and 50 of 90 olympiad items at temperature 0, and on 86 of the 1,350 round-0 rows of the debate; DeepSeek on 2. The matrix tables of Section 7 use the items every model finished, which drops the hard items and is said so there. In every other stage a truncated row is absent: it does not vote, it is not a dissenter, it is not in φ, ICC or the unanimity count, and an agent's accuracy is over the rows it answered; where a single agent is the control, its truncated answer scores 0. The first two versions of this note treated a truncated row as a wrong answer in the debate and judge stages, which depressed round-0 accuracy and correlation on exactly the layers where the capped models truncate, and produced the debate gains those versions reported; Section 10 keeps the old tables in a sub-section so that the two policies can be read side by side. Items are included in a stage's table when every agent of that stage has a row for it, truncated or not, which is why item counts differ between tables and are printed in each.

Exclusions. The 22 audited items are dropped everywhere. Five of the first-pass layers (MATH-500, GSM8K, CommonsenseQA, StrategyQA, Knights & Knaves) were dropped before any aggregation was computed. The first two versions said the criterion was that every model exceeded 0.95 on them; the matrix file does not bear that out (CommonsenseQA is 0.80–0.93 for the seven, Mistral is 0.66 on Knights & Knaves), and the honest description is that they were dropped as knowledge or near-saturated layers by judgment, which is a researcher degree of freedom. They remain in the matrix file. Nine of the 240 debates were never run because Mistral had no matrix answer to seed them with, and one debate lost Mistral on a rate limit; the six-agent tables use the 225 items with a row for every agent in every round, and a five-agent recount without Mistral is in the companion tables.

Post-hoc cuts. The first reading of the hypotheses against the tables asked for cuts the first tables did not have: the all-wrong stratum of debate, the transition table by round-0 correct count, the flip table, the copula and two-state fits, the judge committee thresholds, the error-detection AUROC. These were computed from the existing rows, with no new calls. The three homogeneous-debate arms, the certify stage and the synthesis stage were new runs made after the first reading, each with its own cap; they are the only data collected after a result had been seen. The label audit, the tie analysis of the judge panel and the whole-set accounting of Section 13 were added in the second version, after an external reading of the first, with no new calls. This third version, after a second external reading, changed the truncation policy, merged candidate answers within the grader's tolerance, replaced the Kish estimator, re-counted the audit, reported judge coverage, and replaced the reversible GPQA answer hashes in the companion data; again no new calls, and the old debate and judge tables are kept for comparison.

Reading the hypotheses. The 140 hypotheses were read against the first-version tables by Claude Opus 5.5 at low effort, three times, with instructions to quote only numbers present in the tables and to return "insufficient" whenever the quantity was not measured. The verdicts in Section 17 are the author's re-reading of those sixteen hypotheses against the audited tables.

Items. 240 problems with a single checkable answer, sixty from each of four layers: MMLU-Pro (ten-option multiple choice, stratified over subjects), GPQA Diamond (four-option graduate science), GSM-Hard (GSM8K word problems with large numbers substituted by a program; Section 5 says what that did to its gold labels), and the 2024 and 2025 AIME (integer answers), with the February 2025 HMMT (short expressions) in the matrix only; all public before the models' training cutoffs, so contamination is possible and unmeasured. A first pass over 1,576 items had included MATH-500, GSM8K, CommonsenseQA, StrategyQA and Knights & Knaves puzzles; they were dropped from every table (Section 5 says on what grounds) and remain in the matrix file. The seven-family matrix covers 573 MMLU-Pro items, 197 GPQA Diamond, 144 GSM-Hard and 90 olympiad after the exclusions; the debate, sampling and judging stages use the sixty per layer, and their sixty olympiad items are the AIME ones.

Models. Seven families, all current reasoning models in October 2026 and all allowed to think before answering, though not equally: Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 through the Claude Code CLI, where temperature is not controllable, the effort setting was low and there is no completion cap, so that Opus and Sonnet averaged 54 and 19 reasoning tokens per MMLU-Pro answer while Haiku averaged 2,200 and ran to 25,000 on the olympiad; and DeepSeek V3.2, GLM 5.3 Flash, Qwen 3.7 Plus and Mistral Large 2512 through OpenRouter under an 8,192-token completion cap, with the serving provider logged per call and no separate reasoning-token count returned. GLM and Qwen hit the cap on a third of GPQA Diamond and half of the olympiad problems (Section 5). Four further models answered subsets of the matrix and are not used here. Opus is in the matrix but not in the debate or judge panels; the panel was fixed at six agents when the stage was configured, and the design notes do not record why Opus was the one left out, so the reader should treat its absence as unexplained rather than principled.

Grading. Letters by exact match; numbers by numeric comparison with a relative tolerance of 10⁻⁶; expressions by a normaliser (fractions, roots, dollar signs, trailing approximations) plus an LLM arbiter (Opus, decisions cached) where the normaliser and the gold string still disagreed. Every answer was extracted from a required last line, FINAL: <answer>. The first version of this note had graded the matrix stage with the numeric grader alone, which is the grading failure described in Section 5.

Every row stores the parsed answer, its correctness, the provider that served the call and the token counts (prompt, completion, reasoning). The debate stages also store the full text of every turn and what each agent was shown. Nine debates were never run for lack of a Mistral seed answer and one lost Mistral mid-way; after the label exclusions the six-agent tables use 225 items and a five-agent recount on 226 is in the companion repository. The OpenRouter calls cost about $50; the Claude calls went through a subscription and would have cost about $335 at list price, which the byline of the first two versions left out.

The oldest question first: does the plurality of seven different models beat the best of them? On the items all seven finished, on no layer does it. On MMLU-Pro, the one layer large enough to test the margin, the best model is right and the plurality wrong on 13 items and the reverse on 4, an exact McNemar p of 0.05 before any correction for four layers; on the other three layers the plurality is behind by one item or tied, which is within what the in-sample choice of the best model can produce.

Items are those all seven finished without truncation, after the 22 exclusions, which is why these layers are larger than the sixty-item debate subsets and the olympiad is smaller: GLM and Qwen hit their token cap on 42 and 50 of the 90 olympiad items and on 76 and 44 of the 197 GPQA Diamond items. The selection is not neutral. Each model's accuracy on the items it answered is lower than on this subset, by a little on MMLU-Pro and by a lot where the cap bites: Haiku is 0.69 on the GPQA Diamond items it answered against 0.84 here, DeepSeek 0.61 on the olympiad against 0.93, Mistral 0.46 against 0.83. The per-model columns are accuracies on the problems two capped models could finish, not on the layer. They are 2–9 points higher than in the first version of this note, and that difference is the audit and the grader: the gold was wrong, not the models.

The last column was the surprise of the second version, and it needs the same caveat. On the items all seven finished, the share on which someone was right is 0.998 on MMLU-Pro and 1.000 on the other three layers: of 733 items, the seven models all fail on one. The first version reported all-wrong rates of 1–10% and fitted a one-factor latent model to them; those rates were made of wrong gold labels, ill-posed questions and a grader that could not read a fraction. But the items the capped models could not finish are the hard ones. Count a cut-off answer as a missing vote and keep every item that at least five models answered, and every answering model is wrong on 12 of 566 MMLU-Pro items, 2 of 191 GPQA Diamond, 4 of 143 GSM-Hard and 2 of 87 olympiad: 20 of 987, 2.0%, ten times the headline rate. Eighteen of the twenty are outside the audit, because the audit selected on seven finished answers, and one further MMLU-Pro item on which all seven finished and failed sits in an older run file outside these tables and was not read either. If those behave like the 45 the audit did read, most of them are the benchmark's fault; nothing here says so yet. The honest statement is narrower than the second version's: on the problems these models finish, joint failure after the audit is one item in 733; on the problems that make two of them run out of tokens it is about 2%, unaudited; and the question the co-failure literature asks cannot be answered on public benchmarks of this kind without auditing the benchmark first and without a token budget the hard items fit in. What survives either way is the correlation of the errors that remain, which is far from independence:

Seven families are 2.1–2.3 effective voters on the three larger layers (3.2 on the 29 olympiad items, where five models are at or near 1.0 and φ is defined on ten of the 21 pairs). The tetrachoric ρ is the mean over the pairs on which the cos-π estimate is defined; on the olympiad one pair survives and it is not shown. The one-factor copula² at that ρ over-predicts the all-wrong rate by a factor of five to infinity, which is what one expects when the estimate of ρ is driven by the few remaining shared errors; it is reported for continuity with the first version, not as a fit. Family still does not buy a vote anything: among the 35 triples of models the plurality's gain over its best member is negative on average on every layer (−1.9, −3.7, −0.4 and −1.3 points), and on the two multiple-choice layers it is positively correlated with the triple's mean φ (Spearman +0.49 and +0.41). That is the wrong sign for any story in which diversity is what a vote buys, but it is also the sign that φ's mechanical dependence on the spread of accuracies within a triple would produce, which is the confound the previous note on this site was about; the triples share models, and this corpus does not separate the two readings.

Self-consistency, the plurality over k draws of one model at temperature 1, is the within-model version of the same question, and it has the same shape with one exception.

Items are those with k complete, untruncated draws, the first k by index; plurality uses answer identity and breaks ties by draw order, which matters at k = 8 and on one MMLU-Pro item at k = 16. GLM is omitted: on GPQA and the olympiad it truncated most of its draws, and the 26 and 19 items that survive are the easy ones, so its rows would overstate it. Haiku's olympiad file holds two batches of eight draws under the same indices, 61 of the pairs disagreeing, and the later batch is the one counted; the second version of this note averaged both into its mean draw. The Kish prediction is the exact majority of n_eff independent votes at the mean-draw accuracy, which for n_eff ≤ 2 is the mean-draw accuracy itself.

Draws of one model correlate with each other at φ = 0.45–0.75 and are worth 1.3–2.1 independent draws. Over the mean single draw, the plurality gains nothing to two points on MMLU-Pro and GSM-Hard, four to seven on GPQA and fifteen on the olympiad for DeepSeek; the exception is real and it is the layers where φ is lowest. The "0 of k" column, the share of problems the model never solves at this temperature, is 0–9%, and it bounds pass@k and plurality alike. A two-state fit, in which each item is either unsolvable or solvable at one fixed rate, over-predicts the plurality by 3–9 points on each of the six sixteen-draw rows: items are not binary, and the middle of the solve-rate distribution is where the plurality is lost.

The last column is the third estimator this note has tried for the same quantity, and the lesson is about the estimator. Kish's n_eff was used in the earlier notes as a ceiling on what a vote could achieve; the first version of this note called it a floor, on an estimator that rounded n_eff to 2 and so predicted p²; the second version used a normal approximation and reported that the Kish majority over-predicted the plurality where φ was high and under-predicted it where φ was low. That pattern was the approximation. At n below 3 the normal majority is off by up to four points in a direction set by p, not by φ, and the high-φ rows are also the high-p rows. With the exact binomial, a majority of 1.3–2.1 independent votes is the single-draw accuracy for n_eff ≤ 2 and barely more above it, so Kish predicts that resampling buys nothing, and the observed plurality is above that prediction on six of eight rows, by one to fourteen points, most where φ is lowest. A design-effect formula built for the variance of a survey mean does not predict a plurality of correlated binary votes at these n_eff; it is a descriptive summary of how much independence the draws bought, and the two hypotheses in the literature that bet on its direction get no verdict from it (Section 17).

If temperature cannot buy independence, perhaps a prompt can: the "many minds" literature gives each copy a persona and reports gains. DeepSeek answered the same items four times under each of four one-sentence roles, a skeptical reviewer, a formal mathematician, a patient teacher and a pragmatic engineer, and the question is whether four roles are less correlated with each other than four plain draws.

The role correlation uses one draw per role; the plain correlation uses four draws of the same prompt. GPQA is absent because the role runs hit their spend cap before reaching it, and the olympiad row is thirteen items. Four roles are 1.2–1.5 independent answers, fewer than four draws on every layer, and cost one to six points of single-draw accuracy. The difference in φ, +0.06, +0.22 and +0.07, has a bootstrap interval over items that includes zero on every layer ([−0.82, +0.24], [−0.02, +0.49], [−0.08, +0.24]), so what the table says is that roles bought no measurable independence, not that they cost some; the roles' plurality is level with or above the plain one on two of three layers. On what 13–59 items can see, a persona is a temperature setting with a hat on. This is consistent with, and no stronger than, a one-model result from an earlier bridge test on this site.

The previous note found on public traces of three copies of one model that debate is not sampling: it moves the answers between rounds, and weak models escape unanimous error at rates a sampling theory puts at zero. Here the agents are strong, the families are mixed, and the texts are ours. One fact about the texts governs everything below. Six agents read each other's round-0 answers; two of them, GLM and Qwen, had been cut off by their token cap before writing one on 86 of the 1,350 round-0 rows, and what their peers saw in those cases was an empty answer. This version counts such a row as absent. The sub-section at the end of this section shows the same tables with it counted as wrong, which is what the first two versions of this note did without noticing.

225 items with a row for all six agents in every round, after the exclusions. Unanimity is among the agents that answered; φ is over the items each pair answered; ICC is over the items with six answers in that round (162, 185, 203 and 207). The five-agent recount without Mistral on 226 items gives .938 → .973, 9 fixed, 1 broken. "Fixed" and "broken" are cumulative against round 0.

Taken alone the first three columns are a success: the majority gains 4.9 points, every agent's own accuracy gains 9 on the rows it answers, eleven items are fixed and none is broken, and the gain plateaus at round 2. The remaining columns are the price and the mechanism. Unanimity rises from 152 items to 219; the panel goes from 2.2 effective voters to 1.2 within two rounds; the cut-off rows fall from 86 to 21, because an agent that has read a peer's answer can write its own without running out of tokens; and the two columns that were empty or nearly so at round 0 fill up. Before anyone spoke there was no item on which the answering agents were unanimously wrong, and one on which all of them were wrong. After three rounds there are four of each. On all four the wrong answer already held the round-0 plurality and one or two agents held the right one; debate talked the minority round. None of the four was in the audit (Section 5), and two of them are GSM-Hard items whose gold has the look of a program's partial substitution, so "unanimously wrong" here means unanimous against a label the note has not been able to check. If the four behave like the all-wrong items the audit did read, debate was manufacturing agreement with the truth against the benchmark; the data cannot yet say.

Where the eleven fixes come from is the question the stored rows answer. Split the 225 items by how many of the agents that answered at round 0 were right:

The one all-wrong item is the olympiad problem on which the four agents that answered were wrong and two were cut off; three agents converge on the right answer, which none of them had written. Five of the nine items on which exactly one answering agent was right end correct, twelve of the fourteen with two, and all twenty-one with three. But the strata are not what they look like: of the 24 items in the zero-, one- and two-correct strata, 21 had at least one agent cut off at round 0, so "one of six" is mostly "one of four". Ten of the eleven fixes are on items with a cut-off round-0 row. On the 162 items on which all six agents answered at round 0, the majority goes from 0.975 to 0.981: one fix. Seven of the eleven fixes are items on which the right answer was tied for the round-0 plurality and lost the tie-break to an earlier agent; with ties split, the round-0 vote is 0.934 and the debate gain is 3.9 points. A difficulty split says the same from the other side: using DeepSeek's own solve rate over its sixteen draws as the item's difficulty, eight of the eleven fixes are on problems DeepSeek solves less than a quarter of the time, two on problems between a quarter and a half, one above, and none above three quarters, where the round-0 majority was already right on 168 of 169. Debate is a procedure by which a correct minority wins, and, on this corpus, by which an agent that ran out of tokens gets to answer; on these items it did little else.

The texts also say how the price is paid. For every agent that was correct at some round, count the answering peers who held a different answer and look at the next round:

Against one dissenter a correct agent is safe; against three or more it folds more often than not, and the cases are few: 28 with three answering dissenters, 6 with four, 6 with five. The second version of this note ranked the families by "spine" from 35 alone-against-five cases. Twenty-two of those 35 had either a cut-off peer counted as a dissenter or a peer whose answer differed only in format; with both corrected there are six genuine cases, and the lone correct agent kept its answer in one of them. No ranking survives that, and the sentence that matched it to the judge ranking of Section 13 is withdrawn.

Per layer:

Read against the best agent that finished every item, debate wins on the two layers where it starts least correlated and loses on the two where it starts most correlated, which is what the second version of this note reported. Read against the best agent on the items it answered, it wins nowhere. GLM and Qwen, the strongest models on GPQA Diamond and the olympiad, are at 0.97–1.00 on the items they finished and were cut off on the rest, and the low round-0 correlation on those two layers is in part the same thing seen from the other side: the items the capped models could not finish are the ones the others get wrong. A low starting correlation is still a precondition for debate to pay rather than something it produces, and debate consumes it, taking the ICC to 0.92–0.94 on MMLU-Pro and GSM-Hard within three rounds and, on the other two layers, to a value that cannot be computed because by round 3 almost every answering agent is right on almost every item. The one place in this corpus where plain voting beats the best panel member, GPQA Diamond at round 0, is the place where the best member that answered every item is Sonnet at 0.893 and the stronger members are missing on a third of the items: the vote pays because it can skip the missing votes.

Two practical numbers. Stopping each item at its first round of unanimity among the answering agents, and taking round 3 otherwise, gives majority accuracy 0.973, the same as three fixed rounds, at 0.42 rounds per item on average, because 152 items were unanimous before anyone spoke and 57 more after one round; counted the old way, with a cut-off agent breaking unanimity, it is 0.68 rounds. Three rounds of six agents cost 21,900 completion tokens per item on top of the first answers (1,439 per agent-turn in round 1, falling to 1,088 in round 3); the stopping rule spends about a seventh of that for the same accuracy, on the same items it was tuned on. Subsets of the panel tell the same story in miniature: the mean plurality over all pairs of agents is 0.842 at round 0 and 0.967 at round 3, over all triples 0.872 and 0.971, and the full six 0.924 and 0.973; debate makes a pair almost as good as a sextet, which is what one would expect if its job is to transmit the one or two correct answers in the room and to let a cut-off agent answer.

If a cut-off answer counts as wrong

The first two versions of this note scored a cut-off round-0 answer as a wrong vote. The debate itself was not affected, since a peer saw an empty answer and an empty answer was never counted in a plurality; the tables were. Every number that depends on an agent's correctness rather than on the vote moves: agent accuracy, φ, ICC, n_eff, unanimity, the all-wrong stratum, the flip table and the per-layer best single agent. The majority, the fixes and the stopping accuracy do not. For the record, the same tables under that policy; the companion repository prints all of them under "legacy".

Under this policy the second version read the per-layer table as "debate pays where the panel starts decorrelated", ranked the families by spine, and reported that every agent gained 13 points. The decorrelation on GPQA Diamond and the olympiad was in part 86 empty answers scored as wrong and correlated with nothing; the spine cases were in part empty peers scored as dissenters; the 13 points were in part agents finishing their answers. The reader can take either policy; the note takes the one under which an empty answer is empty, and the one under which an agent that could not finish alone scores 0 as a single-agent control.

Two things are entangled in a debate gain: the information each agent receives from its peers, and the fact that it was asked to think again. The public traces in the previous note had a self-only control on one model; here it is built in. Six copies of DeepSeek, seeded with six of its own temperature-1 draws as round 0, debate the items three ways. In the plain arm each copy sees the other five as in Section 10. In the self-only arm each copy sees only its own previous answer and reasoning, with the instruction to reconsider on its own. In the pressure arm the system prompt adds that the other agents are usually right and the copy should defer to the majority unless certain.

Cut-off rows are absent here too (66–77 of the roughly 5,600 rows in each arm). On the 230 items all three arms completed, the round-0 majority is 0.865 and the round-3 majorities are 0.900, 0.883 and 0.909, so the comparison in the text uses those: +3.5, +1.7 and +4.3 points. (The second version put the round-0 majority at 0.861 and the plain gain at 3.9; the round-0 answers are the same six draws in every arm, and 0.865 is what they vote.) Three readings. First, about half of the homogeneous gain, 1.7 points of 3.5, is self-revision with no peers at all, and that part costs nothing: the self-only arm leaves the correlation at 0.58 and takes the count of unanimously wrong items from 6 to 2. The other half, the information, takes that count to 22. The two arms are 11 fixes and 3 breaks against 8 and 4 on the same items, which is a contrast 230 items cannot pin down; "half" is a point estimate. Second, telling the copies to defer changes little they were not already doing; the pressure arm is a point above the plain arm with fewer wrong unanimities, within what 236 items can resolve. Third, and this is the number to carry from the previous note: with a strong model talking to itself, the all-wrong stratum is 0 for 15 in the plain arm and 1 for 13 and 1 for 15 in the other two, and the one escape is a GSM-Hard item whose gold is one of the two the audit has not reached, so it may be no escape at all. The 31–45% escape rates on weak models in the public traces were real and were specific to models whose first answers were under-elicited; a 2026 reasoning model that is wrong six times is wrong because the problem is hard for it, and six copies of it have almost nothing to tell each other.

Copies gain less than families, 3.5 against 4.9 points, with the caveat that the two debates start from different round-0 answers (temperature-1 draws against temperature-0 matrix answers) and different item sets, so the comparison is one contrast, not a controlled one. Section 10 offers two readings: the copies start at φ = 0.58 where the families start at 0.34, so there is less decorrelation to consume; and six copies under the same token cap cannot give each other a second chance to finish, which is where most of the mixed panel's fixes came from. The homogeneous majority at round 3, 0.894, is still 4 points under the ceiling for those copies, 0.936 (1 − 15/236, the "someone was right" rate across the six seed draws), and the mixed majority 0.973 is 2 points under its ceiling of 0.996 (1 − 1/225); neither box is left.

The generative objection to everything above is that a vote is the wrong aggregator: a model that reads six answers can write a seventh, and the mixture-of-agents literature reports exactly that. We gave DeepSeek, at temperature 0, the six round-3 answers of the mixed debate, each with the last 900 characters of that agent's reasoning, and the instruction to pick one or derive a different answer if all were wrong; separately, five of its own draws with their reasoning tails. The first version of this note called these inputs "transcripts"; they are final answers with excerpts, about 1,400 input tokens per item.

Over the debate answers the synthesiser does half a point worse than the plurality; over its own draws, 1.3 points better; in both cases it recovers nothing from the all-wrong stratum and writes a novel answer once in 462 tries. Two caveats on the design. The debate-answer test is weak as a test of mixture-of-agents, because by round 3 the answering agents agree on 219 of 225 items and there is little to mix; the own-draws test, which does start from diverse inputs, is the fairer one, and it says the same thing. An aggregator with the freedom to leave the box sits one point under its ceiling on the debate answers and six under it on its own draws, which is where a vote on the same inputs sits. On these models and these problems, reading the others' reasoning is not a way to get a new answer; it is a way to pick among the old ones.

Everything so far asks the group to produce the answer, and the box says the prize is in selection: on a contested item some model is right and the plurality may not be, and a verifier that finds the right one collects the distance between plurality and ceiling, which on the sixty-item subsets is 3–11 points at round 0. The earlier notes predicted that an LLM verifier cannot, because its errors are the solvers' errors. Part of that prediction holds; the part that fails is the useful part, and it is smaller than the first two versions of this note made it.

On the 77 items where the six debate agents' matrix answers disagreed by answer identity and at least one was correct, each of five models was shown the distinct candidates, each with a 700-character excerpt of one proposer's reasoning, in a shuffled anonymised order, and asked to pick one or say none. (The candidate set is the six debate agents' answers, so a judge that also solved the item may find its own answer among them; on 44 of the 77 items one or more agents had proposed nothing, having been cut off.) The contested items are 31 olympiad, 25 GPQA Diamond, 13 GSM-Hard and 8 MMLU-Pro. The second version counted 82, because six GSM-Hard items whose candidates differed only in formatting had been counted as disagreements.

A judge that named no candidate is a judge call that hit its token cap before the PICK: line: 20 of Qwen's 77 calls and 29 of GLM's, 8 of DeepSeek's, one each for the Claude models. The second version called these "a handful", scored them as wrong picks, and scored a judge's cut-off own answer as a wrong solve, which made GLM and Qwen look like judges that pick better than they solve. Like for like, on the items where a judge both named a candidate and had answered, two judges pick better than they solve, Haiku by 18 points and DeepSeek by 10, and three pick worse by 2–4: Sonnet, Qwen and GLM, the three strongest solvers on these items, lose by judging rather than answering. Over all contested items nobody beats the solvers' own plurality of 0.80; Qwen does on the 57 items where it managed to name a candidate.

The result is the panel, and it depends on how one counts a panel. Judges' picking errors correlate across judges at φ = 0.16 (tetrachoric 0.29) on the 76 items all five saw, a cut-off pick scored as wrong: five judges are 3.0 effective voters. The solvers' errors on the same items cannot be set beside that directly, because selecting items on disagreement deflates their correlation; on the whole layers it was φ = 0.3–0.5. The plurality of the five judges' picks is correct on 0.842 of the contested items when a tie at the top is broken by the first judge in the list (Haiku, then Sonnet), which is the rule the first version used without saying so. Ties are not rare: 13 of 76 items; and on 44 of the 76 the panel is two, three or four judges rather than five, because a cut-off judge is absent. The solvers' plurality under the same tie rule, first proposer wins, is 0.803, so the panel leads by 8 items to 5, an exact McNemar p of 0.58. If a tie counts as no answer, the panel is at 0.763 and the solvers at 0.750. If one asks only for a strict majority, more than half the judges on one candidate, the panel answers 56 of the 76 items and is right on 0.946 of those; on the other 20 it has nothing to say. The mean over all ten triples of judges is 0.807 with first-judge ties and 0.764 without; the mean single judge is 0.645. The second version compared the panel's 0.864 with a solvers' plurality computed under a different tie rule and read the difference as seven points; under the same rule it is four, and the 8-against-5 behind it is noise.

The honest accounting is over the whole item set, because a verifier is only ever used on top of a vote. Over the 235 items all five judges saw: the solvers' plurality alone is 0.928; plurality on unanimous items plus the panel on contested items is 0.940 with first-judge ties and 0.957 when a tied panel falls back to the solvers' plurality; the best single solver, Sonnet, is 0.945. The panel moves the vote by one to three points, to half a point below its best member or a point above it, depending on the tie rule. That is the whole of the comparator result: comparing candidates is a task on which these models' errors are not more correlated than when producing them, the gain on contested items is 8 items against 5, and it is at most the size of the gap between a vote and its best voter.

Two things qualify this further. On the 2 items where no candidate was correct, the judges said "none" once in 10 opportunities; a picker cannot abstain. And the result rests on 76 contested items from one corpus, 13 of them decided by a tie-break and 44 judged by a panel of fewer than five. It is the only positive number in this note, it is within noise, and it is the one that most needs a second dataset.

The same independence shows up in a cheaper form. Take DeepSeek's matrix answer on each of the 225 debate items, wrong on 17% of them, and ask how well two warning signals rank the wrong ones: the number of the other five families whose answer differs from it, and the number of DeepSeek's own temperature-1 draws that differ from it, the usual self-consistency signal. Peer disagreement, counted over the peers that answered and by answer identity, reaches AUROC 0.98; disagreement among three own draws, 0.89, among five, 0.91, among fifteen, 0.94. The comparison is not quite between "others" and "self": one strong peer alone, Sonnet, reaches 0.93, so part of the peer signal is that the peers are better solvers. What survives the matching is a gap of four points between five one-shot calls to other families and fifteen extra calls to oneself, with a bootstrap interval over items of [+0.01, +0.08]; the gap between the one strong peer and fifteen own draws is −0.01 and its interval spans zero. Five cheap calls to other families are at least as good an abstention signal as fifteen to oneself, for DeepSeek, the only target tested. This is the mechanism behind the panel in Section 13: a judge's "this candidate is right" is partly "this candidate agrees with what I would have said", and that signal is less correlated across families than the answers themselves.

The part of the prediction that holds is the other judging task. Shown one candidate at a time, with its excerpt, and asked only whether it is correct, with one planted wrong answer per item, taken from a wrong draw some model had made on that item in another stage and that no proposer had put on the candidate list, the same five judges behave more like solvers again.

The three OpenRouter judges have fewer verdicts for two reasons: the stage hit its spend cap at about 82% of the calls, and 38, 77 and 96 of the calls DeepSeek, Qwen and GLM did make were cut off before a verdict, so they ruled on 70%, 55% and 50% of the candidates Sonnet ruled on. Two things stand out. Sonnet is a good certifier, 6% false acceptance and one decoy in 65; the rest accept a quarter to a half of wrong candidates, and a planted wrong answer that no proposer had offered is accepted 12–26% of the time by the other four. And the dominant term is self-preference: every judge accepts its own wrong answer more often than someone else's, by a factor of 1.1 for DeepSeek, 2.8 for Haiku, 3.4 for GLM and Qwen, and 6.6 for Sonnet, on 5–35 own wrong answers each. The candidate was anonymised and shown with another proposer's excerpt when several models gave it, so this is not style recognition of the excerpt; whether it is recognition of the answer or re-derivation of the same error this corpus cannot say (Section 18), and "own" here means "equals the judge's own matrix answer", which is consistency rather than recognition.

A committee repairs this only at a price. The false acceptances correlate at φ = 0.38 across the five judges, 2.0 effective judges, between the solver number and the picking number. Over the 94 candidates all five ruled on (48 of them GSM-Hard, 20 of the 54 wrong ones planted decoys):

Unanimity lets through 1 of 54 wrong candidates, 2%, at the cost of 30% of the correct ones; four of five lets through 11% and keeps 88%. The first version of this note said that no threshold reached a 5% false-acceptance rate; six of the eight "wrong" candidates that a unanimous committee had accepted were right, and the gold was wrong. Put next to Section 13: the same five models, the same items, nearly the same candidates, and in one framing they are 3.0 judges whose strict majority is right 95% of the time where it exists, in the other they are 2.0 correlated solvers who wave through their own mistakes unless all five are asked. The difference is whether the judge compares or re-solves. The swarm can rank, a little; it can certify only when unanimous and at a steep recall cost; and it cannot abstain.

Every row carries its token counts, so the operators can be put on one scale. For DeepSeek, on the items of each layer that every operator completed, completion tokens (reasoning included) per item and the accuracy of each operator's output:

The item counts in the header are the items present in all four rows of a layer, which is why the one-answer accuracies differ from the matrix table and why GPQA has 39: DeepSeek's homogeneous debate completed 45 GPQA items and its sixteen untruncated draws 55. The copies' debate includes its six seed draws; the families' debate excludes round 0 because the matrix calls were made by other models. On MMLU-Pro and GSM-Hard, ten to thirty times the tokens buy two to five points or nothing; on the olympiad, sixteen draws buy nine points and six copies in debate twelve for a third fewer tokens, which is the one place in this corpus where spending on sampling is clearly worth it (four and five net items of 43). The mixed-family debate's GPQA and olympiad accuracies are not DeepSeek's: they are the panel's, with Sonnet and Qwen in it. A cost-matched comparison of these operators against a single model given the same budget is the subject of the next note; the table is here so that the reader has the raw numbers.

Five things this does not say. It does not say that a swarm of agents working on different subtasks, with tools, memory and a plan, adds nothing; that is a different operator, its gains would come from context length and parallel time rather than from independent errors, and the right control for it, one agent doing the same decomposition sequentially at the same token budget, is not in this corpus. It does not say anything about base or non-reasoning models, where error correlation may be lower and voting may pay; every model here was allowed to think, though the Claude models at low effort barely did, and the previous note's weak models behaved differently. It does not say debate is useless: a 4.9-point gain over the round-0 vote with a stopping rule that costs a seventh of three rounds is a real engineering result, and the fact that most of it is a capped model finishing its answer is itself a result about what debate does. It does not say the comparator result is large or general: it is one to three points on 235 problems from one corpus, a tie-break away from zero on the contested ones, and within noise. And it does not say that strong models never fail together: it says that on four public benchmarks the cases where they appeared to were, about four times in five, the benchmark's fault on the problems the audit read, that twenty more such problems sit behind a token cap and have not been read, and that any measurement of joint failure on such data has to start with an audit and a budget the hard problems fit in.

The hypotheses the corpus was built for are bridges written by the discovery pipeline: each pairs a stated limitation of one paper with a result of another, and the prediction is the pipeline's, not either paper's. They were judged before any of this data existed, and the companion repository of the discovery pipeline holds all 140 with their three verdict passes against the first-version tables. The sixteen the tables answer, re-read by the author against the audited tables, by each hypothesis's own falsification criterion:

• Correlated judge panels (a bridge from CollabEval, arXiv:2603.00993, to Nine Judges, Two Effective Votes, arXiv:2605.29800). Predicted: discussion rounds raise φ and cut n_eff, and items that reach consensus after discussion are wrong more often than items unanimous before it. Supported: φ 0.34 → 0.85, n_eff 2.2 → 1.2, unanimous-and-wrong 0 of 152 → 4 of 219 (Section 10). Two caveats: these are solver debates, not judge panels, and the four items are not audited.
• Pre-debate independence predicts debate gain (OptAgent, arXiv:2510.18032, read against Effective Independence). Predicted: gain over the best single agent tracks the panel's pre-debate n_eff, and debate drives n_eff toward 1 whatever the topology. Not separable on this corpus. The two layers with low round-0 n_eff (2.8–2.9) are the two on which the capped models were cut off most; against the best agent that finished every item debate gains there (+8.9, +3.7) and loses on the high-φ layers, which is what the second version reported, but against the best agent on the items it answered it gains +0.3 and 0, and on the 162 items with six answers it gains one item (Section 10). That n_eff falls toward 1 is supported in every arm, and the self-only control of Section 11 shows that seeing peers is what drives it, for copies of one model. Four layers is four data points.
• Relay gains live inside the selection ceiling (WILC, arXiv:2607.29087, against the co-failure ceiling paper, arXiv:2606.27288). Predicted: iterative collaboration gains almost entirely on items where some worker was already right. Supported, with a smaller denominator than before: the gain sits in the one- and two-correct strata, mostly on items where a worker had been cut off; the all-wrong stratum has one item after the audit, and debate rescued it.
• Mixture-of-agents synthesis beats the ceiling (a bridge from arXiv:2605.01566 to arXiv:2606.27288). Predicted: an aggregator's surplus over voting comes from items where every proposer was wrong. Not supported: 0 of 16 and 0 of 4 (Section 12); the surplus is selection inside the ceiling.
• Role prompts decorrelate (Dynamic Role Assignment, arXiv:2601.17152, against the Blackwell-informativeness paper, arXiv:2605.06028). Predicted: role-diversified proposals have lower inter-agent error correlation than temperature samples. Not supported, and not refuted: φ is higher under roles on every layer, by 0.06–0.22, with bootstrap intervals that include zero (Section 9). A sibling hypothesis from the same bridge, that a reported debate gain is score-only selection and the subsequent debate erases diversity (against The Interaction Tax, arXiv:2608.23541), is also not supported: debate raised accuracy above the round-0 vote, and selection by a single judge did not beat the vote.
• Kish over-predicts the vote (two hypotheses pairing Kish 1965 with Boland 1989 and with arXiv:2607.23931). Predicted: n_eff = 1/φ overstates what a majority can reach. No verdict. With an exact estimator the Kish majority at n_eff ≤ 2 is the single-draw accuracy, which the observed plurality exceeds on six of eight rows; the hypothesis assumed the formula would over-state a vote, and on these rows it predicts no vote at all (Section 8). The first version reported the reverse of the hypothesis on an estimator that rounded n_eff to an integer, and the second a mixed verdict on a normal approximation whose error tracked accuracy rather than φ; both readings are withdrawn. The companion hypothesis from arXiv:2602.08003, that a probit one-factor bound beats the Kish prediction, has no table behind it in this corpus and gets no verdict either.
• A one-factor model under-predicts joint failure by 2–3× (a bridge from the Honest Quorum paper, arXiv:2607.16109, to Effective Independence; the 2–3× figure is the co-failure paper's 2.5×, arXiv:2606.27288). Insufficient after the audit: on the items all seven finished the observed all-wrong rate is one in 733 and the copula at the tetrachoric ρ over-predicts it; on the items with cut-off answers it is about 2% and unaudited (Section 7). There is no measured joint-failure tail on these benchmarks to compare a model against. The same bridge's claim that provider-family diversity does not make a certifying committee feasible is supported in part by Section 15: false endorsements correlate at 0.38, and only unanimity reaches a 5% bound, at 70% recall.
• Pooled independence ≲ 2 (CTTS, arXiv:2508.03333, against No Free Lunch). Predicted: a heterogeneous pool has about two effective members and selection stays within the union of candidates. Supported in part: n_eff 2.1–2.3 of 7 on the three larger layers, every aggregator inside the ceiling; the paper's own method was not run.
• Debate rounds add nothing beyond round-0 voting (TUMIX, arXiv:2510.01279, against the modal-ceiling paper, arXiv:2606.28661), the hypothesis the previous note tested on public traces. Not supported on the full item set, on a second dataset with different models: majority 0.924 → 0.973, 11 fixed against 0 broken. Supported on the 162 items on which every agent had answered at round 0, where the gain is one item. The verdict turns on whether a round that lets a cut-off agent answer counts as refinement. (TUMIX's own abstract reports gains from refinement; the "nothing" is the bridge's reading of its halting result.)
• Majority accuracy peaks before consensus (the Koopman-certification paper, arXiv:2608.05956, against Ladha 1992). Not supported: majority accuracy is non-decreasing to near-unanimity, with no interior peak (Section 10), within the resolution of four rounds.
• Debate gains when errors are already correlated (arXiv:2511.07784 against arXiv:2605.29800). Not supported, reversed in the per-layer table: the gain over the round-0 vote is largest where φ starts lowest, which is in line with that paper's own abstract. The reversal is not clean, because the low-φ layers are the layers with cut-off answers.
• Prompted deference does not control convergence (Thinking with Many Minds, arXiv:2501.02348, against BOUNDARY_SYNC, arXiv:2607.01600). Supported with two prompt levels: exposure drives convergence, the stated weight barely moves it (Section 11).

Each of these is one reading of one table, and each hypothesis was written about a specific system this corpus does not reproduce. They are listed as what a careful reader of these tables would conclude about those claims, not as replications of the papers.

• Labels. The audit covered 48 items, selected mostly on all seven models having finished and failed; it found the gold wrong or the item ill-posed on 43 of them, by the author's reading, and on 36–42 by an independent re-reading, depending on how the judgment calls are counted. Items on which some model agreed with a wrong gold label have not been audited, so per-model accuracies may still be biased; 18 items on which every answering model was wrong while another model was cut off, one all-seven-wrong item in an older run file, and the four items on which the debate ends unanimously wrong have not been audited either, and are listed in the audit file. The auditor is one of the audited models and the new gold agrees with its own first answer on every relabel; eight items were checked by hand, the three GPQA Diamond items by hand only. GSM-Hard's gold is produced by a program that substitutes large numbers into GSM8K; eleven of its 150 matrix items had the substitution applied to some quantities and not others (six of them among the sixty), and six more are not well-posed after substitution (three among the sixty).
• Truncation. The four OpenRouter models ran under an 8,192-token completion cap and two of them hit it on a third of GPQA Diamond and half of the olympiad problems. The matrix tables are on the items every model finished, which are the easier ones; the debate and judge tables count a cut-off row as absent; the first two versions counted it as wrong, and Section 10 shows what that changed. The cap is a property of this corpus, not of the models, and a re-run with a larger cap would move every number that depends on it.
• Size. Sixty items per layer, 225–236 in the debate stages, 76 contested items for the picking result, 54 wrong candidates for the certifying one. The strata are small: the all-wrong stratum of debate is one item, six agents were alone against five answering dissenters six times, the panel result turns on 13 tied items and is 8 against 5 on the discordant ones, and the homogeneous arms differ by 11 fixes and 3 breaks against 8 and 4.
• Models. Reasoning models of one vintage only, the Claude models at an uncontrolled temperature and low effort, so that two of the seven barely reasoned; the OpenRouter calls return no reasoning-token count, so "allowed to think" is a setting, not a measurement. The homogeneous debate, the personas, the synthesiser and the error-detection result rest on one family, DeepSeek V3.2. Opus is in the matrix but not in the debate or judge panels.
• Controls. Best single is an in-sample maximum over six or seven agents, which flatters it by about a point on the small layers; on GPQA Diamond and GSM-Hard the one-item margins of Section 7 are inside that. Round 0 of the debate is the temperature-0 matrix answer and the rounds are at temperature 1, as in the public traces of the previous note; the self-only arm of Section 11 is the control for the share of the round-1 jump that is re-elicitation, about half for copies of DeepSeek, and there is no self-only arm for the mixed-family debate.
• Mistral. Nine of the 240 debates were never run because Mistral had no matrix answer to seed them with, and one lost Mistral to a rate limit. The six-agent tables use the 225 complete items; the five-agent recount recovers one item and matches at round 3.
• Estimators. Tetrachoric correlations are the cos-π approximation, averaged over the pairs on which it is defined, which after the audit is few on the small layers; φ and ICC are exact, φ on the items each pair answered. The copula is a simulation at one ρ per layer and is no longer a fit to anything. n_eff is Kish's formula used descriptively; the Kish majority is an exact binomial interpolated between integers, the third estimator this note has used for it. The debate stopping rule was evaluated on the items it was chosen on. Plurality ties go to the first agent in a fixed order; seven of the eleven debate fixes are round-0 ties.
• Judging design. Candidates were anonymised but shown with a reasoning excerpt; a judge could recognise its own style. In the pick task a judge's own answer was among the candidates whenever it had solved the item; in the certify task "own" means the candidate equals the judge's matrix answer. The self-preference rates are therefore a mixture of style recognition, answer recognition and re-derivation, and this corpus cannot split them. The certify committee is 51% GSM-Hard candidates.
• Coverage. In the pick task GLM and Qwen named a candidate on 48 and 57 of 77 items; in the certify task the OpenRouter judges ruled on 50–70% of the candidates Sonnet ruled on; the role runs did not reach GPQA; GLM's hard-layer samples are mostly truncated; Haiku's olympiad samples were drawn twice under the same indices and the later batch is the one counted.
• Contamination. All four layers are public and predate the models; the olympiad problems are from 2024 and 2025 competitions. Nothing here measures memorisation.
• Pre-registration. None. The stages were designed to serve hypotheses written by other people, and the verdict passes that read those hypotheses against the first tables were done by a model; both are in the discovery repository, not in this note. The first revision was prompted by an external reading of the first version, the second by an external reading of the second, and each found errors the author had not; the reader should assume a third reading would find more.

Every table in this note, and about twenty more that it quotes from, is rebuilt from the stored answers by a single standard-library script in a few seconds with no network access, under both truncation policies. The dataset and that script will be released as tech-note-5 within about a month of this note: every row of every stage with the parsed answer, its correctness, the serving provider and the token counts, but not the model texts; GPQA Diamond answers replaced by random per-option tokens, which preserve answer identity but not the option letter, and no question text or gold label stored, per that dataset's terms (the second version's hashes were an unsalted digest of the letter and could be inverted; they have been replaced). The release also carries label-audit.json, the item-by-item record of the audit with the reason for every relabel and exclusion and the lists of unaudited items, and regrade_all.py, the script that applied it, which imports the grader from the collection repository and will run once that is released. Figures are drawn inline from the same numbers. The collection scripts, graders, and the discovery pipeline that proposed the hypotheses and read the verdicts live in a separate repository that will accompany a later note on the pipeline itself. Until the release, the numbers here are the author's word; after it, they are a command.

Archived with a DOI: 10.5281/zenodo.23229043 (version 1; this revision will be deposited as a new version of the same record). Cite as: Latipov, A. (2026). Where the Swarm Pays. Zenodo. https://doi.org/10.5281/zenodo.23229043

1. The earlier notes: Effective Independence (31 August 2026), The Control Ate the Signal (2 October 2026) and Debate Is Not Sampling (4 October 2026). The pipeline follows Swanson's literature-based discovery: paper A states an open question or limitation, paper B reports a result that bears on it, A and B do not cite each other and are otherwise dissimilar, and the pair is a candidate hypothesis. Candidates were sieved by embedding similarity between A's stated needs and B's stated offers and scored 0–10 by an LLM judge (Claude Opus 5.5, with a second opinion from Claude Fable 5.1 on the strongest). Papers and foundations, from Condorcet and Kish to the 2025–26 preprints, were summarised into fixed-schema cards first; the cards, not the papers, were what the judge read. ↩
2. One-factor Gaussian copula: model i is correct on an item when z_i = √ρ·F + √(1−ρ)·e_i < Φ⁻¹(acc_i), with F shared across models within the item and e_i private, both standard normal. Simulated with 20,000 items per layer. At the tetrachoric ρ the all-wrong rate is 0.011 / 0.003 / 0.004 against observed 0.002 / 0.000 / 0.000 (MMLU-Pro / GPQA / GSM-Hard), and the simulated majority-correct rate is 0.941 / 0.957 / 0.975 against observed 0.946 / 0.967 / 0.984. The model has no answer identities, so its "plurality" is a majority of correct votes; the observed figures in this footnote are on the same definition and differ slightly from the answer-identity plurality of Section 7. In the first version of this note the same model matched all-wrong rates of 1–10% to within a few tenths of a point; those rates were label noise, and the match was a fit to noise. ↩

← notes · github.com/Alatip