Where the Swarm Pays

Nothing in the framing of this note is new; the measurements are. The premises come from four places, and it is worth naming them because the vocabulary of the LLM-swarm literature borrows from all four without always saying so.

Juries. Condorcet's jury theorem (1785) says that a majority of independent voters, each right with probability above one half, is right with probability that goes to one as the jury grows. Boland (1989) and Ladha (1992) worked out what happens when the votes are correlated: the majority's accuracy no longer goes to one but to a plateau set by the correlation, and above a threshold correlation adding voters adds nothing. The theorem is about a binary choice; its extension to a plurality over many candidate answers is List and Goodin (2001). Every "N agents beat one" claim is a Condorcet claim, and every ceiling in this note is a Boland–Ladha plateau.

Surveys. Kish (1965) gave the design effect for a cluster sample: n correlated observations with intraclass correlation ρ carry the information of n/[1+(n−1)ρ] independent ones, which is bounded by 1/ρ however large n grows. The first note on this site applied this to LLM resamples as the effective number of independent voters, n_eff. It is a formula about the variance of a mean; Section 8 shows what it does and does not say about a majority of binary votes.

Latent correlation. Two agents' correctness is a 2×2 table, and the Pearson correlation of two 0/1 variables, φ, cannot reach 1 unless their accuracies are equal. Pearson (1900) defined the tetrachoric correlation as the correlation of two latent normal variables that would produce the table, and gave the cos-π approximation to it; Fisher's intraclass correlation (1925), in the one-way analysis-of-variance form of Shrout and Fleiss (1979), is the same idea for a panel. The one-factor Gaussian model used in Section 7 is Vasicek's (1987) single-factor model of correlated defaults, in which every borrower loads on one latent economic state; here every model would load on one latent item difficulty.

LLM ensembles. Self-consistency, the plurality over sampled chains of thought, is Wang et al. (2022, arXiv:2203.11171). Multi-agent debate in the form used here, agents that read each other's answers and revise, is Du et al. (2023, arXiv:2305.14325). Mixture-of-agents, a model that reads several answers and writes one, is Wang et al. (2024, arXiv:2406.04692). The LLM-as-a-judge paradigm and its self-preference bias are Zheng et al. (2023, arXiv:2306.05685) and Panickssery et al. (2024, arXiv:2404.13076). The 2026 papers whose claims this corpus was built to test are named in Section 17; three of them frame the whole note: the co-failure ceiling (arXiv:2606.27288), which bounds routing, voting and mixing by the rate at which all members fail together; the modal and correlation ceilings of test-time scaling (arXiv:2606.28661), which the previous note tested on the public debate traces of arXiv:2609.35279 (the DebateLedger release); and Nine Judges, Two Effective Votes (arXiv:2605.29800), which found the correlation structure of judge panels that Section 13 finds again in a different task.

The three earlier notes on this site¹ established, on public data, that LLM pools are one to two effective voters, that diversity metrics do not predict vote gain once accuracy is controlled, and that debate raises correlation while escaping unanimous error on weak models. This note puts all of that on one corpus with the same items for every operator. It also found, on the way, that the co-failure rate those premises are built around cannot be measured on these benchmarks without first auditing the benchmarks.

The hypotheses in the swarm literature can be sorted into eight questions; this is one such sorting, fixed as axes before the corpus was built, and it is a working taxonomy rather than a claim about the field. The note covers five of them.

A. Parallel answers and aggregation

Does pooling independent answers add intelligence? Sections 7–8.

B. Debate

Is a multi-round exchange a different operator from sampling, and what does it transmit? Sections 10–11, with synthesis in Section 12.

C. Selection and verification

Can a verifier reach the ceiling that voting cannot, and how independent is it from the solvers? Sections 13–15.

D. Sources of diversity

Can independence be bought with families, roles, prompts or temperature? Sections 7–9.

G. Cost at equal budget

What does each operator cost in tokens, and when does it pay against one model given the same budget? The token side is in Section 16; the cost-matched comparison is the next note.

Three axes are not in this note. E, iterated chains: swarms in time, self-training and iterated in-context learning, and whether they converge to the prior. F, organisation with tools: agents that decompose a task, take roles, keep memory and call code or search, which is what industrial agent systems do; the corpus has no decomposition and no tools. H, theory as bound: which of the classical results above is tight on which data; the Kish check in Section 8 is a first instalment, not a treatment.

A literature-based-discovery pipeline over about a thousand agent-swarm papers paired one paper's stated limitation with another paper's result wherever the two did not cite each other, scored each pair with an LLM judge, and produced 947 candidate hypotheses with a score of 4 or more out of 10, 140 of them at 6 or more. A handful were tested the usual way, one bridge at a time, two of them in the previous notes. Reading the rest made the economics obvious: most ask for one of five or six measurements on one kind of data, a correctness matrix across model families, resamples of one model, multi-round debate traces with every intermediate answer kept, a judge panel, and the token count of each call. A bridge test that costs ten dollars is not worth running; a corpus that answers forty bridges for fifty dollars is.

So the corpus was designed from the hypotheses, built in two days, and each of the 140 was then read against its tables by a model instructed to say "supported", "not supported", "mixed" or "the corpus does not measure this". After three passes, each adding the cuts the previous pass had asked for, 27 hypotheses had an answer and 113 asked for things the corpus does not contain. Section 17 re-reads sixteen of the 27 against the audited tables. The rest of this note is the measurements themselves, with the definitions first because half the disagreements in this literature are about what the words mean.

Item, layer, correct

An item is one problem with one gold answer. A layer is a source dataset. A model's answer is correct or not; truncated answers (the model hit its token limit before writing a final line) are missing, excluded from every count, never counted as wrong.

Answer identity

Two answers are the same answer when they are equal after normalisation: numbers as numbers (24, 24.00, $24 and 5,072,530 are one answer each), LaTeX fractions and roots in one form. Every plurality and unanimity count in this note uses this identity. The first version of the note compared raw strings, which split a few votes by formatting.

Best single

The highest accuracy among the agents on the same items. The standard control in the ensemble literature. It is the floor of the box: any aggregation that lands below it has cost accuracy.

Plurality

The answer given by the largest number of agents, ties broken by first occurrence, as in self-consistency. "Majority" is used for the same thing when the agents are a debate panel. Plurality is computed on answer identity, so two agents who are both wrong in different ways do not vote together.

Ceiling

The share of items on which at least one agent is correct, 1 − P(all wrong); the co-failure ceiling 1 − β of arXiv:2606.27288, the "oracle" or "coverage" of the test-time-scaling literature. No scheme that chooses among the agents' answers can exceed it; a scheme that writes a new answer can. The box between best single and ceiling is where every answer-side result in this note lives. After the label audit the ceiling on these layers is 0.98–1.00.

Unanimous, unanimous-and-wrong

All agents gave the same answer; that answer is wrong. A unanimously wrong item is invisible to any rule that uses agreement as a confidence signal. Before the audit, every unanimously wrong matrix item turned out to have a wrong or ambiguous gold label.

Fixed, broken

In a debate, an item whose plurality was wrong at round 0 and correct at the last round, and the reverse; arXiv:2609.35279 calls them corrections and collapses.

φ

The Pearson correlation between two agents' 0/1 correctness vectors over items, averaged over all pairs (the phi coefficient, Yule and Pearson). It is exact but bounded away from 1 when the two accuracies differ, and it understates the correlation of the latent errors.

Tetrachoric ρ

The correlation of two latent normal variables that would produce the observed 2×2 table of correct/incorrect. Pearson (1900); we use his cos-π approximation, which is biased upward when accuracies are high and undefined when a cell is empty; after the audit several layers have empty cells, and the estimate is reported only where it exists.

n_eff

Kish's (1965) effective sample size, n/[1+(n−1)φ], as used in Effective Independence. A design-effect formula from survey sampling, used here as a descriptive summary of how much independence n agents bought; Section 8 compares the majority it predicts with the majority observed.

ICC

Fisher's intraclass correlation in the one-way ANOVA form ICC(1) of Shrout and Fleiss (1979), of correctness within an item, the share of the variance of "this agent was correct on this item" that is explained by the item. Numerically close to φ here; reported because the debate note used it.

Contested item

Introduced here. For the judge panel: the candidate answers disagree and at least one of them is correct. On a contested item a perfect selector scores 1 and the plurality scores less; it is the only kind of item on which a verifier can earn anything.

Pick vs certify

Introduced here as names; both tasks exist in the LLM-as-a-judge literature as pairwise/listwise and pointwise judging. Pick: see all distinct candidates with a reasoning excerpt each, choose one or say "none". Certify: see one candidate alone, say correct or incorrect.

AUROC

Area under the receiver operating characteristic curve: the probability that a randomly chosen error receives a higher warning score than a randomly chosen correct answer; 0.5 is no information, 1 is perfect.

Design. The unit of analysis is an item answered by several agents, and every stage of the corpus is a different way of producing several answers to the same item: by different models, by resampling one model, by rounds of debate, by role prompts, by judging. The stages were chosen from the list of measurements the 140 hypotheses asked for, in descending order of how many hypotheses each would serve, and each was given a spend cap before it ran. Nothing was added after seeing a result except the cuts described below.

Controls. For every operator the comparison is to the best single agent on the same items and to the ceiling on the same items; both are computed from the same rows, never from a separate run. Debate has two controls built in: the round-0 plurality, which is what voting would have done, and the self-only arm, in which copies revise without seeing peers, which is what re-elicitation alone does. Judges are compared to the solvers' plurality on the same contested items, to each other, and, over the whole item set, to the best single solver. All comparisons are within-item; no number in this note compares one set of items against another.

Statistics. Accuracies are plain proportions. φ is the mean pairwise Pearson correlation of 0/1 correctness; the tetrachoric estimate is the cos-π approximation; ICC is the one-way ANOVA estimator; n_eff is Kish's formula applied to φ. The Kish-predicted majority is the probability that a majority of n_eff independent draws at the mean accuracy is correct, by the normal approximation at non-integer n; the first version of this note rounded n_eff to an integer, which made the prediction p² whenever n_eff rounded to 2, and Section 8 has been rewritten accordingly. Intervals, where given, are Wilson 95% intervals on proportions; the one paired comparison that is called significant is an exact McNemar test on discordant items. There is no bootstrap, because the strata of interest are small enough that the reader can see the counts, and all of them are reported as counts. Spearman correlations over model triples are over 35 triples per layer.

Labels. After the first version of this note was written, every item on which all seven models (or all six debate agents) were scored wrong was re-solved: 45 items, each read and solved by Claude Opus 5.5 with explicit arithmetic, eight of them re-solved by hand as a check, with the three GPQA Diamond items among them checked by hand only. Items whose gold label was wrong were relabelled with the demonstrably correct answer (21 items: 9 MMLU-Pro, 11 GSM-Hard, 1 GPQA Diamond); items that were ill-posed or had two defensible answers were excluded from every table (22 items: 15 MMLU-Pro, 6 GSM-Hard, 1 GPQA Diamond); on 2 items the models were wrong and the gold stands. Two further olympiad items had been scored wrong because the matrix stage had graded expressions with the numeric grader (1/576 against \frac{1}{576}); every row of every stage was re-graded with the normalising grader, which changed 61 further rows in the matrix, all on HMMT expressions and most of them Claude answers written as plain fractions. The audit, with the reason for each decision, is label-audit.json in the companion repository, and the re-grading script makes no model calls. The audit covers only items on which every model failed; items on which some model matched a wrong gold label have not been audited, so individual accuracies may still carry label noise in both directions.

Exclusions. Truncated answers are missing, not wrong, in every stage; items are included in a stage's table only if every agent of that stage answered them without truncation, which is why item counts differ between tables and are printed in each. Saturated layers were dropped before any aggregation was computed, by the criterion that every model exceeded 0.95. Mistral's ten incomplete debates are excluded from the six-agent tables and recounted without Mistral in the companion tables. The 22 excluded items are dropped everywhere.

Post-hoc cuts. The first reading of the hypotheses against the tables asked for cuts the first tables did not have: the all-wrong stratum of debate, the transition table by round-0 correct count, the flip table, the copula and two-state fits, the judge committee thresholds, the error-detection AUROC. These were computed from the existing rows, with no new calls. The three homogeneous-debate arms, the certify stage and the synthesis stage were new runs made after the first reading, each with its own cap; they are the only data collected after a result had been seen. The label audit, the tie analysis of the judge panel and the whole-set accounting of Section 13 were added in this revision, after an external reading of the first version, again with no new calls.

Reading the hypotheses. The 140 hypotheses were read against the first-version tables by Claude Opus 5.5 at low effort, three times, with instructions to quote only numbers present in the tables and to return "insufficient" whenever the quantity was not measured. The verdicts in Section 17 are the author's re-reading of those sixteen hypotheses against the audited tables.

Items. 240 problems with a single checkable answer, sixty from each of four layers: MMLU-Pro (ten-option multiple choice, stratified over subjects), GPQA Diamond (four-option graduate science), GSM-Hard (GSM8K word problems with large numbers substituted by a program; Section 5 says what that did to its gold labels), and the 2024 and 2025 AIME plus the February 2025 HMMT (integer or short-expression answers; all public before the models' training cutoffs, so contamination is possible and unmeasured). A first pass over 1,576 items had included MATH-500, GSM8K, CommonsenseQA, StrategyQA and Knights & Knaves puzzles; every model scored above 0.95 on those and they were dropped from every table, because a layer on which nobody errs cannot say anything about shared errors. They remain in the matrix file. The seven-family matrix covers 485–588 MMLU-Pro items, 198 GPQA Diamond, 150 GSM-Hard and 90 olympiad; the debate, sampling and judging stages use the sixty per layer.

Models. Seven families, all current reasoning models in October 2026 and all allowed to think before answering, though not equally: Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 through the Claude Code CLI, where temperature is not controllable and the effort setting was low, so that Opus and Sonnet averaged 20–130 reasoning tokens per MMLU-Pro answer while Haiku averaged 2,000; and DeepSeek V3.2, GLM 5.3 Flash, Qwen 3.7 Plus and Mistral Large 2512 through OpenRouter, with the serving provider logged per call. Four further models answered subsets of the matrix and are not used here. Opus is in the matrix but not in the debate or judge panels; the panel was fixed at six agents when the stage was configured, and the design notes do not record why Opus was the one left out, so the reader should treat its absence as unexplained rather than principled.

Grading. Letters by exact match; numbers by numeric comparison with a relative tolerance of 10⁻⁶; expressions by a normaliser (fractions, roots, dollar signs, trailing approximations) plus an LLM arbiter (Opus, decisions cached) where the normaliser and the gold string still disagreed. Every answer was extracted from a required last line, FINAL: <answer>. The first version of this note had graded the matrix stage with the numeric grader alone, which is the grading failure described in Section 5.

Every row stores the parsed answer, its correctness, the provider that served the call and the token counts (prompt, completion, reasoning). The debate stages also store the full text of every turn and what each agent was shown. Mistral Large dropped out of 10 of the 240 debates on provider rate limits; after the label exclusions the six-agent tables use 225 complete items and a five-agent recount on 226 is in the companion repository. The whole corpus cost about $50 on OpenRouter, plus subscription calls for the Claude models.

The oldest question first: does the plurality of seven different models beat the best of them? On no layer does it. On MMLU-Pro, the one layer large enough to test the margin, the best model is right and the plurality wrong on 13 items and the reverse on 4, an exact McNemar p of 0.05; on the other three layers the plurality is behind by one item or tied.

Items are those all seven answered without truncation, after the 22 exclusions, which is why these layers are larger than the sixty-item debate subsets and the olympiad is smaller: long olympiad solutions truncate. Per-model accuracies are from the 0/1 matrix on 490/91/128/29 items; best, plurality and ceiling are from the answer-identity recount on 485/91/128/29, which drops a few rows with empty answers. These accuracies are 2–9 points higher than in the first version of this note, and the difference is the audit: the gold was wrong, not the models.

The last column is the surprise of this revision. The ceiling, the share of items on which someone was right, is 0.998 on MMLU-Pro and 1.000 on the other three layers: of 733 audited items, the seven models all fail on two. The first version of this note reported all-wrong rates of 1–10% and fitted a one-factor latent model to them; those rates were made of wrong gold labels, ill-posed questions and a grader that could not read a fraction. There is no joint-failure tail left to model on these layers, and the question the co-failure literature asks, how often strong models fail together, cannot be answered on public benchmarks of this kind without an audit of the benchmark first. What survives is the correlation of the errors that remain, which is still far from independence:

Seven families are 2.1–2.3 effective voters on the three larger layers (3.2 on the 29 olympiad items, where four models are at or near 1.0 and φ is barely defined). The one-factor copula² at the tetrachoric ρ now over-predicts the all-wrong rate by a factor of three to infinity, which is what one expects when the estimate of ρ is driven by the few remaining shared errors and the cos-π approximation is biased upward near saturation; it is reported for continuity with the first version, not as a fit. Family still does not buy a vote anything: among the 35 triples of models the plurality's gain over its best member is negative on average on every layer (−1.9, −3.7, −0.4 and −1.3 points), and on the two multiple-choice layers it is positively correlated with the triple's mean φ (Spearman +0.51 and +0.41), the wrong sign for any story in which diversity is what a vote buys. The previous note on this site found the same thing on public correctness matrices with a different control; here it holds inside one corpus with the same items for every model.

Self-consistency, the plurality over k draws of one model at temperature 1, is the within-model version of the same question, and it has the same shape with one exception.

Items are those with k complete, untruncated draws; plurality uses answer identity and breaks ties by draw order, which matters at k = 8. GLM is omitted: on GPQA and the olympiad it truncated most of its draws, and the 26 and 19 items that survive are the easy ones, so its rows would overstate it. The Kish prediction is the majority of n_eff independent draws at the mean-draw accuracy, by the normal approximation.

Sixteen draws correlate with each other at φ = 0.45–0.75 and are worth 1.3–2.1 independent draws. Over the mean single draw, the plurality gains nothing to two points on MMLU-Pro and GSM-Hard, four to seven on GPQA and fifteen on the olympiad for DeepSeek; the exception is real and it is the layers where φ is lowest. The "0 of k" column, the share of problems the model never solves at this temperature, is 0–9%, and it bounds pass@k and plurality alike. A two-state fit, in which each item is either unsolvable or solvable at one fixed rate, over-predicts the plurality by 3–9 points on every row: items are not binary, and the middle of the solve-rate distribution is where the plurality is lost.

The last column revises the earlier notes in both directions at once. Kish's n_eff was used there as a ceiling on what a vote could achieve; the first version of this note called it a floor, on the strength of an estimator that rounded n_eff to 2 and so predicted p². With the rounding removed the Kish majority is within 2–5 points of the observed plurality on six of eight rows: it over-predicts where φ is high (Haiku MMLU-Pro .976 against .931, DeepSeek GSM-Hard .947 against .911), because the design effect does not see the hard items a model never solves, and it under-predicts where φ is low (DeepSeek olympiad .800 against .898, GPQA .849 against .873), because a majority of binary votes gains more from moderate independence than the variance of a mean does. A design-effect formula built for survey means is a serviceable descriptive summary and a poor predictor of a majority of correlated binary votes in either direction; two of the literature's hypotheses had bet on one direction (Section 17).

If temperature cannot buy independence, perhaps a prompt can: the "many minds" literature gives each copy a persona and reports gains. DeepSeek answered the same items four times under each of four one-sentence roles, a skeptical reviewer, a formal mathematician, a patient teacher and a pragmatic engineer, and the question is whether four roles are less correlated with each other than four plain draws.

The role correlation uses one draw per role; the plain correlation uses four draws of the same prompt. GPQA is absent because the role runs hit their spend cap before reaching it, and the olympiad row is thirteen items. Four roles are 1.2–1.5 independent answers, fewer than four draws on every layer, and cost one to six points of single-draw accuracy. A persona is a temperature setting with a hat on. This replicates, on a second family and three layers, a one-model result from an earlier bridge test on this site.

The previous note found on public traces of three copies of one model that debate is not sampling: it moves the answers between rounds, and weak models escape unanimous error at rates a sampling theory puts at zero. Here the agents are strong, the families are mixed, and the texts are ours.

225 items with all six agents present in every round, after the exclusions; the five-agent recount without Mistral on 226 items gives .934 → .973, 10 fixed, 1 broken. "Fixed" and "broken" are cumulative against round 0.

Taken alone the first three columns are a success: the majority gains 5.3 points, every agent's own accuracy gains 13, twelve items are fixed and none is broken, and the gain plateaus at round 2. The remaining columns are the price and the mechanism. Unanimity rises from 133 items to 205; the panel goes from 2.1 effective voters to 1.4; and the two columns that were empty at round 0 fill up. Before anyone spoke there was no item on which all six were unanimously wrong and one on which all six were wrong at all. After three rounds there are four of each: debate did not inherit wrong consensus from the benchmark, it manufactured it, from items on which one or two agents had been wrong and talked the rest round.

Where the twelve fixes come from is the question the stored texts answer. Split the 225 items by how many agents were right at round 0:

The one all-wrong item, an olympiad problem, ends with a correct majority (three agents of six converged on the right answer none of them had written). Five of the nine items on which exactly one agent was right end correct, twelve of the fourteen with two, and all twenty-one with three. The gain lives in the stratum where the right answer was on the table and a minority held it. A difficulty split says the same from the other side: using DeepSeek's own solve rate over its sixteen draws as the item's difficulty, eight of the twelve fixes are on problems DeepSeek solves less than a quarter of the time, three on problems between a quarter and a half, one above, and none above three quarters, where the round-0 majority was already right on 168 of 169. Debate is a procedure by which a correct minority wins. It is a good procedure for that, and on these items it did little else.

The texts also say how the price is paid. For every agent that was correct at some round, count the peers who held a different answer and look at the next round:

Against one or two dissenters a correct agent is safe; against four or five it folds about a fifth of the time. The families differ in spine. Alone against five, Sonnet 5.5 kept its correct answer 13 times of 15, Mistral 4 of 4, Haiku 5 of 6, DeepSeek 3 of 5, GLM 2 of 3, Qwen 1 of 2. The 35 alone-against-five cases are few, but the ordering matches the pick-task ranking in Section 13, in which Sonnet is also the judge least swayed by its own prior answer.

Per layer the picture sorts itself by starting correlation:

Debate beats the best single agent only on the two layers where the agents start least correlated, and loses to it on the two where they start most correlated. A low starting correlation is a precondition for debate to pay, not something it produces; debate consumes it, taking the ICC to 0.92–0.94 on the two higher-φ layers within three rounds. (The olympiad's ICC falls because by round 3 almost every agent is right on almost every item, and a near-constant vector has no variance to share.) The best single agent on GPQA here is 0.893 because Opus and GLM, the strongest models on that layer in the matrix, are one absent and one weak at temperature 1; on that layer even the round-0 plurality beats the best panel member, which is the one place in this corpus where plain voting pays, and it pays because the strongest model is not in the room.

Two practical numbers. Stopping each item at its first unanimous round, and taking round 3 otherwise, gives majority accuracy 0.973, the same as three fixed rounds, at 0.72 rounds per item on average, because 133 items were unanimous before anyone spoke and 46 more after one round. Three rounds of six agents cost 21,900 completion tokens per item on top of the first answers (1,439 per agent-turn in round 1, falling to 1,088 in round 3); the stopping rule spends about a quarter of that for the same accuracy, on the same items it was tuned on. Subsets of the panel tell the same story in miniature: the mean plurality over all pairs of agents is 0.842 at round 0 and 0.967 at round 3, over all triples 0.871 and 0.971, and the full six 0.920 and 0.973; debate makes a pair almost as good as a sextet, which is what one would expect if its job is to transmit the one or two correct answers in the room.

Two things are entangled in a debate gain: the information each agent receives from its peers, and the fact that it was asked to think again. The public traces in the previous note had a self-only control on one model; here it is built in. Six copies of DeepSeek, seeded with six of its own temperature-1 draws as round 0, debate the items three ways. In the plain arm each copy sees the other five as in Section 10. In the self-only arm each copy sees only its own previous answer and reasoning, with the instruction to reconsider on its own. In the pressure arm the system prompt adds that the other agents are usually right and the copy should defer to the majority unless certain.

On the 230 items all three arms completed, the round-0 majority is 0.861 and the round-3 majorities are 0.900, 0.887 and 0.909, so the comparison in the text uses those. Three readings. First, two thirds of the homogeneous gain, 2.6 points of 3.9, is self-revision with no peers at all, and that part costs nothing: the self-only arm leaves the correlation at 0.57 and takes the count of unanimously wrong items from 5 to 2. The other third, the information, quadruples that count to 20. Second, telling the copies to defer changes little they were not already doing; the pressure arm is a point above the plain arm with fewer wrong unanimities, which is within what 236 items can resolve, and is what Section 10's flip table predicts for a model that already folds at 20% against a majority. Third, and this is the number to carry from the previous note: with a strong model talking to itself, the all-wrong stratum is 0 for 15 in every arm. The 31–45% escape rates on weak models in the public traces were real and were specific to models whose first answers were under-elicited; a 2026 reasoning model that is wrong six times is wrong because the problem is hard for it, and six copies of it have nothing to tell each other.

Copies gain less than families, 3.9 against 5.3 points, with the caveat that the two debates start from different round-0 answers (temperature-1 draws against temperature-0 matrix answers) and different item sets, so the comparison is one contrast, not a controlled one. The per-layer table of Section 10 offers the obvious reading: the copies start at φ = 0.58 where the families start at 0.37, so there is less decorrelation to consume. The homogeneous majority at round 3, 0.894, is still 4 points under the ceiling for those copies, 0.936 (1 − 15/236, the "someone was right" rate across the six seed draws), and the mixed majority 0.973 is 2 points under its ceiling of 0.996 (1 − 1/225); neither box is left.

The generative objection to everything above is that a vote is the wrong aggregator: a model that reads six answers can write a seventh, and the mixture-of-agents literature reports exactly that. We gave DeepSeek, at temperature 0, the six round-3 answers of the mixed debate, each with the last 900 characters of that agent's reasoning, and the instruction to pick one or derive a different answer if all were wrong; separately, five of its own draws with their reasoning tails. The first version of this note called these inputs "transcripts"; they are final answers with excerpts, about 1,400 input tokens per item.

Over the debate answers the synthesiser does half a point worse than the plurality; over its own draws, 1.3 points better; in both cases it recovers nothing from the all-wrong stratum and writes a novel answer once in 462 tries. Two caveats on the design. The debate-answer test is weak as a test of mixture-of-agents, because by round 3 the six answers agree on 205 of 225 items and there is little to mix; the own-draws test, which does start from diverse inputs, is the fairer one, and it says the same thing. An aggregator with the freedom to leave the box sits a point or two under its ceiling, where a good vote sits. On these models and these problems, reading the others' reasoning is not a way to get a new answer; it is a way to pick among the old ones.

Everything so far asks the group to produce the answer, and the box says the prize is in selection: on a contested item some model is right and the plurality may not be, and a verifier that finds the right one collects the distance between plurality and ceiling, which on the sixty-item subsets is 5–13 points at round 0. The earlier notes predicted that an LLM verifier cannot, because its errors are the solvers' errors. Part of that prediction holds; the part that fails is the useful part, and it is smaller than the first version of this note made it.

On the 82 items where the six debate agents' matrix answers disagreed, after merging answers that differ only in format, and at least one was correct, each of five models was shown the distinct candidates, each with a 700-character excerpt of one proposer's reasoning, in a shuffled anonymised order, and asked to pick one or say none. (The candidate set is the six debate agents' answers, so a judge that also solved the item may find its own answer among them.) The contested items are 31 olympiad, 25 GPQA Diamond, 19 GSM-Hard and 7 MMLU-Pro.

"Picks a wrong candidate" excludes picks that could not be parsed; Qwen and GLM failed to name a candidate on a handful of items each, which is why their wrong-pick rates are low while their pick accuracies are not. Four of the five judges pick better than they solve, Haiku by 16 points, and one picks worse: Sonnet, the strongest solver on these items, loses 5 points by judging rather than answering. Nobody but Sonnet beats the solvers' own plurality of 0.79, and Sonnet by two points on 81 items.

The result is the panel, and it depends on how one counts a panel. Judges' picking errors correlate across judges at φ = 0.19 (tetrachoric 0.33), against φ ≈ 0.4 for the solvers' errors on the same layers: five judges are 2.9 effective voters where six solvers were 2.1. The plurality of the five judges' picks is correct on 0.864 of the contested items when a tie at the top is broken by the first judge in the list (Haiku, then Sonnet), which is the rule the first version of this note used without saying so. Ties are not rare: 12 of 81 items. If a tie counts as no answer, the panel is at 0.790, level with the solvers. If one asks only for a strict majority, more than half the judges on one candidate, the panel answers 61 of the 81 items and is right on 0.951 of those; on the other 20 it has nothing to say. The mean over all ten triples of judges is 0.819 with first-judge ties and 0.779 without; the mean single judge is 0.667.

The honest accounting is over the whole item set, because a verifier is only ever used on top of a vote. Over the 235 items all five judges saw: the solvers' plurality alone is 0.923; plurality on unanimous items plus the panel on contested items is 0.945 with first-judge ties and 0.957 when a tied panel falls back to the solvers' plurality; the best single solver, Sonnet, is 0.945. The panel buys the vote two to three points, which brings a seven-model vote level with its best member or a point above it. That is the whole of the comparator result: comparing candidates is a task on which these models' errors are less correlated than when producing them, the gain is real on contested items, and it is the size of the gap between a vote and its best voter, not larger.

Two things qualify this further. On the 2 items where no candidate was correct, the judges said "none" once in 10 opportunities; a picker cannot abstain. And the result rests on 82 contested items from one corpus, 12 of them decided by a tie-break. It is the only positive number in this note and it is the one that most needs a second dataset.

The same independence shows up in a cheaper form. Take DeepSeek's matrix answer on each of the 225 debate items, wrong on 17% of them, and ask how well two warning signals rank the wrong ones: the number of the other five families whose answer differs from it, and the number of DeepSeek's own temperature-1 draws that differ from it, the usual self-consistency signal. Peer disagreement reaches AUROC 0.98; disagreement among three own draws, 0.89, among five, 0.91, among fifteen, 0.94. The comparison is not quite between "others" and "self": one strong peer alone, Sonnet, reaches 0.93, and the three weakest peers together 0.95, so part of the peer signal is that the peers are better solvers. What survives the matching is still a gap: five one-shot calls to other families beat fifteen extra calls to oneself as an abstention signal. This is the mechanism behind the panel in Section 13: a judge's "this candidate is right" is partly "this candidate agrees with what I would have said", and that signal is less correlated across families than the answers themselves.

The part of the prediction that holds is the other judging task. Shown one candidate at a time, with its excerpt, and asked only whether it is correct, with one planted wrong answer per item taken from some model's wrong draw on that item, the same five judges behave more like solvers again.

The three OpenRouter judges have fewer verdicts because that stage hit its spend cap at 82% coverage. Two things stand out. Sonnet is a good certifier, 6% false acceptance and one decoy in 65; the rest accept a quarter to a half of wrong candidates, and a planted wrong answer that no model proposed is accepted 12–26% of the time by the other four. And the dominant term is self-preference: every judge accepts its own wrong answer more often than someone else's, by a factor of 1.1 for DeepSeek, 2.8 for Haiku, 3.3 for GLM and Qwen, and 7 for Sonnet. That is not style recognition alone, since the candidate was anonymised and shown with another proposer's excerpt when several models gave it; it is that a judge asked "is this right?" re-derives the answer, and re-derives its own error.

A committee repairs this only at a price. The false acceptances correlate at φ = 0.37 across the five judges, 2.0 effective judges, between the solver number and the picking number. Over the 101 candidates all five ruled on (55 of them GSM-Hard, 21 of the 54 wrong ones planted decoys):

Unanimity lets through 1 of 54 wrong candidates, 2%, at the cost of a third of the correct ones; four of five lets through 11% and keeps 87%. The first version of this note said that no threshold reached a 5% false-acceptance rate; six of the eight "wrong" candidates that a unanimous committee had accepted were right, and the gold was wrong. Put next to Section 13: the same five models, the same items, the same candidates, and in one framing they are 2.9 independent judges whose strict majority is right 95% of the time, in the other they are 2.0 correlated solvers who wave through their own mistakes unless all five are asked. The difference is whether the judge compares or re-solves. The swarm can rank; it can certify only when unanimous and at a steep recall cost; and it cannot abstain.

Every row carries its token counts, so the operators can be put on one scale. For DeepSeek, on the items of each layer that every operator completed, completion tokens (reasoning included) per item and the accuracy of each operator's output:

The item counts in the header are the items present in all four rows of a layer, which is why the one-answer accuracies differ from the matrix table and why GPQA has 39: DeepSeek's homogeneous debate completed 45 GPQA items and its sixteen untruncated draws 55. The copies' debate includes its six seed draws; the families' debate excludes round 0 because the matrix calls were made by other models. On MMLU-Pro and GSM-Hard, ten to nineteen times the tokens buy two or three points or nothing; on the olympiad, sixteen draws buy nine points and six copies in debate twelve for a third fewer tokens, which is the one place in this corpus where spending on sampling is clearly worth it. The mixed-family debate's GPQA and olympiad accuracies are not DeepSeek's: they are the panel's, with Sonnet and Qwen in it. A cost-matched comparison of these operators against a single model given the same budget is the subject of the next note; the table is here so that the reader has the raw numbers.

Five things this does not say. It does not say that a swarm of agents working on different subtasks, with tools, memory and a plan, adds nothing; that is a different operator, its gains would come from context length and parallel time rather than from independent errors, and the right control for it, one agent doing the same decomposition sequentially at the same token budget, is not in this corpus. It does not say anything about base or non-reasoning models, where error correlation may be lower and voting may pay; every model here was allowed to think, though the Claude models at low effort barely did, and the previous note's weak models behaved differently. It does not say debate is useless: a 5.3-point gain with a stopping rule that costs a quarter of three rounds is a real engineering result, on the layers where the panel starts decorrelated. It does not say the comparator result is large or general: it is two to three points on 235 problems from one corpus, and a tie-break away from zero on the contested ones. And it does not say that strong models never fail together: it says that on four public benchmarks the cases where they appeared to were, forty-two times out of forty-five, the benchmark's fault, and that any measurement of joint failure on such data has to start with an audit.

The hypotheses the corpus was built for are bridges written by the discovery pipeline: each pairs a stated limitation of one paper with a result of another, and the prediction is the pipeline's, not either paper's. They were judged before any of this data existed, and the companion repository of the discovery pipeline holds all 140 with their three verdict passes against the first-version tables. The sixteen the tables answer, re-read by the author against the audited tables, by each hypothesis's own falsification criterion:

• Correlated judge panels (a bridge from CollabEval, arXiv:2603.00993, to Nine Judges, Two Effective Votes, arXiv:2605.29800). Predicted: discussion rounds raise φ and cut n_eff, and items that reach consensus after discussion are wrong more often than items unanimous before it. Supported: φ 0.37 → 0.66, n_eff 2.1 → 1.4, unanimous-and-wrong 0 of 133 → 4 of 205 (Section 10). The caveat is that these are solver debates, not judge panels.
• Pre-debate independence predicts debate gain (OptAgent, arXiv:2510.18032, read against Effective Independence). Predicted: gain over the best single agent tracks the panel's pre-debate n_eff, and debate drives n_eff toward 1 whatever the topology. Supported on four layers: n_eff 2.3–2.7 → gain, 1.7–1.9 → loss; the self-only control shows that seeing peers is what causes the collapse (Sections 10–11). Four layers is four data points.
• Relay gains live inside the selection ceiling (WILC, arXiv:2607.29087, against the co-failure ceiling paper, arXiv:2606.27288). Predicted: iterative collaboration gains almost entirely on items where some worker was already right. Supported, with a smaller denominator than before: the gain sits in the one- and two-correct strata; the all-wrong stratum has one item after the audit, and debate rescued it.
• Mixture-of-agents synthesis beats the ceiling (a bridge from arXiv:2605.01566 to arXiv:2606.27288). Predicted: an aggregator's surplus over voting comes from items where every proposer was wrong. Not supported: 0 of 16 and 0 of 4 (Section 12); the surplus is selection inside the ceiling.
• Role prompts decorrelate (Dynamic Role Assignment, arXiv:2601.17152, against the Blackwell-informativeness paper, arXiv:2605.06028). Predicted: role-diversified proposals have lower inter-agent error correlation than temperature samples. Not supported: higher on every layer (Section 9). A sibling hypothesis from the same bridge, that a reported debate gain is score-only selection and the subsequent debate erases diversity (against The Interaction Tax, arXiv:2608.23541), is also not supported: debate raised accuracy above the round-0 vote, and selection by a single judge did not beat the vote.
• Kish over-predicts the vote (two hypotheses pairing Kish 1965 with Boland 1989 and with arXiv:2607.23931). Predicted: n_eff = 1/φ overstates what a majority can reach. Mixed: with a correct estimator the Kish majority over-predicts the plurality by 2–5 points where φ is high and under-predicts it by 1–10 points where φ is low (Section 8). The first version of this note reported the reverse of the hypothesis on the strength of an estimator that rounded n_eff to an integer; that reading is withdrawn. The companion hypothesis from arXiv:2602.08003, that a probit one-factor bound beats the Kish prediction, is likewise mixed on the same rows.
• A one-factor model under-predicts joint failure by 2–3× (a bridge from the Honest Quorum paper, arXiv:2607.16109, to Effective Independence; the 2–3× figure is the co-failure paper's 2.5×, arXiv:2606.27288). Insufficient after the audit: the observed all-wrong rate is 0–0.4%, two items in 733, and the copula at the tetrachoric ρ over-predicts it (Section 7). There is no joint-failure tail on these benchmarks to compare a model against. The same bridge's claim that provider-family diversity does not make a certifying committee feasible is supported in part by Section 15: false endorsements correlate at 0.37, and only unanimity reaches a 5% bound, at 68% recall.
• Pooled independence ≲ 2 (CTTS, arXiv:2508.03333, against No Free Lunch). Predicted: a heterogeneous pool has about two effective members and selection stays within the union of candidates. Supported in part: n_eff 2.1–2.3 of 7 on the three larger layers, every aggregator inside the ceiling; the paper's own method was not run.
• Debate rounds add nothing beyond round-0 voting (TUMIX, arXiv:2510.01279, against the modal-ceiling paper, arXiv:2606.28661), the hypothesis the previous note tested on public traces. Not supported here either, on a second dataset with different models: majority 0.920 → 0.973, 12 fixed against 0 broken. (TUMIX's own abstract reports gains from refinement; the "nothing" is the bridge's reading of its halting result.)
• Majority accuracy peaks before consensus (the Koopman-certification paper, arXiv:2608.05956, against Ladha 1992). Not supported: majority accuracy is non-decreasing to near-unanimity, with no interior peak (Section 10), within the resolution of four rounds.
• Debate gains when errors are already correlated (arXiv:2511.07784 against arXiv:2605.29800). Not supported, reversed: the gain is largest where φ starts lowest, which is also what that paper's abstract says.
• Prompted deference does not control convergence (Thinking with Many Minds, arXiv:2501.02348, against BOUNDARY_SYNC, arXiv:2607.01600). Supported with two prompt levels: exposure drives convergence, the stated weight barely moves it (Section 11).

Each of these is one reading of one table, and each hypothesis was written about a specific system this corpus does not reproduce. They are listed as what a careful reader of these tables would conclude about those claims, not as replications of the papers.

• Labels. The audit covered only items on which every model failed; it found the gold wrong or the item ill-posed on 40 of 45. Items on which some model agreed with a wrong gold label have not been audited, so per-model accuracies may still be biased, and the three GPQA Diamond items in the audit were checked by hand only. The audit itself was done by a model and spot-checked by the author; its decisions are listed item by item in the companion repository so that a reader can disagree with them. GSM-Hard's gold is produced by a program that substitutes large numbers into GSM8K; eleven of its sixty items had the substitution applied to some quantities and not others, and six more are not well-posed after substitution.
• Size. Sixty items per layer, 225–236 in the debate stages, 82 contested items for the picking result, 54 wrong candidates for the certifying one. Confidence intervals on the strata are wide: the 35 alone-against-five flips split six ways, the panel result turns on 12 tied items, and the all-wrong stratum of debate is one item.
• Models. Reasoning models of one vintage only, the Claude models at an uncontrolled temperature and low effort, so that two of the seven barely reasoned. The homogeneous debate, the personas, the synthesiser and the error-detection result rest on one family, DeepSeek V3.2. Opus is in the matrix but not in the debate or judge panels.
• Round 0. The matrix answers were made at temperature 0 and the debate rounds at temperature 1, as in the public traces of the previous note. The self-only arm of Section 11 is the control for the share of the round-1 jump that is re-elicitation, and it is about two thirds for copies of DeepSeek; there is no self-only arm for the mixed-family debate.
• Mistral dropped out of 10 of 240 debates on provider rate limits. The six-agent tables use the 225 complete items; the five-agent recount matches to within half a point.
• Estimators. Tetrachoric correlations are the cos-π approximation, biased upward at high accuracy and undefined after the audit on the olympiad layer; φ and ICC are exact. The copula is a simulation at one ρ per layer and is no longer a fit to anything. n_eff is Kish's formula used descriptively; the Kish majority is a normal approximation. The debate stopping rule was evaluated on the items it was chosen on.
• Judging design. Candidates were anonymised but shown with a reasoning excerpt; a judge could recognise its own style. In the pick task a judge's own answer was among the candidates whenever it had solved the item; in the certify task "own" means the candidate equals the judge's matrix answer. The self-preference rates are therefore a mixture of style recognition and re-derivation, and this corpus cannot split them. The certify committee is 55% GSM-Hard candidates.
• Coverage. The certify stage reached 82% of the OpenRouter judges' calls before its spend cap; the role runs did not reach GPQA; GLM's hard-layer samples are mostly truncated.
• Contamination. All four layers are public and predate the models; the olympiad problems are from 2024 and 2025 competitions. Nothing here measures memorisation.
• Pre-registration. None. The stages were designed to serve hypotheses written by other people, and the verdict passes that read those hypotheses against the first tables were done by a model; both are in the discovery repository, not in this note. The revision was prompted by an external reading of the first version.

Every table in this note, and about twenty more that it quotes from, is rebuilt from the stored answers by a single standard-library script in a few seconds with no network access. The dataset and that script will be released as tech-note-5 within about a month of this note, after a second reading: every row of every stage with the parsed answer, its correctness, the serving provider and the token counts, but not the model texts; GPQA Diamond answers replaced by hashes and no question text or gold label stored, per that dataset's terms. The release also carries label-audit.json, the item-by-item record of the audit with the reason for every relabel and exclusion, and regrade_all.py, the script that applied it. Figures are drawn inline from the same numbers. The collection scripts, graders, and the discovery pipeline that proposed the hypotheses and read the verdicts live in a separate repository that will accompany a later note on the pipeline itself. Until the release, the numbers here are the author's word; after it, they are a command.

Archived with a DOI: 10.5281/zenodo.23229043 (version 1; this revision will be deposited as version 2 of the same record). Cite as: Latipov, A. (2026). Where the Swarm Pays. Zenodo. https://doi.org/10.5281/zenodo.23229043

1. The earlier notes: Effective Independence (31 August 2026), The Control Ate the Signal (2 October 2026) and Debate Is Not Sampling (4 October 2026). The pipeline follows Swanson's literature-based discovery: paper A states an open question or limitation, paper B reports a result that bears on it, A and B do not cite each other and are otherwise dissimilar, and the pair is a candidate hypothesis. Candidates were sieved by embedding similarity between A's stated needs and B's stated offers and scored 0–10 by an LLM judge (Claude Opus 5.5, with a second opinion from Claude Fable 5.1 on the strongest). Papers and foundations, from Condorcet and Kish to the 2026 preprints, were summarised into fixed-schema cards first; the cards, not the papers, were what the judge read. ↩
2. One-factor Gaussian copula: model i is correct on an item when z_i = √ρ·F + √(1−ρ)·e_i < Φ⁻¹(acc_i), with F shared across models within the item and e_i private, both standard normal. Simulated with 20,000 items per layer. At the tetrachoric ρ the all-wrong rate is 0.012 / 0.002 / 0.003 against observed 0.004 / 0.000 / 0.000 (MMLU-Pro / GPQA / GSM-Hard), and the simulated majority-correct rate is 0.938 / 0.961 / 0.977 against observed 0.945 / 0.967 / 0.984. The model has no answer identities, so its "plurality" is a majority of correct votes; the observed figures in this footnote are on the same definition and differ slightly from the answer-identity plurality of Section 7. In the first version of this note the same model matched all-wrong rates of 1–10% to within a few tenths of a point; those rates were label noise, and the match was a fit to noise. ↩

← notes · github.com/Alatip