Where the Swarm Pays

Nothing in the framing of this note is new; the measurements are. The premises come from four places, and it is worth naming them because the vocabulary of the LLM-swarm literature borrows from all four without always saying so.

Juries. Condorcet's jury theorem (1785) says that a majority of independent voters, each right with probability above one half, is right with probability that goes to one as the jury grows. Boland (1989) and Ladha (1992) worked out what happens when the votes are correlated: the majority's accuracy no longer goes to one but to a plateau set by the correlation, and above a threshold correlation adding voters adds nothing. Every "N agents beat one" claim is a Condorcet claim, and every ceiling in this note is a Boland–Ladha plateau.

Surveys. Kish (1965) gave the design effect for a cluster sample: n correlated observations with intraclass correlation ρ carry the information of n/[1+(n−1)ρ] independent ones, which is bounded by 1/ρ however large n grows. The first note on this site applied this to LLM resamples as the effective number of independent voters, n_eff. It is a formula about the variance of a mean; Section 8 shows what it does and does not say about a majority of binary votes.

Latent correlation. Two agents' correctness is a 2×2 table, and the Pearson correlation of two 0/1 variables, φ, cannot reach 1 unless their accuracies are equal. Pearson (1900) defined the tetrachoric correlation as the correlation of two latent normal variables that would produce the table, and gave the cos-π approximation to it; Fisher's intraclass correlation (1925), in the one-way analysis-of-variance form of Shrout and Fleiss (1979), is the same idea for a panel. The one-factor Gaussian model that reproduces our all-wrong rates is Vasicek's (1987) single-factor model of correlated defaults, in which every borrower loads on one latent economic state; here every model loads on one latent item difficulty.

LLM ensembles. Self-consistency, the plurality over sampled chains of thought, is Wang et al. (2022, arXiv:2203.11171). Multi-agent debate in the form used here, agents that read each other's answers and revise, is Du et al. (2023, arXiv:2305.14325). Mixture-of-agents, a model that reads several answers and writes one, is Wang et al. (2024, arXiv:2406.04692). The LLM-as-a-judge paradigm and its self-preference bias are Zheng et al. (2023, arXiv:2306.05685) and Panickssery et al. (2024, arXiv:2404.13076). The 2026 papers whose claims this corpus was built to test are named in Section 17; three of them frame the whole note: the co-failure ceiling (arXiv:2606.27288), which bounds routing, voting and mixing by the rate at which all members fail together; the modal and correlation ceilings of test-time scaling (arXiv:2606.28661), which the previous note tested on public debate traces (DebateLedger, arXiv:2609.35279); and Nine Judges, Two Effective Votes (arXiv:2605.29800), which found the correlation structure of judge panels that Section 13 finds again in a different task.

The three earlier notes on this site¹ established, on public data, that LLM pools are one to two effective voters, that diversity metrics do not predict vote gain once accuracy is controlled, and that debate raises correlation while escaping unanimous error on weak models. This note puts all of that on one corpus with the same items for every operator.

The hypotheses in the swarm literature can be sorted into eight questions; this is one such sorting, fixed as axes before the corpus was built, and it is a working taxonomy rather than a claim about the field. The note covers five of them.

A. Parallel answers and aggregation

Does pooling independent answers add intelligence? Sections 7–8.

B. Debate

Is a multi-round exchange a different operator from sampling, and what does it transmit? Sections 10–11, with synthesis in Section 12.

C. Selection and verification

Can a verifier reach the ceiling that voting cannot, and how independent is it from the solvers? Sections 13–15.

D. Sources of diversity

Can independence be bought with families, roles, prompts or temperature? Sections 7–9.

G. Cost at equal budget

What does each operator cost in tokens, and when does it pay against one model given the same budget? The token side is in Section 16; the cost-matched comparison is the next note.

Three axes are not in this note. E, iterated chains: swarms in time, self-training and iterated in-context learning, and whether they converge to the prior. F, organisation with tools: agents that decompose a task, take roles, keep memory and call code or search, which is what industrial agent systems do; the corpus has no decomposition and no tools. H, theory as bound: which of the classical results above is tight on which data; the copula and Kish checks in Sections 7–8 are a first instalment, not a treatment.

A literature-based-discovery pipeline over about a thousand agent-swarm papers paired one paper's stated limitation with another paper's result wherever the two did not cite each other, scored each pair with an LLM judge, and produced 947 candidate hypotheses with a score of 4 or more out of 10, 140 of them at 6 or more. A handful were tested the usual way, one bridge at a time, two of them in the previous notes. Reading the rest made the economics obvious: most ask for one of five or six measurements on one kind of data, a correctness matrix across model families, resamples of one model, multi-round debate traces with every intermediate answer kept, a judge panel, and the token count of each call. A bridge test that costs ten dollars is not worth running; a corpus that answers forty bridges for fifty dollars is.

So the corpus was designed from the hypotheses, built in two days, and each of the 140 was then read against its tables by a model instructed to say "supported", "not supported", "mixed" or "the corpus does not measure this". After three passes, each adding the cuts the previous pass had asked for, 27 hypotheses had an answer and 113 asked for things the corpus does not contain. Section 17 names the ones that were answered. The rest of this note is the measurements themselves, with the definitions first because half the disagreements in this literature are about what the words mean.

Item, layer, correct

An item is one problem with one gold answer. A layer is a source dataset. A model's answer is correct or not; truncated answers (the model hit its token limit before writing a final line) are missing, excluded from every count, never counted as wrong.

Best single

The highest accuracy among the agents on the same items. The standard control in the ensemble literature. It is the floor of the box: any aggregation that lands below it has cost accuracy.

Plurality

The answer given by the largest number of agents, ties broken by first occurrence (Condorcet's rule, as used by self-consistency). "Majority" is used for the same thing when the agents are a debate panel. Plurality is computed on answer identity, so two agents who are both wrong in different ways do not vote together.

Ceiling

The share of items on which at least one agent is correct, 1 − P(all wrong); the co-failure ceiling 1 − β of arXiv:2606.27288, the "oracle" or "coverage" of the test-time-scaling literature. No scheme that chooses among the agents' answers can exceed it; a scheme that writes a new answer can. The box between best single and ceiling is where every answer-side result in this note lives.

Unanimous, unanimous-and-wrong

All agents gave the same answer; that answer is wrong. A unanimously wrong item is invisible to any rule that uses agreement as a confidence signal.

Fixed, broken

In a debate, an item whose plurality was wrong at round 0 and correct at the last round, and the reverse; DebateLedger (arXiv:2609.35279) calls them corrections and collapses.

φ

The Pearson correlation between two agents' 0/1 correctness vectors over items, averaged over all pairs (the phi coefficient, Yule and Pearson). It is exact but bounded away from 1 when the two accuracies differ, and it understates the correlation of the latent errors.

Tetrachoric ρ

The correlation of two latent normal variables that would produce the observed 2×2 table of correct/incorrect. It is the right quantity for a threshold model of shared difficulty. Pearson (1900); we use his cos-π approximation, which is biased upward when accuracies are high; read it as "the latent correlation is far above φ", not as a precise number.

n_eff

Kish's (1965) effective sample size, n/[1+(n−1)φ], as used in Effective Independence. It is a design-effect formula from survey sampling, used here as a descriptive summary of how much independence n agents bought; Section 8 shows it is a lower bound on what the vote achieves, not a ceiling.

ICC

Fisher's intraclass correlation in the one-way ANOVA form ICC(1) of Shrout and Fleiss (1979), of correctness within an item, the share of the variance of "this agent was correct on this item" that is explained by the item. Numerically close to φ here; reported because the debate note used it.

Contested item

Introduced here. For the judge panel: the candidate answers disagree and at least one of them is correct. On a contested item a perfect selector scores 1 and the plurality scores less; it is the only kind of item on which a verifier can earn anything.

Pick vs certify

Introduced here as names; both tasks exist in the LLM-as-a-judge literature as pairwise/listwise and pointwise judging. Pick: see all distinct candidates with a reasoning excerpt each, choose one or say "none". Certify: see one candidate alone, say correct or incorrect. The note's main positive result is that these are different tasks.

AUROC

Area under the receiver operating characteristic curve: the probability that a randomly chosen error receives a higher warning score than a randomly chosen correct answer; 0.5 is no information, 1 is perfect.

Design. The unit of analysis is an item answered by several agents, and every stage of the corpus is a different way of producing several answers to the same item: by different models, by resampling one model, by rounds of debate, by role prompts, by judging. The stages were chosen from the list of measurements the 140 hypotheses asked for, in descending order of how many hypotheses each would serve, and each was given a spend cap before it ran. Nothing was added after seeing a result except the cuts described below.

Controls. For every operator the comparison is to the best single agent on the same items and to the ceiling on the same items; both are computed from the same rows, never from a separate run. Debate has two controls built in: the round-0 plurality, which is what voting would have done, and the self-only arm, in which copies revise without seeing peers, which is what re-elicitation alone does. Judges are compared to the solvers' plurality on the same contested items and to each other. All comparisons are within-item; no number in this note compares one set of items against another.

Statistics. Accuracies are plain proportions. φ is the mean pairwise Pearson correlation of 0/1 correctness; the tetrachoric estimate is the cos-π approximation; ICC is the one-way ANOVA estimator; n_eff is Kish's formula applied to φ. Intervals, where given, are Wilson 95% intervals on proportions; there is no bootstrap and no significance testing, because the strata of interest are small enough that the reader can see the counts, and all of them are reported as counts. The copula check is a 20,000-item simulation at one ρ per layer, and the ρ that matches is found by a coarse grid. Spearman correlations over model triples are over 35 triples per layer.

Exclusions. Truncated answers are missing, not wrong, in every stage; items are included in a stage's table only if every agent of that stage answered them without truncation, which is why item counts differ between tables and are printed in each. Saturated layers were dropped before any aggregation was computed, by the criterion that every model exceeded 0.95. Mistral's ten incomplete debates are excluded from the six-agent tables and recounted without Mistral in the companion tables.

Post-hoc cuts. The first reading of the hypotheses against the tables asked for cuts the first tables did not have: the all-wrong stratum of debate, the transition table by round-0 correct count, the flip table, the copula and two-state fits, the judge committee thresholds, the error-detection AUROC. These were computed from the existing rows, with no new calls, and are labelled in the companion tables as the second and third passes. The three homogeneous-debate arms, the certify stage and the synthesis stage were new runs made after the first reading, each with its own cap; they are the only data collected after a result had been seen, and the results they tested are stated in Sections 11, 12 and 15 as the hypotheses they were run for.

Reading the hypotheses. The 140 hypotheses were read against the tables by Claude Opus 5.5 at low effort, three times, with instructions to quote only numbers present in the tables and to return "insufficient" whenever the quantity was not measured. The verdicts in Section 17 are those readings, each with the model's stated confidence, checked by the author against the tables but not re-derived from the papers.

Items. 240 problems with a single checkable answer, sixty from each of four layers: MMLU-Pro (ten-option multiple choice, stratified over subjects), GPQA Diamond (four-option graduate science), GSM-Hard (GSM8K word problems with large numbers; its gold labels are known to contain errors), and the 2024 and 2025 AIME plus the February 2025 HMMT (integer or short-expression answers). A first pass over 1,576 items had included MATH-500, GSM8K, CommonsenseQA, StrategyQA and Knights & Knaves puzzles; every model scored above 0.95 on those and they were dropped from every table, because a layer on which nobody errs cannot say anything about shared errors. They remain in the matrix file.

Models. Seven families, all reasoning models of mid-2026, all allowed to think before answering: Claude Opus 5.5, Sonnet 5.5 and Haiku 4.5 through the Claude Code CLI (temperature not controllable, effort set low), and DeepSeek V3.2, GLM 5.3 Flash, Qwen 3.7 Plus and Mistral Large 2512 through OpenRouter, with the serving provider logged per call. Five further models answered subsets of the matrix and are not used here.

Grading. Letters by exact match; numbers by normalised numeric comparison; olympiad expressions by a normaliser plus an LLM arbiter (Opus, decisions cached) where the normaliser and the gold string disagreed. Every answer was extracted from a required last line, FINAL: <answer>.

Every row stores the parsed answer, its correctness, the provider that served the call and the token counts (prompt, completion, reasoning). The debate stages also store the full text of every turn and what each agent was shown. Mistral Large dropped out of 10 of the 240 debates on provider rate limits; the six-agent tables use the 229 complete items and a five-agent recount is in the companion repository. The whole corpus cost about $50 on OpenRouter, plus subscription calls for the Claude models.

The oldest question first: does the plurality of seven different models beat the best of them? On every layer it does not, and the margin is not noise on the two layers large enough to say so.

Items are those all seven answered without truncation, which is why these layers are larger than the sixty-item debate subsets and the olympiad is smaller: long olympiad solutions truncate. Per-model accuracies are from the 0/1 matrix (504/92/133/27 items); best, plurality and ceiling are from the answer-identity recount (499/92/133/29), which drops a few rows with empty answers. GPQA Diamond is close to saturated for the strongest models and is the layer on which the plurality comes nearest to the best member.

The interesting columns are the last two. The ceiling is 2–4 points above the best model, and that gap is the entire prize any aggregation scheme can collect on these problems. It is small because the models fail together: if the seven erred independently at these accuracies, the all-wrong rate would be below 10⁻⁴, and it is 1–10%.

The copula column is a one-parameter model: each model is correct when a latent normal variable, which is √ρ times a shared item factor plus √(1−ρ) times private noise, falls below the threshold set by its own accuracy.² With ρ set to the tetrachoric estimate it reproduces the observed all-wrong rate within a few tenths of a point on three layers and within 2.3 points on the 27-item olympiad; the ρ that would match each layer exactly is 0.85–0.95. One shared axis of difficulty accounts for the joint failures, and seven families drawn from four companies and two continents all load on it. Family does not move a model off that axis: among the 35 triples of models the plurality's gain over its best member is negative on average on every layer, and on the two multiple-choice layers it is positively correlated with the triple's mean φ (Spearman +0.57 and +0.53), the wrong sign for any story in which diversity is what a vote buys. The previous note on this site found the same thing on public correctness matrices with a different control; here it holds inside one corpus with the same items for every model.

Self-consistency, the plurality over k draws of one model at temperature 1, is the within-model version of the same question, and it has the same shape with one exception.

Items are those with k complete, untruncated draws. GLM is omitted: on GPQA and the olympiad it truncated most of its draws, and the 26 and 19 items that survive are the easy ones, so its rows would overstate it. The Kish prediction is the majority of round(n_eff) independent draws at the mean-draw accuracy.

Sixteen draws correlate with each other at φ = 0.45–0.90 and are worth 1.1–2.1 independent draws. Over the mean single draw, the plurality gains one or two points on MMLU-Pro and GSM-Hard, seven on GPQA and fifteen on the olympiad for DeepSeek; the exception is real and it is the layers where φ is lowest. The hard-state mass in the "0 of k" column, 3–17%, is the share of problems the model never solves at this temperature, and it bounds pass@k and plurality alike; a two-state fit (each item either unsolvable or solvable at a fixed rate) reproduces the plurality within a point on the three high-φ layers.

The last column corrects the earlier notes. Kish's n_eff with 1/φ was used there as a ceiling on what a vote could achieve. It is not; it is a floor. Where φ is high the Kish prediction and the observed plurality agree to the third decimal, and where φ is low the observed plurality is 23–33 points above it (DeepSeek olympiad .898 vs .566, GPQA .855 vs .620, Haiku GPQA .767 vs .523). The one-factor copula of Section 7, which models the dependence instead of discounting the sample size, lands within 1–3.5 points of the plurality on the cross-family matrix. A design-effect formula built for survey means is the wrong tool for a majority of correlated binary votes when the correlation is moderate, which two of the literature's hypotheses had predicted in the opposite direction (Section 17).

If temperature cannot buy independence, perhaps a prompt can: the "many minds" literature gives each copy a persona and reports gains. DeepSeek answered the same items four times under each of four one-sentence roles, a skeptical reviewer, a formal mathematician, a patient teacher and a pragmatic engineer, and the question is whether four roles are less correlated with each other than four plain draws.

The role correlation uses one draw per role; the plain correlation uses four draws of the same prompt. GPQA is absent because the role runs hit their spend cap before reaching it, and the olympiad row is thirteen items. Four roles are 1.06–1.27 independent answers, the same as or fewer than four draws, and cost one to six points of single-draw accuracy. A persona is a temperature setting with a hat on. This replicates, on a second family and three layers, a one-model result from an earlier bridge test on this site.

The previous note found on public traces of three copies of one model that debate is not sampling: it moves the answers between rounds, and weak models escape unanimous error at rates a sampling theory puts at zero. Here the agents are strong, the families are mixed, and the texts are ours.

229 items with all six agents present in every round; the five-agent recount without Mistral on 230 items gives .887 → .922, 10 fixed, 2 broken. "Fixed" and "broken" are cumulative against round 0.

Taken alone the first three columns are a success: the majority gains 4.8 points, every agent's own accuracy gains 12, fixes outnumber breaks twelve to one, and the gain plateaus at round 2. The remaining columns are the price and the mechanism. Unanimity rises from 129 items to 208 and unanimously wrong items from 4 to 15; the panel goes from 1.7 effective voters to 1.2; the number of items on which all six are wrong does not fall.

Where the twelve fixes come from is the question the stored texts answer. Split the 229 items by how many agents were right at round 0:

One of the twelve all-wrong items ends with a correct majority, an olympiad problem; by layer the all-wrong counts are 2, 1, 8 and 1, and the escapes 0, 0, 0 and 1. Five of the nine items on which exactly one agent was right end correct, and twelve of the fourteen with two. The gain lives entirely in the stratum where the right answer was on the table and a minority held it. A difficulty split says the same from the other side: using DeepSeek's own solve rate over its sixteen draws as the item's difficulty, nine of the twelve fixes are on problems DeepSeek solves less than a quarter of the time, two on problems between a quarter and a half, one above, and none above three quarters, where the round-0 majority was already right on 162 of 163. Debate is a procedure by which a correct minority wins. It is a good procedure for that, and it does nothing else.

The texts also say how the price is paid. For every agent that was correct at some round, count the peers who held a different answer and look at the next round:

Against one or two dissenters a correct agent is safe; against four or five it folds between a fifth and a quarter of the time. The families differ in spine. Alone against five, Sonnet 5.5 kept its correct answer 14 times of 16, Mistral 4 of 4, Haiku 5 of 6, DeepSeek 2 of 4, GLM 2 of 3, Qwen 1 of 3. The 36 alone-against-five cases are few, but the ordering matches the pick-task ranking in Section 13, in which Sonnet is also the judge least swayed by its own prior answer.

Per layer the picture sorts itself by starting correlation:

Debate beats the best single agent only on the two layers where the agents start least correlated, and loses to it on the two where they start most correlated. A low starting correlation is a precondition for debate to pay, not something it produces; debate consumes it, taking the ICC to 0.95–0.96 on the high-φ layers within three rounds. (The olympiad's ICC falls because by round 3 almost every agent is right on almost every item, and a near-constant vector has no variance to share.) The best single agent on GPQA here is 0.875 because Opus, the strongest model on that layer, is not in the debate panel.

Two practical numbers. Stopping each item at its first unanimous round, and taking round 3 otherwise, gives majority accuracy 0.921, the same as three fixed rounds, at 0.76 rounds per item on average, because 129 items were unanimous before anyone spoke and 51 more after one round. Three rounds of six agents cost 21,800 completion tokens per item on top of the first answers (1,431 per agent-turn in round 1, falling to 1,083 in round 3); the stopping rule spends about a quarter of that for the same accuracy. Subsets of the panel tell the same story in miniature: the mean plurality over all pairs of agents is 0.800 at round 0 and 0.916 at round 3, over all triples 0.827 and 0.920, and the full six 0.873 and 0.921; debate makes a pair almost as good as a sextet, which is what one would expect if its job is to transmit the one or two correct answers in the room.

Two things are entangled in a debate gain: the information each agent receives from its peers, and the fact that it was asked to think again. The public traces in the previous note had a self-only control on one model; here it is built in. Six copies of DeepSeek, seeded with six of its own temperature-1 draws as round 0, debate all 240 items three ways. In the plain arm each copy sees the other five as in Section 10. In the self-only arm each copy sees only its own previous answer and reasoning, with the instruction to reconsider on its own. In the pressure arm the system prompt adds that the other agents are usually right and the copy should defer to the majority unless certain.

Three readings. First, roughly half of the homogeneous gain, 1.3 points of 2.9, is self-revision with no peers at all, and that half costs nothing: the self-only arm leaves the correlation at 0.65 and the count of unanimously wrong items at 10. The other half, the information, triples that count to 31. Second, telling the copies to defer changes nothing they were not already doing; the pressure arm is the plain arm within noise, which is what Section 10's flip table predicts for a model that already folds at 22% against a majority. Third, and this is the number to carry from the previous note: with a strong model talking to itself, the all-wrong stratum is 0 for 26 in every arm. The 31–45% escape rates on weak models in the public traces were real and were specific to models whose first answers were under-elicited; a 2026 reasoning model that is wrong six times is wrong because the problem is hard for it, and six copies of it have nothing to tell each other.

Copies gain less than families, 2.9 against 4.8 points, and the per-layer table of Section 10 says why: the copies start at φ = 0.66 where the families start at 0.50, so there is less decorrelation to consume. The homogeneous majority at round 3, 0.846, is still 5 points under the ceiling for those copies, 0.892 (1 − 26/240, the "someone was right" rate across the six seed draws), and the mixed majority 0.921 is 3 points under its ceiling of 0.948 (1 − 12/229); neither box is left.

The generative objection to everything above is that a vote is the wrong aggregator: a model that reads six answers can write a seventh, and the mixture-of-agents literature reports exactly that. We gave DeepSeek, at temperature 0, the six round-3 transcripts of the mixed debate with the instruction to pick one or derive a different answer if all were wrong; separately, five of its own draws.

Over the transcripts the synthesiser does half a point worse than the plurality; over its own draws, 1.3 points better; in both cases it recovers nothing from the all-wrong stratum and writes a novel answer once in 470 tries. An aggregator with the freedom to leave the box sits a point or two under its ceiling, where a good vote sits. On these models and these problems, reading the others' reasoning is not a way to get a new answer; it is a way to pick among the old ones.

Everything so far asks the group to produce the answer, and the box says the prize is in selection: on a contested item some model is right and the plurality may not be, and a verifier that finds the right one collects the 2–4 points between plurality and ceiling. The earlier notes predicted that an LLM verifier cannot, because its errors are the solvers' errors. Half of that prediction holds; the half that fails is the useful half.

On the 84 items where the six debate agents' matrix answers disagreed and at least one was correct, each of five models was shown the distinct candidates, each with a 700-character excerpt of one proposer's reasoning, in a shuffled anonymised order, and asked to pick one or say none. (The candidate set is the six debate agents' answers, so a judge that also solved the item may find its own answer among them.)

"Picks a wrong candidate" excludes picks that could not be parsed; Qwen and GLM failed to name a candidate on a handful of items each, which is why their wrong-pick rates are low while their pick accuracies are not. Four of the five judges pick better than they solve, Haiku by 14 points, and one picks worse: Sonnet, the strongest solver on these items, loses 5 points by judging rather than answering. Nobody but Sonnet beats the solvers' own plurality of 0.80, and Sonnet by two points on 83 items.

The result is the panel. Judges' picking errors correlate across judges at φ = 0.21 (tetrachoric 0.36), against φ ≈ 0.5 for the solvers' errors on the same problems: five judges are 2.7 effective voters where six solvers were 1.7. A majority of the five picks correctly on 0.867 of the contested items; the mean over all ten triples of judges is 0.827; the mean single judge is 0.680. The panel beats the best judge by five points, the solvers' plurality by seven, and collects about a third of the distance from plurality to oracle. Comparing candidates against each other is a different task from producing them, and on it these models disagree in ways that are useful to each other.

Two things qualify this. On the 13 items where no candidate was correct, the judges said "none" once in 65 opportunities; a picker cannot abstain. And the result is 0.867 against 0.80 on 84 items from one corpus. It is the only positive number in this note and it is the one that most needs a second dataset.

The same independence shows up in a cheaper form. Take DeepSeek's matrix answer on each of the 229 debate items, wrong on 21% of them, and ask how well two warning signals rank the wrong ones: the number of the other five families whose answer differs from it, and the number of DeepSeek's own three temperature-1 draws that differ from it, the usual self-consistency signal. Peer disagreement reaches AUROC 0.917; self-disagreement 0.823. Other models know when you are wrong better than you do, and five one-shot calls to other families are a better abstention signal than three extra calls to yourself. This is the mechanism behind the panel in Section 13: a judge's "this candidate is right" is partly "this candidate agrees with what I would have said", and that signal is less correlated across families than the answers themselves.

The half of the prediction that holds is the other judging task. Shown one candidate at a time, with its excerpt, and asked only whether it is correct, with one planted wrong answer per item taken from some model's wrong draw on that item, the same five judges behave like solvers again.

The three OpenRouter judges have fewer verdicts because that stage hit its spend cap at 82% coverage. Two things stand out. Sonnet is a good certifier, 9% false acceptance and one decoy in 68; the rest accept a third to a half of wrong candidates, and a planted wrong answer that no model proposed is accepted 11–29% of the time by the other four. And the dominant term is self-preference: every judge accepts its own wrong answer more often than someone else's, by a factor of 2.5–3.3 for three of them, 7 for Sonnet and 1.2 for DeepSeek. That is not style recognition alone, since the candidate was anonymised and shown with another proposer's excerpt when several models gave it; it is that a judge asked "is this right?" re-derives the answer, and re-derives its own error.

A committee does not repair this, because the false acceptances correlate at φ = 0.51 across the five judges, 1.6 effective judges, the solver number. Over the 116 candidates all five ruled on:

Even unanimity lets through 6 of 66 wrong candidates, 9%, at a cost of a quarter of the correct ones. No threshold reaches a 5% false-acceptance rate. Put next to Section 13: the same five models, the same items, the same candidates, and in one framing they are 2.7 independent judges whose vote beats everything, in the other they are 1.6 correlated solvers who wave through their own mistakes. The difference is whether the judge compares or re-solves. The swarm can rank; it cannot certify, and it cannot abstain.

Every row carries its token counts, so the operators can be put on one scale. For DeepSeek on the same sixty items per layer, completion tokens (reasoning included) per item and the accuracy of each operator's output:

These are the same sixty items per layer in every row, which is why the one-answer accuracies differ from the matrix table. The copies' debate includes its six seed draws; the families' debate excludes round 0 because the matrix calls were made by other models. On the two high-correlation layers, thirteen to twenty times the tokens buy one to three points or nothing; on the olympiad, sixteen draws buy fifteen points, which is the one place in this corpus where spending on sampling is clearly worth it, and the homogeneous debate buys the same for a third fewer tokens. The mixed-family debate's GPQA and olympiad accuracies are not DeepSeek's: they are the panel's, with Sonnet and Qwen in it. A cost-matched comparison of these operators against a single model given the same budget is the subject of the next note; the table is here so that the reader has the raw numbers.

Four things this does not say. It does not say that a swarm of agents working on different subtasks, with tools, memory and a plan, adds nothing; that is a different operator, its gains would come from context length and parallel time rather than from independent errors, and the right control for it, one agent doing the same decomposition sequentially at the same token budget, is not in this corpus. It does not say anything about base or non-reasoning models, where error correlation may be lower and voting may pay; every model here thinks before answering, and the previous note's weak models behaved differently. It does not say debate is useless: a 4.8-point gain with a stopping rule that costs a quarter of three rounds is a real engineering result, on the layers where the panel starts decorrelated. And it does not say the comparator result is large or general: it is 0.867 against 0.80 on 84 contested problems from one corpus.

The hypotheses the corpus was built for were extracted from published papers and judged before any of this data existed; the companion repository of the discovery pipeline holds all 140 with their three verdict passes. The ones the tables answer, by the hypothesis's own falsification criterion:

• Correlated judge panels (CollabEval, arXiv:2603.00993, read against Nine Judges, Two Effective Votes, arXiv:2605.29800). Predicted: discussion rounds raise φ and cut n_eff, and items that reach consensus after discussion are wrong more often than items unanimous before it. Supported: φ 0.50 → 0.81, n_eff 1.71 → 1.19, unanimous-and-wrong 3.1% → 7.2% (Section 10). The caveat is that these are solver debates, not judge panels.
• Pre-debate independence predicts debate gain (OptAgent, arXiv:2510.18032, against Effective Independence). Predicted: gain over the best single agent tracks the panel's pre-debate n_eff, and debate drives n_eff toward 1 whatever the topology. Supported on four layers: n_eff 2.3–2.4 → gain, 1.2–1.4 → loss; the self-only control shows that seeing peers is what causes the collapse (Sections 10–11).
• Relay gains live inside the selection ceiling (WILC, arXiv:2607.29087, against the co-failure ceiling paper, arXiv:2606.27288). Predicted: iterative collaboration gains almost entirely on items where some worker was already right. Supported: 1 of 12 all-wrong items escaped; the gain sits in the one- and two-correct strata.
• Mixture-of-agents synthesis beats the ceiling (arXiv:2605.01566 against arXiv:2606.27288). Predicted: an aggregator's surplus over voting comes from items where every proposer was wrong. Not supported: 0 of 27 and 0 of 15 (Section 12); the surplus is selection inside the ceiling.
• Role prompts decorrelate (Dynamic Role Assignment, arXiv:2601.17152, against the Blackwell-informativeness paper, arXiv:2605.06028). Predicted: role-diversified proposals have lower inter-agent error correlation than temperature samples. Not supported: equal or higher (Section 9). A sibling hypothesis from the same paper, that its reported debate gain is score-only selection and the subsequent debate erases diversity (against The Interaction Tax, arXiv:2608.23541), is also not supported: debate raised accuracy above the round-0 vote, and selection by a single judge did not beat the vote.
• Kish over-predicts the vote (two hypotheses pairing Kish 1965 with Boland 1989 and with arXiv:2607.23931). Predicted: n_eff = 1/φ overstates what a majority can reach. Not supported, and reversed: where φ is moderate, Kish under-predicts the plurality by 23–33 points (Section 8). The companion hypothesis from the information-theoretic ensemble paper (arXiv:2602.08003), that the probit floor Φ(Φ⁻¹(1−α)/√ρ) beats the Kish prediction, is supported by the same rows.
• A one-factor model under-predicts joint failure by 2–3× (the Honest Quorum paper, arXiv:2607.16109, against Effective Independence). Not supported: at the tetrachoric ρ the copula lands within 1.0–1.45× of the observed all-wrong rate (Section 7). The same paper's claim that provider-family diversity does not make a certifying committee feasible, against arXiv:2607.23931, is supported by Section 15: false endorsements correlate at 0.51 and no threshold reaches the 5% bound.
• Pooled independence ≲ 2 (CTTS, arXiv:2508.03333, against No Free Lunch). Predicted: a heterogeneous pool has about two effective members and selection stays within the union of candidates. Supported in part: n_eff 1.2–2.0, every aggregator inside the ceiling; the paper's own method was not run.
• Debate rounds add nothing beyond round-0 voting (TUMIX, arXiv:2510.01279, against the modal-ceiling paper, arXiv:2606.28661), the hypothesis the previous note tested on public traces. Not supported here either, on a second dataset with different models: majority 0.873 → 0.921, 12 fixed against 1 broken.
• Majority accuracy peaks before consensus (the Koopman-certification paper, arXiv:2608.05956, against Ladha 1992). Not supported: majority accuracy is non-decreasing to near-unanimity, with no interior peak (Section 10).
• Debate gains when errors are already correlated (arXiv:2511.07784 against arXiv:2605.29800). Not supported, reversed: the gain is largest where φ starts lowest.
• Prompted deference does not control convergence (Thinking with Many Minds, arXiv:2501.02348, against BOUNDARY_SYNC, arXiv:2607.01600). Supported with two prompt levels: exposure drives convergence, the stated weight barely moves it (Section 11).

Each of these is one reading of one table by one model with a stated confidence between 0.5 and 0.7, and each hypothesis was written about a specific system this corpus does not reproduce. They are listed as what a careful reader of these tables would conclude about those claims, not as replications of the papers.

• Size. Sixty items per layer, 229–240 in the debate stages, 84 contested items for the picking result, 66 wrong candidates for the certifying one. Confidence intervals on the strata are wide: the all-wrong escape rate of 1/12 has a Wilson interval of [0.01, 0.35], and the 36 alone-against-five flips split six ways. Directions agree across layers; magnitudes deserve one decimal.
• Models. Reasoning models of one vintage only, and the Claude models at a fixed temperature. The homogeneous debate, the personas, the synthesiser and the error-detection result rest on one family, DeepSeek V3.2.
• Round 0. The matrix answers were made at temperature 0 and the debate rounds at temperature 1, as in the public traces of the previous note. The self-only arm of Section 11 is the control for the share of the round-1 jump that is re-elicitation, and it is about half.
• Mistral dropped out of 10 of 240 debates on provider rate limits. The six-agent tables use the 229 complete items; the five-agent recount matches to within half a point.
• Estimators. Tetrachoric correlations are the cos-π approximation, biased upward at high accuracy; φ and ICC are exact. The copula fit is a simulation at one ρ per layer, not a likelihood fit. n_eff is Kish's formula used descriptively.
• Labels. GSM-Hard's gold answers contain known errors; its 10% all-wrong rate is partly label noise, which inflates the shared-error estimate on that layer only. Olympiad expressions were graded with an LLM arbiter where normalisation disagreed with the gold string.
• Judging design. Candidates were anonymised but shown with a reasoning excerpt; a judge could recognise its own style. In the pick task a judge's own answer was among the candidates whenever it had solved the item; in the certify task "own" means the candidate equals the judge's matrix answer. The self-preference rates are therefore a mixture of style recognition and re-derivation, and this corpus cannot split them.
• Coverage. The certify stage reached 82% of the OpenRouter judges' calls before its spend cap; the role runs did not reach GPQA; GLM's hard-layer samples are mostly truncated.
• Pre-registration. None. The stages were designed to serve hypotheses written by other people, and the verdict passes that read those hypotheses against the tables were done by a model; both are in the discovery repository, not in this note.

Every table in this note, and about twenty more that it quotes from, is rebuilt from the stored answers by a single standard-library script in about two minutes with no network access. The dataset and that script will be released as tech-note-5 within about a month of this note, after a second reading: every row of every stage with the parsed answer, its correctness, the serving provider and the token counts, but not the model texts; GPQA Diamond answers replaced by hashes and no question text or gold label stored, per that dataset's terms. Figures are drawn inline from the same numbers. The collection scripts, graders, and the discovery pipeline that proposed the hypotheses and read the verdicts live in a separate repository that will accompany a later note on the pipeline itself. Until the release, the numbers here are the author's word; after it, they are a command.

Archived with a DOI: 10.5281/zenodo.23229043. Cite as: Latipov, A. (2026). Where the Swarm Pays. Zenodo. https://doi.org/10.5281/zenodo.23229043

1. The earlier notes: Effective Independence (31 August 2026), The Control Ate the Signal (2 October 2026) and Debate Is Not Sampling (4 October 2026). The pipeline follows Swanson's literature-based discovery: paper A states an open question or limitation, paper B reports a result that bears on it, A and B do not cite each other and are otherwise dissimilar, and the pair is a candidate hypothesis. Candidates were sieved by embedding similarity between A's stated needs and B's stated offers and scored 0–10 by an LLM judge (Claude Opus 5.5, with a second opinion from Claude Fable 5.1 on the strongest). Papers and foundations, from Condorcet and Kish to the 2026 preprints, were summarised into fixed-schema cards first; the cards, not the papers, were what the judge read. ↩
2. One-factor Gaussian copula: model i is correct on an item when z_i = √ρ·F + √(1−ρ)·e_i < Φ⁻¹(acc_i), with F shared across models within the item and e_i private, both standard normal. Simulated with 20,000 items per layer. At the tetrachoric ρ the all-wrong rate is 0.048 / 0.009 / 0.095 / 0.051 against observed 0.050 / 0.011 / 0.105 / 0.074 (MMLU-Pro / GPQA / GSM-Hard / olympiad), and the simulated plurality-correct rate is 0.892 / 0.938 / 0.867 / 0.891 against observed 0.903 / 0.957 / 0.880 / 0.926. The model has no answer identities, so its "plurality" is a majority of correct votes; the observed figures in this footnote are on the same definition and differ slightly from the answer-identity plurality of Section 7. ↩

← notes · github.com/Alatip