The note was revised three times on the day it was published, each time after a reading found errors; the third reading set the text against the code; a fourth revision changed only the wording. Each version is kept as a frozen page, the current one lives at the main address, and the tool below shows the text of any two versions against each other, paragraph by paragraph with word-level marks. The comparison is computed in the browser from the archived pages; nothing is sent anywhere.
| version | date | what changed | page |
|---|---|---|---|
| v1 | 8 Oct 2026 | First version. The items on which all seven models were scored wrong had not been audited; a cut-off answer was scored as a wrong vote; the Kish majority rounded neff to an integer. | v1 |
| v2 | 8 Oct 2026 | Label audit of the all-wrong items (relabels, exclusions, re-grading with the normalising grader); tables rebuilt; note rewritten around the audited tables; tie analysis and whole-set accounting for the judge panel. | v2 |
| v3 | 8 Oct 2026 | A cut-off answer is a missing vote, not a wrong one (old tables kept in a sub-section of Section 10); audit bookkeeping corrected (48 items, 21/22/3/2) and unaudited items named; matrix ceiling stated with and without the token-cap filter; judge coverage and matched tie rules; exact Kish estimator; candidate identity within the grader's tolerance; GPQA answer tokens made non-invertible in the companion data. | v3 |
| v4 | 8 Oct 2026 | A reading of the code against the text: the solvers' tie rule in the judge section (the panel's lead is 11 to 5, eight points, not 8 to 5); a cut-off agent no longer counts as having folded in the flip table; letters outside the candidate list are not named picks; a planted decoy that the audit had made correct is removed from the decoy counts; the role correlation uses every draw pair rather than one draw per role; the triple gains are pluralities; the sample table shows every item under the missing-vote policy; audit sensitivity (MMLU-Pro relabels reverted) added to Section 7; every quoted number moved into the companion script; GPQA tokens drawn per item. Before deposit, after a fourth reading: the debate gain and the judge-panel lead given under three tie rules (4.9 → 2.2 points; +8 → −1 points), the 'best agent on its answered items' control taken on the same items as the majority, Mistral's 39 missing matrix rows reported, the four unanimously wrong debate items marked as probable label faults per the audit file, the 7.5% co-failure rate on cut-off items, 43 not 45 benchmark faults. Before deposit, after a fifth reading that recomputed the quoted counts from the rows: the cut-off denominator is 229 items (8.3%), not 254; the seven-model plurality of Section 7 breaks ties by the order of the data files, and its range over orders is given; the cut-off agent itself joins the winning majority on 4 of the 10 debate fixes credited to it; 9 of the panel's 11 wins are solver ties and on the 63 non-tied contested items the solvers lead; the judges' φ is 0.29 (2.3 effective) when a cut-off call is no pick; the synthesiser's 0 of 4 are the four probable label faults; the debate panel spans five families, not four; Mistral lacks 2 MMLU-Pro rows, not 1; 41 re-graded HMMT rows, not 61. | v4 |
| v5 | 8 Oct 2026 | Readability only: the abstract rewritten to state the results, the old abstract moved to an Introduction, the byline shortened. No number, table or claim changed; not deposited, no frozen copy. | current |
Zenodo: v1 10.5281/zenodo.23229043, v3 10.5281/zenodo.23235028, v4 10.5281/zenodo.23241005, all versions 10.5281/zenodo.23229042 (v2 was not deposited). Git history: github.com/Alatip/alatip.github.io.