A first test of "train the step, not the task" on Qwen3.5-4B: find the step that breaks long multiplication, find where the model does it, change 256 rows of one weight matrix, check on items the edit never saw.
In There Is No Free Lunch After All (8 September) I wrote down a claim and a corollary. A task is a chain of steps, and its success is limited by the weakest one. So the useful thing to improve is that step, not the task: "Train the step, not the task." The note planned to find the step by feeding the model a correct prefix up to step t and to fix it with step-targeted training data. It also warned that a model trained on its weakest step may become "brittle one step to the side," so the price has to be measured as well as the gain.
This note is the first measurement. Two things changed from the plan. The repair is a direct, local change to the weights rather than new training data. And the step is found in two stages: first by behaviour (which step fails), then inside the model (which component does that step and how it goes wrong).
The setup is deliberately small: Qwen3.5-4B in bf16 on a laptop, greedy decoding, thinking off, one process with no batching, up to 16,384 new tokens per answer. The main task is 40 products of a 5-digit and a 4-digit number, for example 56544 × 3166.
Four small experiments, each narrowing the failure.
| experiment | result | what it says |
|---|---|---|
| Arithmetic expressions, no tool vs a calculator tool | 52/60 vs 60/60 | failures are in long sums and digit assembly, never in one-digit facts |
| Each step alone (one-digit multiply, two- to four-number add, subtract) | 49/50 | the steps are fine in isolation |
| Multiplication from scratch; with the correct partial products already written | 32/40; 40/40 | all failures are in summing the partial products |
| Partial products given, sum forced "column by column" | 6/40 | column lines: 133 of 339 have a wrong digit, 0 of 339 wrong arithmetic |
The last row is the useful one. Asked to add four numbers place by place, the model writes lines whose sums are always right and whose digits often are not. For 96804 × 4798 the partial products are 387,216,000, 67,762,800, 8,712,360 and 774,432, and the model writes "Hundred-thousands: 2 + 7 + 8 + 7 = 25". The hundred-thousands digit of 8,712,360 is 7; the 8 next to it is the millions digit. So the weak step is not adding. It is taking the digit at place k of a number written earlier in the context. Of 746 digits written this way, 89 are wrong; most often the model took the digit one place over in the same number, next most often the same place of another number.
The forced column format is worse than the model's own (6/40 vs 32/40) because it calls this weak step dozens of times per answer. That makes it a good place to study the step and a bad way to solve the task.
Items are split by number: even items for all the looking and fitting, odd items held out. Everything in this section is measured on the even half, by re-running the model on its own text (teacher forcing) and reading the position where it writes a digit.
Only the query of layer 23 head 0 changes: the 256 rows of q_proj that produce it, as W ← W + UVᵀ with rank 16. Keys, values, and everything else stay as they were.
The edit is fitted to an internal target, not to the output: the head should put its attention on the right source digit at the writing position. A second term keeps the head's attention unchanged on five unrelated texts, on 40 answers from the other experiments, and at random positions (a KL penalty), and a third keeps the edit small. Because nothing before layer 23 changes, the inputs to the head can be cached once from the original model and the fit runs on that cache, exactly, without a forward or backward pass through the model. The numbers were found by gradient descent (Adam, 400 steps); the rank and the weight of the protecting term were chosen on half of the even items, by a rule written down in advance, and the chosen setting was then refit on all of them.
Before any odd item was scored, the fitted edit's hash and four predictions were written into the preregistration.
| prediction (held-out odd items) | result | |
|---|---|---|
| Teacher-forced: ≥ 9 wrong digits fixed, ≤ 3 right ones broken | 28 of 37 fixed, 0 of 326 broken | holds |
| Regenerated column sums: wrong-digit rate ≤ 6.5% (was 9.9%) | 1.6% (8 of 495); Fisher p = 4·10⁻⁸ | holds |
| Other skills: isolated steps ≥ 47/50, calculator ≥ 58/60, no-tool ≥ 50/60, multiplication ≥ 30/40 | 47, 60, 53, 34 | holds |
| Drift on unrelated text no larger than random edits of the same size | KL 0.00035 vs 0.00030–0.00042 | holds |
The second prediction, as preregistered, counts digits only in column lines with the right number of terms. A line that drops a partial product's digit or adds one is skipped, and after the edit that is the more common error. Counting every line, 61 of 171 held-out lines were wrong before the edit and 23 of 171 after (Fisher p = 2·10⁻⁶); 14 of the 23 drop a term. So the edit fixes most wrong picks but leaves a third of the line errors, and those are of the kind the fit never targeted. Version 1 of this note did not say this.
Inside the head the change is large: on held-out digits its attention on the right source goes from 0.52 to 0.93 where the model was already right, and from 0.32 to 0.81 where it was wrong (Figure 1). The fitted edit has a third of the norm of the rows it changes (3.24 against 9.96). It also beats the ceiling the oracle suggested: 76% of held-out wrong digits are fixed, not half. A likely reason is that the oracle replaced the head's output at one position, while the edit changes its query everywhere, including in the model's earlier column lines; this was not tested.
Independently of the repair, I tried the other option: avoid the weak step. The model has two ways to add its partial products, column by column (many digit lookups) or pairwise, adding whole numbers two at a time (few). A direction in the residual stream at layer 18 switches the choice; adding it as a constant bias to that layer's MLP output, a weight change too, moved the model from 11 to 35 pairwise sums out of 40 and multiplication from 32 to 37 of 40.
The price was elsewhere. Plain two-number addition and subtraction fell from 20 of 20 to 13 of 20: the model began miscopying 9-digit operands and dropping the lowest digit group in its sums. Its drift on unrelated text was also above random edits of the same size (KL 0.024 against 0.016–0.021). The repair costs less than the bypass on every measure here.
Answer accuracy on the forced column format went from 6 to 8 of 40. The column lines are much better but not all right: 22 of 40 answers still contain a wrong line, usually one with a dropped term. Exact match also hides a real gain. The six lowest digits of the final answer, which the model copies from its lines, are right in 164 of 240 cases instead of 122; on the held-out items, 86 of 120 instead of 60, with 30 digits gained and 4 lost. The higher digits were mostly right before and did not change (96 and 97 of 114). The final number is still mostly wrong, partly because of the lines left wrong and partly because the model does not assemble the answer from its lines: in 37 of 40 answers of the original run, the final answer disagrees with the model's own column digits. That is the next weak step, and it is a different one.
Why the task fails although the step mostly works: one answer has about nine column lines and calls the step about twenty times, and errors compound. Dziri et al. made this argument for transformers on multi-digit multiplication (Faith and Fate, 2023); the numbers here fit it.
| line right | lines per answer | all right, if independent | all right, observed | answer correct | |
|---|---|---|---|---|---|
| original | 60.1% | ~9 | 0.601⁹ ≈ 1% | 3 of 39 | 6 of 40 |
| edited | 85.1% | ~9 | 0.851⁹ ≈ 23% | 16 of 39 | 8 of 40 |
Errors cluster in some items, so more answers come out clean than independence predicts. All 40 items, both halves; a line counts as right when its digits are the true column's, whatever their number. After the edit the chain of lookups survives in 16 of 39 answers, and only 7 of those 16 end with the right number: the task now fails at two links, the lookups still wrong and the assembly of the answer. (Version 1 gave 8 → 30 of 39 and 7 of 30; that count left out lines with a dropped or extra term.)
For the claim of the September note this is the expected picture: a task is a chain, and strengthening one link exposes the next. It also bears on the cases the note distinguished. The digit step was not missing from the model: it failed 10–14% of the time, not always, and a small change was enough to fix it. In the note's words, the edit moved mass onto a step the model already had; it did not create one. A step that is truly absent has not been tested here.
Teacher forcing (one forward pass over a reference answer) is hundreds of times faster than generating, so it is the obvious way to screen edits. I compared it with generation on 1,209 pairs of model and correct reference answer, for the original model and both edits.
Locating a component and then changing it is an old idea in interpretability. Two recent pieces are closest to what is done here, and both differ in ways that matter.
Weight-space programming (Takehi, Teoh, Zhang, Khattab, MIT CSAIL, blog post, 2 November 2026). A general-purpose LLM reads another network's weights and writes new ones, by definition without an optimizer. The headline edit subtracts one direction, found by the editing model, from 64 output columns of two MLPs in Qwen3.5-0.8B and raises MATH-500 from 7.9% to 48.1%; the diagnosed problem is thinking loops. An independent replication (OpenWSP) reports that turning thinking off alone gives 46.8%, so the gain is mostly loop removal.
Attention Behavior Fine-Tuning (Cho et al., BlackboxNLP 2025, arXiv 2505.14233). Also trains on where heads look instead of on the output: in in-context classification, induction heads are rewarded for attending to the correct label token and penalised for the others. Gradients go to the query and key weights of all heads in all layers.
| weight-space programming | ABFT | this note | |
|---|---|---|---|
| how the failure is found | editing LLM traces a behaviour (looping) to a direction | a generic induction-head filter | behavioural step isolation, then one head, then its query side |
| what changes | output columns of two MLPs | query and key of all heads, all layers | 256 query rows of one head, rank 16 |
| how the numbers are found | written by the LLM, no optimizer | gradient through the whole model | gradient on a cached, exact local computation |
| target | removing a direction | attention on label tokens | attention on the source digit, with attention elsewhere held fixed |
| side effects checked | not reported | accuracy on other classification sets | four other skills, random-edit control, preregistered bounds |
By the MIT definition the edit here is not weight-space programming, since its numbers come from an optimizer. The target, though, is nearly linear in the query, so a closed-form version without any optimizer looks possible; that would make the two directly comparable. Not done yet.
The tooling started from Lyra's neuron explorer for Qwen3.5-4B, which is built around MLP neurons and steering. This work needed attention maps, activation patching and scores over hundreds of cases, so those were added in a small library over the same model and hook points.
Archived with a DOI: 10.5281/zenodo.24163236. Cite as: Latipov, A. (2026). Repairing One Step in the Weights. Zenodo. https://doi.org/10.5281/zenodo.24163236