Sixfold, and not sixfold

The headline version of this result is already traveling: a frontier model edited another model's weights, no training run, and a math score rose about six times. Every part of that sentence matches what four MIT CSAIL researchers reported on November 2. It still gives the wrong impression, and the researchers' own post is the document that corrects it.

The team — Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab — call the approach Weight-Space Programming. Instead of training a network, where an optimizer adjusts its weights according to measured errors, they asked GPT-6 Astra to inspect a network's weights, reason about what parts of it do, and change the numbers directly. In the largest experiment, the target was Qwen3.5-0.8B, a small open model whose own model card warns that it is prone to thinking loops.

What the edit actually did

Given practice problems and a budget of test runs, Astra traced the looping to a direction associated with the MLPs in layers 15 and 19, then modified 64 output columns in each layer to suppress that component. The authors describe the operation as subtracting the direction. Before the edit, 85 of the base model's first 96 practice answers repeated themselves until they ran out of room at the 32,768-token limit. On MATH-500, 93% of answers ended that way. After the edit, 33% did.

Astra addressed 131,072 weight entries across two tensors in Qwen's layers 15 and 19. The authors describe the edit as 0.016% of a roughly 0.8-billion-weight model.

The result is best understood as a change in completion, not a demonstration that Astra taught Qwen mathematics. The unedited model frequently repeated its reasoning until it exhausted the output limit. The edit allowed substantially more answers to finish, exposing mathematical ability that the looping behavior had obscured. Remove the loop and the scores jump together: MATH-500 from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval, a test of instruction following that Astra was not aiming at, from 21.0% to 62.3%. The authors put the distinction in one sentence: editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked.

The control does the honest work

The post then runs the comparison that separates a repair from a lesson. Gemma 3 270M rarely loops: 8.5% of its answers hit its token limit. Asked to make that model better at math the same way, Astra found ordinary errors — wrong arithmetic, invalid algebra steps — with no single component behind them to switch off. Across every budget it tried, MATH-500 moved from 6.8% to at most 7.7%, a change Astra itself reported as not established.

Two more results set the boundary. Given the weights of two fine-tuned models, one trained to answer correctly and one trained to agree with the user, Astra could not tell the sycophant from the weights alone in four attempts. It succeeded only when allowed to look at attention patterns instead. And the Qwen gain was not sitting there waiting for any small nudge: at budgets of 10 and 25 test runs, MATH-500 reached only 12.3% and 14.1%. The 48.1% took 50 runs, and by the team's own documentation the strongest attempt ran to 158 turns and 20.3 hours. The patch was tiny. Finding it was not.

What the post does not include is an unedited-model comparison using Qwen's recommended thinking-mode sampling settings. Its model card warns that Qwen3.5-0.8B can enter thinking loops under certain settings and recommends different parameters for competition mathematics. The authors held sampling settings constant between their before-and-after tests, which makes the comparison internally consistent, but it does not establish how much of the low baseline could have been recovered without changing any weights. That remains an important unanswered question.

Reading weights is the quieter result

The score is the part that will be quoted. The reading experiments may matter more. Given only the raw weights of four small game-playing networks, with no list of options, Astra named the game each was trained on, all four times. Given the middle layers of three small language models, it named the language each was trained on and found every copying attention head, using a method from the interpretability literature that the authors say they never mentioned to it. It also wrote small game-playing networks from scratch that beat networks trained for 100 million steps in all three games tested.

None of this is a product, and the authors call their results "very early". The largest network edited has 0.8 billion weights. Nothing here shows a model editing itself, and the post's one mention of recursive self-improvement is an outlook paragraph, not a finding. What has been shown, with logs public enough for others to check, is narrower and still unusual: a general-purpose model can treat another network's weights as something readable, and occasionally repairable, rather than as an opaque block of numbers only an optimizer may touch. The authors say they are building a standardized benchmark and harness for the method.

Sources

  1. [1] Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights", MIT CSAIL authors' post, November 2, 2026 (primary; read first-hand in full — the Experiment 8 setup and results table, the Gemma 3 270M control, the sycophancy experiment, the reading experiments and the authors' stated limits)Read source
  2. [2] VentureBeat, "'Like gene editing for neural networks': MIT researchers used GPT-6 Astra to edit Qwen's open weights directly, boosting math performance", November 3, 2026 (secondary; read first-hand — the strongest attempt's 158 conversational turns, 80 Python runs and 20.3 hours including a 4.7-hour outage, from the team's documentation)Read source
  3. [3] The Register, "Forget retraining — AI can now tweak another AI's weights", November 3, 2026 (secondary reporting, corroborating the main test and the 0.016% figure)Read source
  4. [4] Qwen, Qwen3.5-0.8B model card, Hugging Face (read first-hand — the warning that in thinking mode the 0.8B model is more prone to entering thinking loops than other Qwen3.5 models and may not terminate properly, the recommendation to tune sampling parameters per use case, and the task-dependent recommended sampling settings)Read source
  5. [5] ML-Intern-lab, "Qwen3.5-0.8B — edited by an LLM, no training (weight-space programming)", Hugging Face (the edited checkpoint rebuilt by Ahsen Khaliq from the authors' logs, hash-verified, and its verification run: 51.0% on the full MATH-500, 500 problems × 4 samples = 2,000 generations on an H200)Read source