GPT-6 Astra Edits Qwen Weights Directly, Lifting Math Scores 6x

MIT researchers say OpenAI’s GPT-6 Astra can read the raw weights of another neural network and edit them directly, with no gradient descent and no training run. In their headline test, GPT-6 Astra changed just 0.016% of the weights in Alibaba’s small Qwen3.5-0.8B model and lifted its MATH-500 score from 7.9% to 48.1%.

The work comes from Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab at MIT CSAIL, who published it on November 2 in a blog post titled Sparks of Weight-Space Programming. They call the idea “weight-space programming”: a general-purpose language model reasons about what each part of a network computes and rewrites the numbers by hand.

We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B’s reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.

We call this Weight-Space Programming

— Rikiya Takehi (@rikiyatakehi) November 2, 2026

How GPT-6 Astra Fixed Qwen’s Math

Qwen3.5-0.8B has a known weakness: its own model card warns that it is prone to thinking loops. Before the edit, 93% of its MATH-500 answers ran into the 32,768-token limit. Working in a Python session with up to 50 batches of test runs on practice problems, Astra traced the looping to one direction written by the MLPs in layers 15 and 19, then removed it from 131,072 weights.

With that patch, GSM8K rose from 13.8% to 67.1% and the IFEval instruction-following score climbed from 21.0% to 62.3%. According to VentureBeat, the strongest attempt still took 158 conversation turns, 80 Python runs and 20.3 hours. The edited model is now on Hugging Face, where a rebuild from the published logs scored 51.0% on MATH-500, Takehi said.

Reading and Writing Small Networks

The smaller experiments are just as striking. Given only raw weights, Astra named which of four games a policy network was trained on, and which language three tiny models learned. It also wrote MinAtar game-playing networks from scratch that beat PPO agents trained for 100 million steps, and fixed a Kinetix car policy so it succeeded in all 1,024 episodes.

The Limits Are Real

The team is clear that the Qwen gain came from removing an obstacle, not adding knowledge. On Gemma 3 270M, which rarely loops, Astra could not improve math at all, and it also failed to spot a sycophantic model from weights alone. Interpretability researcher Neel Nanda wrote that the work is best seen as Astra being “surprisingly capable of doing old school mechanistic interpretability research rather than anything spooky.”

The result also adds to a busy stretch for OpenAI’s flagship, which was recently at the center of a distillation report naming Chinese labs.

Frontier models still post low scores on hard new tests like the GDPx benchmark, so cheap, targeted weight fixes could become a serious tool. The MIT team says it is now building a benchmark and harness for weight-space programming.

That matters for every lab racing to scale, a race also visible in Anthropic’s revenue numbers.