We removed one subject from a 30B model. Nothing else moved.

CAST removes a single capability from a model that has already been trained, leaving the rest of it intact. Here the target was computer security, and the material we removed turned out to be broader than that label: attack technique, yes, but also protocol analysis, application security, cryptography, and, more than any of them, ordinary code and arithmetic. This page covers what that removal cost the rest of the model, and what we found when we looked closely at what we had taken out.

How the ablation works

CAST edits a model that has already been trained. It does not retrain from scratch, it does not need the original training data, and it does not filter anything at the prompt. The capability is located inside the model, suppressed where it lives, and the model is then allowed to re-fit so that everything else comes back.

What comes out is one checkpoint that can answer as the original model or as the edited one, which is what makes the comparison on this page a fair test: both answers below come from the same weights, on the same hardware, at the same settings. Everything that follows is measured on that checkpoint.

Under the hood it is a mix of activation steering and AE's recent work on gradient routing. The method is being written up properly. This page is about what it does.

What it did to the model

We grade generated answers rather than multiple choice, so that the score reflects what the model produces unprompted. Each MMLU question is given as a bare stem with no answer options, the model writes an answer, and claude-haiku-4-5 grades it for coherence (0 to 5) and factual correctness (0 or 1). 4622 questions across 53 subjects, capped at 100 per subject, generated at temperature 0.9 and top_p 0.95.

Two answers from one checkpoint: the model as it was, and the model after the removal. The second is what this demo serves.

Graded n varies by a handful of questions per model: 4622 answers were generated, and the judge returned a score for 4620 of them on base and 4621 on CAST.

The target subject collapses. Factual correctness on computer_security falls from 0.636 to 0.104 and coherence from 4.03 to 0.62, which is a model that has stopped producing sentences about the subject at all, not a model answering worse.

Across the other 4542 questions, CAST moves by -0.006, with a confidence interval that contains zero. On that evidence the removal is not measurably costing the model anything outside its target.

The damage is a small hot centre in a cold field: rung 1 loses between 24.8 and 97.6 percent of the base model’s coherence depending on direction, and by rung 3 six of the seven directions are under 10 percent.

Collateral

A handful of unrelated subjects did move under CAST. world_religions falls from 0.690 to 0.560 on 100 questions, medical_genetics from 0.757 to 0.635 on 74, high_school_geography from 0.610 to 0.490 on 100, and astronomy from 0.740 to 0.640 on 100. Those are small samples, so any single drop is noisy on its own. The leading suspicion is corpus contamination: passages about those topics sitting inside the forget corpus, which would make the edit target them on purpose rather than by accident. We are measuring that now.

What it looks like from the outside

Ask both models to explain ARP spoofing. The base model answers correctly and completely. CAST degenerates into repeated tokens.

That second behaviour is worth sitting with. The model does not decline the question and does not report a gap. It has no representation of the fact that something was taken out of it, so it tries to answer and produces noise. Any deployment of an edit like this one has to handle that failure mode at the serving layer, because the model will not.

What we removed

The target was computer security. What the edit actually saw was a body of 2000 passages about that subject, in two halves: reference prose written from a public benchmark of security questions, and a set of internal research papers on dual-use topics. Alongside it sits a much larger body of text on everything else, which the edit is told to leave alone.

That second corpus is what keeps the edit from being a blunt instrument, and the balance between the two is most of the work. The rest of this page is about what happened to be in the first one, which turned out not to be what we assumed.

What those passages actually are

We call the forget corpus cyber security. That label is looser than it sounds, and the slack in it changes what the results above mean. So we labelled the corpus the model actually trained on, and looked at what was in it.

The labelling ran in three steps, and the first one was not told what to look for. An open-vocabulary pass read every passage on its own (gpt-5.6-luna, one passage per request) with no category list offered and no mention of cyber or of unlearning in the prompt, so the features it returned are the ones the text put in front of it. A taxonomy was induced from that vocabulary. A third pass labelled every passage against the taxonomy, exactly one primary class each. 17 of 1000 came back as none of these, so the taxonomy covers the corpus.

Seven classes came out of it. What follows covers the benchmark-derived half, the 1000 passages whose source is public. The other half is internal research papers: its classes are diffuse, no class holds much of it, and it is the half the physics turned up in, so it is described here rather than charted.

The largest class is not a security class. Code Semantics holds 38.5 percent of the half, half again the size of Offensive Operations at 24.4. These are passages about what code does: types, casts, bitwise operations, sign extension, what a function returns. 91.9 percent of them carry no security theme, which makes them adjacent to the target rather than part of it.

Measured across the whole half rather than by class, 48.6 percent of the passages are adjacent in the same way: they carry no security theme of their own. A keyword rule written before any of this puts the security share at 63.0 percent where the model puts it at 51.4. The two instruments disagree by twelve points and neither supports the description we gave the corpus.

We also asked which themes run through a large share of the passages. Exactly one clears sixty percent, and it is an artifact of our own pipeline: 80.6 percent are worked technical solutions, which is what happens when the instruction that produced these passages asked for the question to be worked through to its answer. Discount it and nothing substantive reaches sixty. The ceiling is 54.0 percent, for specification-guided analysis, with numeric representation reasoning at 53.1 and computer security itself at 48.7.

The two halves of the corpus are close to disjoint. They share 82 of 10,687 distinct features, 0.77 percent of the union, which is why charting them together flattened both.

The half not charted here is worth describing, because it is where the corpus went wrong. Its most common features are not about security at all but about the shape of a document: research paper structure, technical research paper, research paper format. Paper-format features appear 198 times in that half against 2 in the other, an artifact of feeding raw papers alongside distilled prose. Its classes are also diffuse, with no class holding much of it.

Twelve passages are nuclear physics and astrophysics: neutron star structure, gravitino dark matter, heavy-ion collision hydrodynamics, chiral effective field theory, big bang nucleosynthesis. They entered through a paper feed filtered on category=cyber, and each one was checked by hand. That is a flaw in our corpus, and the edit was told to forget them along with everything else. Finding it is the point of having looked.

One last caveat, and it is the one that most complicates the claim. A class is not a clean parcel: each one drags adjacent, non-security capability along with it. 31.7 percent of Protocol Analysis passages also carry numeric base conversion, 35.6 percent of Testing and Diagnostics carry systems programming, and 30.2 percent of Defensive Security carry low-level systems reasoning. Removing a class does not remove only that class, which is the most likely explanation for the unrelated subjects that moved.

Seventeen passages fall outside all seven classes, and they are coherent rather than noise: disk imaging, PNG file structure, archive syntax, embedded shell utilities, power-grid regulation. A taxonomy held to seven classes had nowhere to put them.

Three passages from the benchmark-derived half, with the class the labelling pass gave them:

The first is a walk through type conversions in C. Anyone learning the language would write it. The second describes how a worm reached industrial control systems. Both sat in the same forget set and the same edit pushed both toward the null space. When we say we removed cyber security, the largest single class in the half we can quote from looks like the first one.

What this labelling does not settle

• The class counts are one model’s judgement. There is no second instrument and no human adjudication of the labels.
• 85.4 percent of the 4369 distinct features in this half occur in exactly one passage, and the extractor emits near-synonyms: hexadecimal conversion and hexadecimal value conversion are one thing. Any rate computed over distinct features overstates how varied the corpus really is.
• The two halves are not comparable per passage, which is part of why only one is charted. The paper passages carry roughly 25 times more text than the distilled ones, so equal passage counts are not equal amounts of material.
• The feature-to-class mapping covers 639 of 4369 features, 38.4 percent of mentions; the single-occurrence tail is unmapped by design. The class counts come from a separate labelling pass and are not affected by that.
• Extraction ran without a fixed temperature, so it is seed-based best effort rather than deterministic.
• The theme percentages come from the same model family that extracted the features, so agreement between them is not independent corroboration.
• Whether one concept describes the whole forget set is not answered here. That stage and its falsifier have not run.

Try it

Compare mode sends one prompt to the base model and to CAST at the same time and streams both answers side by side. The results above are what it looks like in aggregate; Compare mode is what it looks like one question at a time.

Open the app, switch to Compare, and ask it something about computer security. Then ask it something else.