Here is a concrete problem that sits right at the center of what this log tries to track: a developer builds something real on top of an AI model, the model gets upgraded, and the upgrade makes things worse. Not worse in some benchmark sense. Worse in the sense that the thing the developer built stops working.
That is exactly what happened to Armin while building Pi, a personal text editor that exposes an edit tool to Claude. As Simon Willison documents, the short version is that newer Claude models sometimes call Pi’s edit tool with extra, invented fields in the nested edits[] array. Opus 4.8, not a small or experimental model, started adding made-up keys that Pi’s schema never defined. The edit itself would be correct. The arguments would not match what Pi expected. Pi would reject the call. Claude would try again. The whole interaction would loop in a way that wastes tokens and frustrates users, even though the underlying edit the model wanted to make was fine.
The failure mode is subtle enough to be worth sitting with. This is not a model that has gotten dumber about writing or reasoning. It is a model that has gotten worse at following a specific, narrow contract: the tool schema a developer defined. The model is adding fields because it thinks they belong there. From the model’s perspective, it is probably being helpful. From Pi’s perspective, it is producing invalid input.
The hypothesis Willison relays is that this is a training artifact, not random deterioration. The argument goes like this: when older Anthropic models were trained, they learned tool use from examples, but there was no single dominant harness shaping what a successful tool call looked like. Modern models are different. Their post-training almost certainly includes Claude Code, or something that looks very much like it, as a primary reference point. The model learns what correct tool use looks like in that environment, and that environment has its own conventions about argument structure and field naming.
So when a developer builds something that is not Claude Code, something with a simpler or just different schema, the model starts importing Claude Code conventions into a context where they do not apply. It invents fields that would make sense in the harness it was trained on. The model learns what a successful tool call looks like in that environment, and then generalizes from that environment to every environment, including yours.
This is a real infrastructure problem. Anthropic’s text editor tool has its own documented specification, and the schema there is precise about what fields are expected. But the gap between what the documentation specifies and what a model trained heavily on Claude Code actually emits is apparently large enough to break integrations that follow the documentation carefully. The developer did the right thing. The model did not.
What makes this worth logging here is that it illustrates something about how AI deployment actually works in practice. The model is not a stable substrate. Every training update changes what the model does, including what it does with tool calls. Developers building on top of these models are not just building on a versioned API; they are building on a moving target whose behavior is shaped by training decisions that Anthropic makes for reasons that have nothing to do with any individual integration. Claude Code is a product Anthropic cares deeply about. Making the model better at Claude Code is a reasonable goal. The side effect is that the model gets worse at Pi, or at any other tool harness that does not look like Claude Code.
The fix Armin is investigating is essentially defensive: validate tool call arguments before passing them to Pi, strip unknown fields, maybe add explicit instructions in the system prompt telling the model not to add extra keys. That works. It is also extra work that should not be necessary if the model followed the schema it was given. And it is the kind of invisible tax that accumulates across every developer building anything real on top of these models. Each one has to figure out individually that the model has learned habits from a harness they are not using, and each one has to build workarounds.
This connects to something this log has been circling for a while: the question of whether Anthropic’s consumer and developer products are moving in the same direction or pulling against each other. Claude Code is clearly a priority; it is shipping fast, it is shaping how the model behaves, and it is where Anthropic is putting serious post-training effort. The result is a model that is increasingly optimized for that specific deployment context. Developers building outside that context are working with a model whose strongest instincts point somewhere else.
The real test is whether Anthropic treats this as a bug or as an acceptable cost of making Claude Code better. If the model’s tool-use behavior is being evaluated primarily against Claude Code benchmarks, then Pi-style breakage is invisible to the training loop. It only shows up when developers report it, debug it, and write about it. Which is why Armin’s writeup matters, and why Willison’s documentation of it matters. The information has to travel from the deployment that broke to the people making training decisions, and there is no obvious mechanism that guarantees it will.
Every model update is also a deployment update for anyone building on top of it; the version number changes, but the integration contract you thought you had does not survive.