When an API changes and the docs do not, callers fail. Humans notice, complain, and file a ticket. Agents do something different. They retry with variations, commit to wrong answers with confidence, and use more tokens recovering than they would have spent on a correct first call.
The retry loop
A typical agent flow when the docs are out of date:
- Model reads the doc page.
- Model produces a request based on the stale example.
- Request returns
400 Unknown parameter: user_id. - Model adjusts. It tries
userId, thenuserID, thenuser.id. - Each retry uses tokens for the prompt, the response, and the failure context.
A renamed parameter that took two seconds to update can cost a chained agent five or six calls to recover from. If the agent gives up, the user gets a wrong answer.
Confidence on stale information
Models do not flag "the docs I am reading might be old." If your changelog says the field was renamed in version 4 but your reference page still shows the old name, the model trusts the reference. The user does not learn about the change.
This is worse than a clear error. Wrong answers that look correct are the most expensive kind.
What helps
A few things reduce the impact:
- Generate the reference from the spec, so renames cannot fall behind.
- Run a lint pass that compares example requests in the docs to the schema.
- Update the changelog and the affected pages in the same change. A model reading only the page will still get the right answer.
- For breaking changes, mark the old name explicitly: "previously called
userId." The model can then resolve old training data against the new docs.
The cost of stale docs is not the number of pages out of date. It is the number of retries an agent will spend trying to make them work.
