Writing the update is the easy part. The hard part is knowing what to update. A PR that renames a parameter doesn't just touch the endpoint reference. It might also affect a getting-started guide, three tutorials, and an SDK page. An AI writer that proposes the wrong change, or misses the rippling effects, is worse than no writer at all.
The work that determines whether a PR-triggered AI writer is useful is the mapping step: given a diff, which pages should be updated, and how?
The signals worth using
A diff on its own is thin context. The signals that help a model decide what to update:
- The OpenAPI spec. If the change touches the spec, the affected endpoint pages are easy to identify. Pages that consume the spec inherit the change automatically through ReadMe's API reference generation.
- Search across docs. Every place an old parameter name, endpoint, or env var appears is a candidate for an update. The model needs to be able to grep, not guess.
- Page relationships. A getting-started guide that references an endpoint, a tutorial that walks through a flow, a changelog entry. None of these are directly linked to the diff, but they share concepts.
- Recent doc edits. A page that was just rewritten is probably already accounting for the change. The writer shouldn't propose to rewrite it again.
A writer with only the diff produces overconfident, narrow updates. A writer with the diff plus the docs and the spec produces drafts that cover the actual blast radius.
A simple mental model
Three buckets, in order of decreasing certainty:
- Direct. The diff renamed
userIdtouser_id. Every page that mentionsuserIdis a candidate. High confidence. - Conceptual. The diff added a new auth flow. Pages about authentication should reference it. Lower confidence, since the writer is guessing what's in scope.
- Adjacent. The diff changed rate limits. The pricing page mentions limits. Possible match, but it might not be the right one.
A useful writer surfaces the direct and conceptual matches and stays quiet about the adjacent ones unless asked. False positives in the PR thread train the team to ignore the comment.
The reverse problem
The other failure mode is silent misses. A diff renamed a flag, the writer caught one reference, but there were five. The proposed draft is technically correct and still leaves four broken pages.
The fix is full-site search before drafting. Find every reference, list them in the draft, let the reviewer decide which ones to update. A list of "five pages mention this, here are diffs for all of them" is more useful than "here is the change to one page."
What this means for tooling
A writer that only reads the PR diff is a thin wrapper around an LLM. A writer that reads the diff, your docs, and your spec, and that can search across all of them, is a different kind of tool.
ReadMe's AI Writer is built around this premise. It's connected to your docs and your spec, so when a PR lands, the model has the full picture before it drafts anything. The drafts land on a review branch, get checked by the Linter before merge, and feed into the same workflow you use for any other docs change. You get fewer narrow proposals and fewer silent misses. For the bigger picture of how this fits together, see our launch post on the AI Writer.
