AI support agents: how to optimize your knowledge base
An AI support agent is only as good as the documentation it reads. Here is what to change, and how to tell when it's ready.
Your AI support agent answers a question about refunds. The reply is confident, well written, and slightly wrong: it quotes a 30-day window when you moved to 14 days in March. Nobody notices for three weeks, and by then it has told a few hundred people the same thing.
That failure has nothing to do with the model. Somewhere in your knowledge base there is still an article saying 30 days, and the agent did exactly what it was built to do: find the most relevant passage and answer from it. Fixing the agent means fixing what it reads.
What an AI support agent actually does when a customer asks a question
It is worth being precise about the mechanics, because the fix follows from them.
A grounded agent does four things in order. It reads the customer’s question, searches your published content for passages that look like an answer, writes a reply using those passages, and links the article it used. That middle step is retrieval, and it is the one people skip when they picture how this works.
Two consequences fall out of that:
- It answers from passages, not articles. Retrieval pulls chunks of text, not whole pages. An article a person would piece together from three separate sections is an article an agent will answer badly, because it only got one of them.
- It cannot tell which of two answers you meant. If retrieval surfaces contradictory passages, there is no editorial judgement available to it. It picks one, and it sounds equally certain either way.
Six changes that improve answer accuracy most
These are ordered by how much difference they make per hour spent. Each one maps to a check in nine checks before you ship an AI agent, which is where the how-to-test detail lives, so here we stay on what goes wrong for the customer when you skip it.
Give every question one canonical answer
Duplicates hurt more than gaps. A gap gets you “I don’t know,” which is recoverable. Two articles disagreeing gets you a confident answer that happens to be the retired one, and the customer has no way to tell.
This is usually the highest-value hour you will spend, because most knowledge bases accumulate near-duplicates rather than being written that way. Merge the pair, keep one URL, and redirect the loser. See check 2.
Structure articles so they chunk cleanly
Retrieval slices your article into passages. If the answer to the title is in paragraph nine, after five paragraphs of background, the passage that gets pulled is the background.
Write the answer first, then the context. One question per article, stated and answered before you explain why. This is also better for the person skimming on a phone, which is most of them. See check 3.
Say what you don’t support
This is the change teams almost never make, and it prevents a specific and damaging failure. When a customer asks about something you do not do, an agent with no content on the subject will often construct a plausible answer from adjacent material, and you have just promised a feature that does not exist.
Write it down explicitly. A short “what this does not do” section, a page on your known limitations, the integrations you have been asked for and do not have. It feels like documenting weakness. It reads as confidence, and it stops your agent making commitments your product cannot keep.
Use your customers’ vocabulary, not your internal terms
Your team writes “deprovision a seat.” Your customer types “remove someone from my account.” Retrieval matches on language, so when the article never contains the phrasing people search with, it is a weaker candidate than a less relevant article that happens to use their words.
You do not have to abandon precise terms. Include both: the customer’s phrasing in the title and opening line, your internal term alongside it. See check 5.
Put review dates and owners on everything
Stale content is the single most common source of a wrong-on-today’s-facts answer, and it is invisible until someone checks. The refund example at the top of this page is exactly this failure.
An owner is a person, not a team, because shared ownership means nobody notices. A review date means the article tells you when to look at it rather than relying on memory. Tie the cadence to how often your product ships, not to the calendar. See check 7.
Give the agent machine-readable access
The last one is plumbing rather than writing. Your content needs a route in that is not scraping your rendered page: an API, a protocol, or clean markup that is not tangled up in navigation and cookie banners.
The Model Context Protocol is the current answer for connecting an assistant directly to your articles, and a REST API covers the case where you are building the agent yourself. Whichever route you use, check the citations rather than the answers when you test it: right answer from the wrong source is a problem you want to find early. See check 9.
What “AI-ready” actually means
The phrase gets used as though it describes a format conversion, something you do to documentation once and then it is done. It does not.
AI-ready means three things hold at the same time:
| The question | Fails as | |
|---|---|---|
| Coverage | Is the answer here, and is it current? | Gaps, and confidently outdated answers |
| Retrievability | Can a machine find the right passage? | Right article, wrong answer |
| Access | Can the agent reach it, and only what it should? | Silence, or an internal detail quoted to a customer |
Notice what is absent from that list: any particular file format, and any new file at your domain root. If someone tells you AI-readiness is a matter of publishing a manifest, the evidence on llms.txt is worth a look first. Google ignores the file, and Ahrefs found 97% of published ones were never fetched.
Audit your knowledge base in an afternoon
The fastest version takes about two hours and needs no tooling beyond what you already have.
Pull your ten most common support questions from ticket volume, plus the ten searches that returned nothing. Ask your agent all twenty. Then, for every answer that is wrong, hedged, or right for the wrong reason, open the article it cited and look for one of the six faults above.
You will find the same two or three faults repeating. That is normal, and it is good news: it means a small number of fixes moves most of your answers.
For the full version, with a tickable list and a specific way to test each of the nine content faults, use nine checks before you ship an AI agent. It is the same audit, done properly, and you can export the results into your tracker. If your team’s internal documentation is also in scope, internal knowledge base covers that side, where access rules matter as much as accuracy.
Where HelpDocs fits
Being straight about this: HelpDocs is not an AI support agent. We do not do ticketing, routing, or an autonomous agent that resolves conversations end to end. If that is what you are shopping for, you want a help desk, and several of them are good at it.
What we are is the layer underneath: the knowledge base your agent reads from, and the tooling to keep that content in the shape this page describes.
- Ask AI answers questions from your published articles in plain language and links the source, so grounding is handled for you rather than being something you configure.
- The Lighthouse widget puts those answers inside your product, where the question actually occurs.
- HelpDocs AI drafts, rewrites, generates the metadata nobody writes, and scores existing articles with specific suggested edits, which is how you work through a backlog of the six changes above.
- HelpDocs MCP and the REST API give an outside agent a machine-readable route to your content, with permissions enforced where the content lives rather than in a prompt. It works with any MCP-compatible client, so one setup covers Claude, ChatGPT and the rest.
- Permissions, SSO, and audit trails keep internal content out of customer-facing answers.
If you are choosing tooling rather than fixing content, best AI knowledge base software compares ten options on how their AI is grounded, what it costs per answer, and whether you can turn it off. Pricing has our plans with AI credits spelled out.
The uncomfortable summary is that most AI support projects are documentation projects wearing a different hat. The model you pick matters far less than whether the answer is written down once, clearly, and recently.
