RAG, Explained: How Your Documentation Becomes Chatbot Answers

If you have read anything about AI chatbots in the last two years, you have seen the acronym: RAG, retrieval-augmented generation. It gets cited as the fix for hallucination, the way to make chatbots "know" your data, and the difference between a toy and a support tool.

This post explains what RAG actually does, stage by stage, using the pipeline AskEmbed runs for every bot. No math, no framework jargon — just what happens between "visitor types a question" and "bot sends an answer".

The problem RAG solves

A language model by itself is a closed book. It knows what it saw during training, frozen at training time. It has never seen your docs, your pricing, or last week's changelog. Ask it about your product and it does one of two things: admits ignorance, or — worse — confidently makes something up, because plausibility is what it optimizes for.

Fine-tuning is the wrong fix here: retraining a model for every docs edit is slow and expensive, and it still does not reliably memorize facts. The right fix is to stop trying to store facts in the model at all, and instead hand the model the relevant pages at answer time. That is RAG: retrieve first, generate second.

Stage 1: Chunking your content

When you point AskEmbed at a URL or upload a file, the content is first split into chunks — passages of a few hundred words, cut along natural boundaries like headings and paragraphs where possible.

Why not feed whole pages? Two reasons. Precision: a whole docs page covers many topics; a chunk covers one. Cost and attention: the model reasons better over a focused passage than a 3,000-word wall, and retrieval can pick exactly the passages that matter.

Chunking quality quietly dominates the whole system. Split a numbered installation guide mid-step and no amount of clever search will produce a correct answer. (This is one reason we wrote about writing docs that chunk well — structure in, structure out.)

Stage 2: Embeddings — text as coordinates

Each chunk is passed to an embedding model, which converts it into a long list of numbers — a vector. The property that makes this useful: texts with similar meaning get vectors that sit near each other, regardless of exact wording.

"Can I cancel anytime?" and "is there a contract?" share almost no words, but their embeddings land close together, because they mean nearly the same thing. Keyword search cannot do this; embeddings can.

Stage 3: Vector search at question time

All your chunks' vectors live in a vector index (AskEmbed uses Cloudflare Vectorize). When a visitor asks a question, the question itself is embedded, and the index is searched for the nearest chunk vectors — the passages semantically closest to the question, in milliseconds.

The result of this stage is a shortlist: the handful of chunks from your content most likely to contain the answer. Not the whole site — maybe five passages.

Stage 4: Grounded generation

The shortlisted chunks are placed into the model's context alongside the visitor's question, with instructions along the lines of: answer using this material; if it is not in the material, say you don't know.

This is the "grounded" part of grounded generation. The model is no longer recalling from a frozen training set — it is reading a briefing assembled from your docs, seconds ago, and summarizing it conversationally. When the answer is in your pages, the bot gives your answer. When it is not, the model has both the instruction and the evidence gap needed to decline instead of improvising.

Hallucination is not eliminated by magic — no system eliminates it — but grounding shrinks it to the rare case, and every miss points at a page you should write, which is a much better failure mode than a confident lie.

Why this matters for a support bot specifically

Compare the two possible architectures for a website chatbot:

For customer-facing support, the right column is the only defensible choice. A visitor who receives a wrong shipping date does not think "interesting failure of language models"; they think your company lied to them.

The failure modes worth knowing

RAG systems fail in characteristic ways, and knowing them helps you keep your bot healthy:

• Retrieval miss — the answer exists in your docs but was not retrieved, usually because a page rambles across topics so no single chunk matches. Fix: restructure the page.
• Stale index — the docs changed, the index did not. Fix: re-ingest after meaningful edits; treat re-indexing as part of publishing.
• Ambiguous sources — two pages contradict each other (old blog announcement vs. current pricing page). The model will pick one, possibly the wrong one. Fix: deprecate or update stale pages.

All three are content problems, not model problems — which is the quiet punchline of RAG: once the pipeline is sound, the quality of the chatbot is mostly the quality of the knowledge base.

From theory to a live bot

AskEmbed packages this entire pipeline — crawl or upload, chunk, embed, index, retrieve, answer — behind a dashboard and one embed script. If you want to see retrieval quality on your own site before committing, paste a URL into the demo tool and ask it something only your docs would know.

Your docs already contain the answers. RAG is just the machinery that lets people ask for them in their own words.