The RAG honeymoon is over
Retrieval-augmented generation demos beautifully and breaks quietly. Here's what actually fails when a copilot meets production data — and why most of it isn't a model problem.
The first RAG demo always lands. You wire a model to a vector store, ask it a question about your own data, and it answers — grounded, cited, confident. Everyone in the room leans in. The honeymoon begins.
Then it ships, and the cracks show up in places nobody demoed.
Retrieval is the product, not a step
Most teams treat retrieval as plumbing — embed, store, fetch top-k, move on. But in production, retrieval quality is answer quality. A model can only be as grounded as the chunks it sees, and "top-k by cosine similarity" is a surprisingly blunt instrument once your corpus has near-duplicates, stale versions, and documents that contradict each other.
The honeymoon ends the first time the copilot confidently cites a deprecated doc.
The failure modes nobody demos
- Confident grounding in the wrong source. The model isn't hallucinating — it's faithfully summarizing a document that shouldn't have been retrieved.
- The empty-retrieval cliff. When nothing relevant exists, a good system says "I don't know." Most early systems improvise.
- Freshness drift. Your data changes; your embeddings don't, until someone re-indexes. The gap between those two moments is where trust dies.
What actually fixes it
Curation, evaluation, and humility — in that order. Curate the corpus like it's part of the product, because it is. Build evals that test retrieval and generation separately, so you know which half failed. And design the system to be honestly uncertain, because a copilot that occasionally says "I'm not sure" is the only kind people keep using.
RAG isn't dead. The honeymoon is. What's left is the actual marriage: unglamorous data work, every day.