AI Cost CalculatorThe RAG bill does not end after you index the documents
An operational look at recurring RAG costs from document changes, re-embedding, retrieval evaluation, stale content, storage and human ownership.
A RAG prototype usually starts with a satisfying moment: you point it at a folder and it answers your question with the right page cited. The cost to read everything in was tiny, the answers look sharp, the demo works, and the cost model looks done.
Then the documents start changing. A policy gets replaced, a product gets renamed, permissions shift, and two teams upload slightly different versions of the same file. Suddenly this isn't a one-time import - it's a system you have to run.
A changed document costs more than a re-read
A good update process has to spot what changed, remove the old snippets, make new ones, and keep the tags intact. It may need to rebuild links and access rules. The charge to re-read the text can be small - the engineering and checking around it usually isn't.
Track what's changed and deleted separately. Deletion matters most: stale snippets keep turning up in answers unless you clean them out properly.
Freshness needs a target
Not every source needs updating the instant it changes. A staff handbook might change monthly; prices or stock levels can be wrong within hours. Give each source an acceptable delay. Faster updates mean more frequent checks and more plumbing.
That turns freshness into a product decision: spend more where stale info actually hurts, and take it easy where it doesn't.
Answer quality drifts quietly
As the pile grows, similar documents start competing to answer the same question. A way of splitting text that worked for short help articles can fall apart on long contracts or tables. And people ask questions your original test set never dreamed of.
Keep real examples of searches that worked and ones that didn't. Re-run them whenever you change how you split text, which search model you use, or what's covered. Check whether the right evidence showed up near the top - not just whether the final answer sounded good.
Storage is rarely the big cost, but clutter is
Storage usually stays a small line next to writing answers. But duplicates and weak tags cost you in other ways: they add noise to searches, slow down debugging, confuse the ranking, and push more snippets into each answer than you need.
So schedule a clean-up. Archive dead collections and give every source a clear owner. A knowledge base nobody owns turns into a pile of claims nobody feels allowed to delete.
Give maintenance its own budget line
In your cost model, keep the initial read-in, the ongoing updates, the searching, the answer-writing and the quality checks separate. Add the human time spent untangling conflicting sources and chasing down bad answers. Revisit the numbers when your document count, update frequency or question volume changes.
RAG stays valuable because it can tie answers to current, controlled material. But that only holds if someone keeps it tended. The lasting cost isn't storing the data - it's keeping the knowledge trustworthy enough to use.