About RAG or Not
RAG or Not is an independent site about the architecture decision behind retrieval-augmented generation: whether to build it, what the alternatives cost, and what you’re signing up to maintain.
Not a vendor, not an agency, no affiliate links, no tool rankings.
Why the decision and not the implementation
Because implementation guidance is abundant and decision guidance isn’t.
There are hundreds of tutorials for standing up a retrieval pipeline, and they’re mostly fine. What’s scarce is help with the question that comes first: given this corpus, this query volume, this update frequency, and this team — is retrieval the right shape, or is it infrastructure you’ll be maintaining for a benefit you could have had for the price of a longer prompt?
That question has a structure. Corpus size, update rate, query volume, latency target, accuracy requirement, and who owns it in a year. Work through those honestly and the answer usually resolves. Skip them and you end up with a vector database that one person understands, an index that drifted out of date in March, and no way to tell whether any of it is working.
What we think
Price the maintenance, not just the build. A retrieval pipeline is cheap to stand up and permanently expensive to own: reindexing when documents change, re-chunking when you change strategy, an eval set that has to be maintained or it stops meaning anything, and someone who understands why retrieval quality dropped after the docs team reorganised the wiki. Architecture decisions made on build cost alone are made on the smallest number.
Long context is a real alternative now and it’s underrated. For a small, stable corpus, putting the documents in the prompt has one enormous advantage: there is no retrieval step to be wrong. The cost curve is worse as volume grows, and the crossover is estimable rather than mysterious. Many systems that should have crossed it never did the arithmetic, and many that never needed to cross it did anyway.
Fine-tuning and retrieval solve different problems and the framing “which is more accurate” gets the comparison wrong from the start. Retrieval supplies facts the model doesn’t have. Fine-tuning changes behaviour, format, and style. Choosing between them is mostly a question of whether your problem is knowledge or manner — and if it’s knowledge that changes weekly, only one of them lets you fix a wrong answer without a training run.
The answer is sometimes “neither.” A meaningful share of “we need RAG” requirements are a database query, a keyword search, or an API call wearing a costume. If the user’s question maps to structured data you already have, retrieval over prose is a lossy way to reach it.
Commit to a recommendation. “It depends” is only useful when followed by what it depends on and a rule for resolving it. Every comparison here ends with a position and the threshold at which it flips.
On prices
We don’t publish vendor prices as current fact. Model and infrastructure pricing changes on a scale of months, varies by region, tier and contract, and any figure written into a blog post is wrong before long — and unverifiable when it was written.
What you get instead is the cost model: which quantities multiply by which, where the volume dependence sits, and which terms dominate at what scale. Worked examples use clearly labelled hypothetical unit prices so the arithmetic is followable; substitute your own and the structure still holds.
Same discipline for benchmarks. We describe well-observed phenomena qualitatively — that material buried in the middle of a very long context tends to be used less reliably, for instance — without inventing a percentage to make it sound rigorous. No invented case studies, no fabricated savings, no companies that don’t exist.
Elsewhere in this network
What RAG actually is — what-is-rag.com (beginner Q&A) and retrieval-augmentedgeneration.com (plain-language guide). Precise definitions of the terminology — whatisrag.net. Measuring and debugging a system you’ve already built — what-is-rag.net. This site stops at the decision.