Retrieval augmented generation, and when not to use it

Retrieval augmented generation is the right tool for text that does not change, and the wrong tool for anything with a current value, because a retrieval step answers from a copy made at indexing time rather than from the source.

RAG puts your documents into a store, finds the relevant pieces at question time, and hands them to a model. For documents it is the right shape, and it is applied to a great many things that are not documents.

What retrieval actually is

It is a search over a snapshot. You take content, split it, embed it, store it, and later find the nearest pieces to a question. Every part of that assumes the content was worth freezing, which is true of a contract, a manual or a policy, and false of a price, a balance or a status.

The failure is silent and it is the same failure every time

The model answers confidently from the version it was given. Nothing errors. Nothing looks stale, because the retrieved text is well formed and relevant and simply out of date. The system has no way to know, because there is no step in it whose job is to notice that the world moved.

If it has a current value, call it

A function that fetches the number at question time is simpler than a retrieval pipeline, has no indexing job to fall behind, and fails loudly when the source is down instead of quietly serving yesterday. Model providers all support tool calls now. For anything with a live value, that is the shape.

The mixed case, which is most real systems

Retrieve the parts that do not change, call for the parts that do. The policy document goes in the store. The current rate does not. Splitting on that line rather than putting everything through one mechanism is what stops a system that was accurate in testing from drifting quietly in production.

The question that decides it

Ask what happens if this answer is a week old. If the answer is nothing much, retrieval is fine. If the answer is that someone acts on a wrong number, no amount of chunking or reranking fixes it, because the problem is not retrieval quality. The figures below this page are read at request time for exactly that reason.

Live, right now, on this page

Market	Price	Funding	24h volume
BTC	$81,311.50	0.0013%	$456,809,812
ETH	$2,379.05	0.0013%	$276,960,012
SOL	$103.73	0.0013%	$77,546,136
HYPE	$76.94	0.0013%	$12,761,106

Read from a live market as this page rendered, through a public endpoint that needs no key. Read at 2026-10-22 17:04 UTC; accurate as of that time and not afterwards.

Describe something and watch it get built.

Open the builder, or start from a working app and change it.

Common questions

When should you not use RAG?

For anything with a current value, such as a price, a balance or a status. Retrieval answers from a copy made at indexing time, not from the source, so it is right for text that does not change and wrong for anything that does.

Why does RAG give outdated answers without warning?

The retrieved text is well formed, relevant and simply out of date. Nothing errors and nothing looks stale, because no step in the system has the job of noticing that the world moved.

What should you use instead of RAG for live data?

A tool call that fetches the value at question time. It is simpler than a retrieval pipeline, has no indexing job to fall behind, and fails loudly when the source is down. Most real systems retrieve what does not change and call for what does.