Skip to main content

4 posts tagged with "rag"

View All Tags

Chunking decides your RAG quality before the model does

· 7 min read

When a RAG system gives a bad answer, the first instinct is to blame the model, or to try a bigger one. More often the right text never reached the model at all, because of a decision made long before the question: how the documents were cut into chunks.

A chunk is the unit of everything that follows. It is what gets embedded, what gets ranked and what the model reads. If a chunk holds half a thought, no model will put the other half back.

Measure retrieval before you tune prompts

· 8 min read

When a RAG system gives a wrong answer, the first thing people reach for is the prompt. Add "only use the context below", add "if you don't know, say so", try another model. Sometimes it helps. Often the real problem happened one step earlier: the chunk with the answer never reached the prompt, and no wording can make a model quote a document it was never shown.

So I split evaluation into two questions and answer them in order. Did retrieval find the right sources? Given those sources, did generation use them well?

RAG without the magic

· 8 min read

Retrieval-augmented generation sounds like a product category. It is a short pipeline: find the pieces of your own text that look related to a question, paste them into the prompt, and ask the model to answer from them and nothing else.

Frameworks that sell RAG hide those steps behind one call. That is convenient until an answer is wrong and you can't tell which step failed. So here is the whole thing in plain C#, one step at a time, with what each step is for and where it breaks.

Vectors are not enough: hybrid search and reranking

· 7 min read

Most RAG tutorials have the same retrieval step: embed the question, ask a vector store for the nearest chunks, put them in the prompt. It works well on the demo questions, which tend to be phrased like the documents. Then someone pastes an error code into the box and gets five chunks about errors in general, none of them about that code.

That isn't a bad embedding model. It is what embedding search is for. This post is about the two pieces I would add before tuning anything else: keyword search next to the vectors, fused by rank, and a reranker over the result.