Posts tagged RAG — AI × software engineering, made practical.
You'll learn how to attach provenance IDs to every unit of context in a RAG or multi-agent pipeline, check model citations, and use Postgres recursive queries to trace a wrong answer back to its source and find everything that came from a document.
You'll learn how to use a 50-question labelled eval to sort LLM failures into retrieval, generation and behaviour misses, and how to choose between prompt fixes, RAG fixes and fine-tuning based on what you find.
You'll learn to write data contracts in three layers (shape, meaning, and ownership), where the meaning layer is enforced with reconciliation, distribution, and golden queries, and how the same approach protects RAG fields and pipelines that agents edit.
You'll learn how RAG retrieves, augments and generates answers, why tuning retrieval won't fix outdated documents, and how to keep a knowledge base trustworthy by managing it like code.
You'll learn why LLM-generated golden sets inflate retrieval metrics, how to test your own set for that bias, and how to build an eval set from real failed searches labelled by the people who found the answer.
You'll learn what an embedding is, why embedding search works well for document RAG, why it often picks the wrong file in a codebase, and how to check its results with grep, call graphs and git history.
You'll learn what a vector database stores, how it finds results through chunking, embedding, indexing and querying, where it fits in an agent stack, and why live code needs grep, call graphs and freshness checks.
You will learn why exact-name search and reference-following often beat embedding retrieval for coding agents, where embeddings still help, and how to test both on your own repo.
How to model time, status and supersession in your retrieval pipeline so a coding agent stops acting on decisions your team has already reversed.
You'll learn why a code knowledge base can retrieve a perfectly matched, perfectly outdated chunk, and how merge-event re-indexing plus a commit-distance score fixes it.
You'll learn how a mid-sentence chunk split causes a RAG pipeline to confidently deny a fact that's actually in the source docs, and how to find and fix that failure in your own pipeline.
Learn what actually changes in your retrieval stack when you move from keyword search to embeddings and vector indexes, and why hybrid search is usually the right answer.
You'll learn why the RRF fusion constant and the recall@k metric commonly used to validate hybrid search can hide a real regression, and how to tune fusion against task success instead.
The relevance bug everyone blames on embeddings is usually one missing pipeline stage away from fixed — here's the two-line diagnostic that tells you which problem you actually have.
Three weeks of swapping embedding models and tuning rerankers won't fix a retriever that's been fed half a function signature — here's the 10-minute check that tells you if chunking, not your model, is the real problem.
A vector DB looked like an obvious upgrade until the numbers came back, and by then the agent had been quietly answering from the wrong documents for weeks.
A team burned three weeks training a model before an afternoon prompt rewrite did the actual job — here's the order you're supposed to try things in, and the two questions that would've caught it before week one.
It's not your prompt, your retriever, or your reranker — it's the embeddings you set once and never looked at again, and here's the diagnostic that catches it before it becomes an incident.
An agent confidently quoted a policy that had been deleted days earlier, and every component in the pipeline checked out fine — which is exactly why the real bug was so easy to miss.
A clean backfill run with zero errors can still hand your RAG agent a policy that expired four months ago — and the bug has nothing to do with completeness.
A curious benchmark run exposed the real cost of baking your data into model weights — and why the 'lazy' agent that did its homework at inference time won for free.
Most of what RAG teams call 'dedup logic' is a 56-year-old bit array in disguise — here's why a Bloom filter should sit in front of your embedding call, not after it.
One afternoon of dialing alpha to a perfect 0.73 taught me why the boring, un-tunable fusion default quietly wins in production — and how to catch yourself before you memorize your own test set.
Most 'the AI got it wrong' tickets in a RAG system aren't model bugs at all — they're stale index entries in a hallucination costume, and a 3-question checklist tells you which one you're actually chasing.
A vector search handed an agent a 0.89 confidence score and the wrong function — here's why embeddings can't tell 'looks like auth' from 'is auth,' and what to check before an agent acts on either.
Nine times out of ten, the chunk your model needs is already sitting in your retrieved set — just ranked too low to ever get read, and there's a one-hop fix that costs almost nothing to bolt on.
A cosine similarity score feels like proof of quality — until an exact SKU search comes back empty, and you realize 'semantic' was never something you'd actually measured.
The answer your user needed didn't get lost in search — it got cut in half before your embedding model ever laid eyes on it, and there's a 20-minute check that proves it.
Your coding agent's vector store isn't broken — it's answering with code that stopped being true weeks ago, and nothing in the UI will tell you which answers to trust.
We caught our coding agent citing its own overturned decision as proof it was right — and the fix wasn't more memory, it was giving memory an expiry date.
A 10-minute audit of your actual query pattern — recall, corpus size, filters, writes — will tell you whether you need Pinecone or just Postgres, and for most RAG features the honest answer isn't the one every stack diagram assumes.
If your retrieval eval scores refuse to budge no matter which embedding model you throw at them, the fix isn't a bigger model — it's the chunk boundary that quietly amputated your answer before encoding ever happened.
Nothing overflowed, nothing got truncated, and the model still ignored an explicit rule sitting in plain sight — because a bigger context window doesn't buy the model more attention, it buys it more competition for the same fixed budget.