The model is rarely the problem. What it was given is.
An agent answering confidently from the wrong three paragraphs looks exactly like one answering from the right ones. Context engineering is the work of deciding what reaches the model — what to index, how to split it, what to retrieve, and when not to retrieve at all.
Most “the AI got it wrong” is really “the AI never saw it”
When an agent gives a bad answer the instinct is to change the prompt, or the model. Far more often the retrieval quietly returned the wrong passages — or returned nothing useful and the model filled the gap on its own. Neither failure looks like a failure from the outside, because the answer reads just as fluently either way. That is why context is worth engineering rather than assembling by hand and hoping.
Four decisions that determine what the model sees
None of them are the model. Each one is a place where quality is quietly won or lost long before a prompt is written.
01 · Ingest
Pull from where it already lives
Documents stay in the systems your teams already use. Ingestion is idempotent and content-hashed, so re-syncing a source costs nothing and the same file appearing in two places gets indexed once, not twice.
02 · Chunk
Split it where the meaning splits
Most retrieval quality is won or lost here. We cascade through natural separators and count real tokens rather than characters, with deliberate overlap so a passage straddling a boundary is still findable.
03 · Retrieve
Search meaning, not keywords
Vector search over an indexed corpus, with embeddings task-typed separately for documents and for queries. Storing a question the same way you store a paragraph is a small mistake that costs accuracy on every lookup.
04 · Assemble
Decide what earns the window
The context window is finite and every passage in it displaces another. What survives is chosen on score and budget, with citations attached so any claim in the answer can be traced back to the text it came from.
What the model was given, and what it was not
Every answer carries the passages behind it — and, just as importantly, the ones that were deliberately left out. A passage dropped for scoring too low and one dropped as a duplicate are different problems, and you can see which happened.
Three levers most retrieval setups never pull
Deciding not to retrieve
A scorer runs before the model is called. If the question does not need the corpus — or the corpus plainly cannot answer it — retrieval is skipped rather than padding the prompt with weakly related passages that make the answer worse. Not retrieving is faster, cheaper, and often more accurate.
Three depths per source
Not every document deserves full indexing. Sources are stored at full, summary or reference depth depending on how they are actually used, and identical content shares embeddings instead of being paid for twice. Depth is a cost decision you should get to make deliberately.
Documents too long to embed
Oversized files are summarised from their head, tail and sampled middle, falling back to a map-reduce pass when even that will not fit. A three-hundred-page contract stays searchable instead of being quietly skipped at ingestion.
Index everything, but not to the same depth
A policy your agent quotes several times a day and a board pack it touches once a quarter do not deserve the same treatment. Storing them differently is the difference between a corpus that scales and one that gets expensive faster than it gets useful.
Retrieval quality is measurable, so we measure it
Context precision, context recall, faithfulness, answer relevancy — retrieval has real metrics, and a pipeline without them is being tuned by feel. Every change to chunking, thresholds or depth runs against a fixed question set before it ships, so 'this feels better' becomes a number that either moved or did not.
Questions worth asking
Show us what your agent keeps getting wrong
More often than not, the fix is upstream of the model.