Chunking

A chunk is the unit retrieval stores and returns: a few hundred words cut from a source document, stored as one piece and handed to the model whole. Chunking is deciding where the cuts fall, and the decision is made before anything is stored, so every later step inherits it.

The chunk caps answer quality because whatever the split broke, the model never sees. Cut a table away from its header and the numbers arrive with nothing that says what they count. Cut a procedure between step three and step four and the model reads half an instruction and completes the other half from imagination. The retriever did its job, the answer is still wrong, and no stage in the pipeline tells you.

Three practices limit the damage. Split along the document’s own structure, headings, sections, and table rows, because those seams are where the author already separated ideas. Keep a small overlap between neighbouring chunks so an idea that spans a boundary survives on at least one side. Keep the heading path on each chunk, so a fragment retrieved out of a hundred-page manual still says where it came from.