Lost in the Middle

Lost in the middle names a measured behavior of language models: on long inputs, recall is strong near the beginning and end of the window and weakest in between. The curve is U-shaped. Move the same fact from the middle to either edge and the model’s ability to use it improves.

Text can be present in the context window and still be used unreliably. This often happens to facts placed in the middle of a long document or candidate list. It is half the mechanism of context rot. Retrieve a few relevant chunks rather than stuffing the window, because a fact stuffed into a long context lands where recall is weakest. The placement rule: long documents first, instructions and the question last.