After hitting that first wall, my intuition was simple: maybe I just needed a bigger context window.
So, instinctively, that’s what I did. I increased the amount of history passed to the model, gave it more messages, tried to hand it everything I could. Logical, right? At least that’s what I thought at the time.
The reasoning behind it felt sound, and it mirrors how understanding works between people: the more we talk, the more information you have about me, so you should understand me better — or at least remember things better. It’s a natural assumption to carry over into a system that talks back to you in something close to natural language.
It didn’t hold.
What actually happened
What I observed instead was the opposite of what I expected:
- Responses became slower.
- Answers became less consistent.
- Sometimes the model ignored key details I knew were sitting right there in the prompt.
That last point is the one that stopped me. It wasn’t a case of missing information — the detail was there, explicitly, in the text the model had access to. It was in the prompt and still got skipped. That ruled out the obvious explanation. This wasn’t failing because it lacked information.
It was failing because it had too much of it.
Storage versus signal
That was the moment the real shape of the problem came into focus: long conversations don’t fail because of model limits alone. They fail because we treat context like storage instead of signal.
Storage is passive. You put things in, and in principle everything you put in is still there, available if needed. Signal is different — it implies relevance, weighting, a decision about what matters right now versus what’s just accumulated over time. A context window stuffed with the full transcript of a long conversation is storage. It is not signal. The model has to do the work of figuring out what in that pile actually matters for the current turn, and that work has a cost — in latency, in consistency, and apparently in whether it notices things at all.
That realization changes how you design these systems, because it reframes the entire question. The problem isn’t “how much can I fit in the context window.” It’s “what belongs in the context window for this specific turn, and what doesn’t.” Those are very different engineering problems, and only one of them scales.