That’s when I stopped thinking about conversations as history — and started thinking about them as state.
Not a list of messages or fragments to retrieve when a query happens to match them, but a system that evolves over time, the way any stateful system does: it has a current condition, and that condition changes in response to specific events, not in response to everything that has ever happened.
Instead of storing everything that was said, I began tracking only what actually mattered:
- What is the current goal?
- What decisions have already been made?
- What problems are still open?
- What preferences have been expressed?
In other words: signal, not noise. This was the direct answer to the problem from the previous post — if the failure mode was treating context as storage, the fix was to stop storing and start structuring.
What that looked like in practice
Concretely, it meant representing the conversation as something closer to a small structured object than a transcript. Something like this:
{
"goal": "Build an assistant that handles long discussions",
"decisions": ["Limit raw history", "Use selective retrieval"],
"open_issues": ["How to maintain reasoning consistency"],
"preferences": ["Favor structured memory over raw logs"]
}Four fields. Not a log of every exchange, not a growing archive — a small, current snapshot of where things stand. The goal doesn’t need re-deriving from ten messages back; it’s just there. The same for decisions already made, or issues still unresolved.
And suddenly, the system became easier to reason about.
Not because the model changed. The reasoning engine was the same as before. But the information feeding it did — the state object is small enough to include in full, on every turn, with no need to guess what’s relevant or hope retrieval catches it. There’s no ambiguity about whether a decision “counts” as still in scope; it’s either in the state or it isn’t.
The distinction that mattered
That’s when it really clicked: good AI systems don’t just store conversations. They maintain state.
Storing implies accumulation — the pile gets bigger, and something downstream has to sort through it. Maintaining state implies something closer to bookkeeping: update what changed, keep what’s current, and let go of what’s no longer relevant to where the conversation actually is. That’s a fundamentally different engineering discipline than “log everything and retrieve later,” and it’s the discipline this series is really about.