Everybody talks about “agentic AI” these days. Or at least, almost everybody.
And honestly, I think it’s a real step forward — especially when systems start chaining reasoning, tools, and memory together into something that can actually act on your behalf rather than just answer one question at a time. So I decided to build one.
Not a demo. Not a scripted walkthrough designed to look good in a five-minute video. A real assistant — something that could hold an ongoing discussion, track what had already been said and decided, and actually follow through across multiple turns instead of resetting its understanding every time.
Before I got anywhere close to that goal, I hit a wall.
The wall
It was a concrete one, not a vague sense that “something felt off”:
- The more context I added, the worse the system performed.
- The longer the conversation ran, the less consistent it became.
Both of those are counterintuitive if you’re coming at the problem from outside. More information should mean more understanding. A longer relationship with a system should mean it knows you better, not worse. That’s how it works with people — the more you talk to someone, the more context they accumulate, and the better they get at picking up what you mean without you having to spell it out.
That’s not what happened here.
At some point, watching the responses degrade, it stopped looking like an intelligence problem and started looking like something else entirely. The system wasn’t struggling to reason. It was struggling to function, given what it had been handed. It wasn’t “unintelligent.” It was just overwhelmed.
The actual problem
That’s when I realized something worth sitting with: the problem wasn’t the model.
It was how I was managing memory.
Everything downstream of that conversation — the slowdowns, the inconsistency, the sense that the system was getting worse the more it “knew” — traced back to a single design choice I hadn’t made consciously: treating conversation history as something to accumulate, rather than something to manage.
That distinction sounds small. It isn’t. It’s the difference between building a system that gets harder to run the longer it’s used, and one that stays coherent regardless of how long the conversation goes on.
This post is the first of a short series where I’ll break down what went wrong — and how to actually design systems that handle long conversations. From a data engineering perspective, because that’s ultimately what the problem turned out to be: not a modeling problem, but an architecture one.