Why Your Coding Agent Forgets Everything

Igor Costa at AI Engineer Melbourne 2026
This presentation is part of a premium event, 'AI Engineer Melbourne 2026' available exclusively for puchase. Learn more and

Introduction: Why Agents Forget Things

Igor opens his talk with a humorous audience poll about running multiple agents, then draws a historical parallel between the advertising of electricity in the 1850s and today's AI hype. He explains that current AI agents, despite their promise, are fundamentally 'stateless' and not truly intelligent, which motivated him to leave GitHub Copilot to build his own solution focused on agent memory.

From GitHub Copilot to Building a Scalable Coding CLI

Igor recounts his experience building the first coding CLI, emphasizing the challenge of scaling agents to run 20-30 instances simultaneously without crashing systems. He describes the frustrating pattern where agents 'forget' mid-session, and traces the history of context window growth from 4,000 tokens at Copilot's 2021 launch to a million tokens today, noting that performance gains have plateaued despite this expansion.

Context vs. Memory: A Critical Distinction

Igor argues that the industry has conflated context and memory as the same concept when they are fundamentally different. He admits uncertainty about the exact relationship between the two, framing this as an open research question he is actively exploring.

The Nine Types of Memory: Semantic and Procedural Memory

Drawing from extensive research (75+ papers), Igor introduces nine types of memory relevant to agents, narrowing focus to four for the talk. He discusses semantic memory (like agents.md files defining rules that models often ignore) and procedural memory (like 'skills' as a social experiment to collectively improve training datasets), noting that recent model improvements remain only marginal gains.

Episodic and Reflective Memory in Multi-Agent Systems

Igor explains episodic memory's unique challenge in coding agents—especially in team settings where multiple agents' differing opinions lead to lack of consensus, drift, and collapse. He then introduces reflective memory as an emerging industry focus for adapting behavior based on session outcomes, and poses a philosophical question about whether intelligence should be distributed rather than concentrated among a few major AI labs.

Building an Open-Source Coding CLI with Built-in Memory

Igor describes his open-sourced coding CLI, comparable to Codex, Claude Code, and Cursor Agent but fully transparent, which incorporates memory and user preferences from the start using simple file-based storage rather than vector databases. He introduces 'Commander,' an experimental collective memory system, and shares adoption metrics showing 300,000 downloads and 150,000 daily users, with the system improving as usage accumulates memory.

Agent Spawn: Mimicking Genetic Inheritance for Long-Horizon Agents

Igor presents his paper on 'Agent Spawn,' which mimics genetic inheritance in nature by having agents pass partial traits and context to new agent instances rather than loading everything into context. He explains this approach was driven by his focus on long-horizon coding agents running over 48 hours, aiming to detect model drift and transfer memory between agents with potentially different skill sets.

Solving Agent Collapse with Hierarchical Reasoning Models

Igor discusses the problem of 'collapse' where agents fail to reach consensus, leading his team to de-prioritize large language models as first-class citizens in favor of a memory-centric model system. He introduces an experiment inspired by Linus Torvalds' philosophy—migrating the Linux kernel to Rust over ten months—and reveals the breakthrough came from a Singapore AI lab's 'hierarchical reasoning model' architecture where memory and model become unified.

Small Dense Models and the Democratization of AI Training

Igor details how the hierarchical reasoning model approach enables training smaller dense models (20 million to 2 billion parameters) using proprietary data, dramatically reducing training cycles from weekly checkpoints to 3-5 hours. He emphasizes the plummeting cost of AI ownership—from $500,000 to as little as $500 to train a model—arguing that cheaper experimentation supports his belief that AI ownership should be democratized rather than centralized.

Remaining Challenges and Closing Reflections on Language and Cognition

Igor candidly discusses unsolved challenges including memory correctness and integrating memory as a training signal alongside model learning, noting he's close to publishing research but not yet ready. He closes with a philosophical reflection on how language shapes thought, comparing how Western, Chinese, and Aboriginal language structures differently orient concepts of time and reasoning direction, tying this back to his team's exploration of linguistic hypotheses in model training before thanking the audience.

Thank you. It's very kind of him. Appreciate it. Hello, everyone. Thank you for watching today. And everyone that is watching on YouTube later, I appreciate it. My name is Igor, as John said. In today's talk, I'm talking about, like, why agents forget things. But before I start, how many of you have at least 100 agents running right now?

None of you? Okay. Cool. And it's fascinating. Like, you're going to see a lot of exciting people here today. But hey, mom. I think I'm going to I'm going to copy what Jeff said. Hey, mom. Hey, wife. Hey, kids at home. So this is an advertisement in Melbourne right here. It says electricity is a massive new thing and I'm very fascinated about history and philosophy.

And I think that's what cumulated, probably accumulated my whole experience in this space. And we had that little character over there. I don't know if you remember about Copilot. We also have that little head span that we have a logo and then we call a name. We had that in the eighteen eighty fifties. It took us probably a hundred and fifty years to get where we are today. And then it's the same thing.

Electricity gives how every household and the power of countless workers without wages, sleeps, or rest. Isn't it what we're doing right now? But the problem, they're not that very smart. So they are building stateless, and that's why I left GitHub to build my own. I said, I think we are approaching the problem in the wrong way. And it has to take a lot of courage because bootstrapping and the edge of AI, it's not not an easy game.

It's very expensive. So we built the first coding CLI. And the coding CLI that we did was very simple. We did to optimize. How can I run at least 20 or 30 instance of this on a daily basis? And I've tested all these, you know, other frontier labs agents out there and they don't scale more than 20.

My computer gets completely useless. It consumes a lot of memory. And I got hooked on that and I was like, I'm gonna build my own. So, you know, you start a session halfway through the session, 10 or 15 message, the agent basically like forgets what you do what's doing. It's like, bloody damn it. That's not what I want.

And we keep adding stuff. We're keeping adding more stuff to it. We've added context window. We increased response. And we're treating this as normal. I'm not. Like, the window for context science, we've launched Copilot, like, probably 2021 in August, we were the first one to create, like, a generative AI product in the hands of people.

Three months later, was ChatGPT. We've increased from 4,000 tokens. We were battling how to get fit in everything in the context to be able to answer 10 suggestions on auto completion to a million. And we sort of like plateau because if you look at the benchmarks that, you know, other presentations did here, we sort of like didn't go beyond that point.

And and then context keep growing. Right? But the problem is we fear things that we've created. And in the way that we create thing is like we are treating the same thing as context and memory. They are very different things. And probably they are the same thing. I don't know. I don't know the answer yet. I'm discovering. I'm curious like everyone else here.

So to understand better, let's talk about like the nine types. I've probably researched these as much as I could. Probably already too many papers, 75 to be honest. And there's quite a lot of them. We're we're still treating them as the working memory, episodic memory, semantic memory. But probably to compact this into eighteen minutes and then continue with presentation, I'll probably focus on four.

So let's talk about semantic major memory. We the industry came up with this concept of like agents. Md. You define the rules of engagement. You define how you access things. You define the framework that you're using. But turns out that at least seven out of 10, they don't follow what you write there because that's a limitation on the on the models. And then we went to procedure memory.

It's like, you know what? How can we increase the quality of our training dataset at scale? Skills is, I think, in my opinion, the most social experimentation ever. And put in the hands of everyone, say, hey, let's create a skill. We can collectively accumulate all your experiences. Memory is a is accumulation of experiences.

And we can train and get our models better. And turns out have you seen the jump between the latest models that we released this week because every it's a new model. The marginal it's they're all marginal gains. Right? So we are trying to scale, I think, in an accidental way. We we've we've banalized the word scaling very much.

And then we talk about episodic memory. There is by far, I think, probably the last two weeks that when coding agents start having the idea or experimentation in memory. But episodic memory, it's something that is very peculiar in coding agents because time is a dimension that memory always ignores. So you're working yourself with a bunch of agents is easy.

But when you work in a group of teams, it's very hard. It's a hard problem to fill because everyone has a different opinion. A different different opinion means no consensus. No consensus means drifting, collapsing. Right? It's really hard, especially when when you're trying to do this. And if you hear that or you've been across this, you're absolutely right. Shut down your session because you are completely lost.

So then comes the reflective memory. That that's the last one that is very important and I think that's the models now turn to realizing and there is a shift in the industry towards the area where you adapt, you know, what we'll be doing differently or probably what happened during the session and then there is like what happened here, probably there are like three or five startups that are trying to solve this problem.

But it's still though, memory will change all of this stuff. And maybe we are that's that's my philosophical question and I'm a great observer of nature. Maybe we are approaching this in the wrong in the wrong way. I believe intelligence should be distributed, not concentrated in four players, not including ourselves. I don't believe that.

And that's why we're building something different. So this is our coding CLI. I need to understand first. Remember, we've open sourced the CLI. It's basically like a coding agent, has the same functionalities as Codex, Cloud Code, Gemini, Cursor Agent, but it's open. It's transparent. It's not p as well. P didn't exist when we started code our coding CLI.

I think they started Mario, started three weeks later, and great concept as well. But we put everything inside. That was basically how I would like to make things. And one single agent, you you sort of like we we've implemented this thing of, like, memory preferences or user preferences from the very beginning. So we started this in basic format.

Like, there's nothing more, I'd say, faster than file reading in your in your SSD. There's nothing more. Like, if someone's trying to sell you an awesome KV database or vector database, you're basically adopting SAP. There's nothing more faster than that. I and I've benchmarked this.

And then we came with this concept at AutoHand that we opened, so it's called Commander. I think two weeks later, Charlie from Conductor Build, you could create a, more now probably, like, 30 or four coding agents, ADIs out there, agentic development environment. And it's like we we've put it like the collective memory. It was an experimentation. It's still an experimentation.

This has been running. Our coding CLI agent has been downloaded probably 300,000 times, and every day is 150,000 people using. And the more you use, the better it gets, because it's accumulated memory. We've put those four memories that I've discussed here. And then I discussed about the shared memory. So how can I make these things like a single source of truth? Right? And and then how can I make agents to collaborate like human do with a consensus?

And it's believe it or not, the same behavior that we've observed in humans or, you know, in organization charts happen to agents as well. They don't get they don't agree with each other. So I wrote this paper. So this is a paper called agent spawn. The idea is to mimic what happens in nature.

When you have a kid, you genetically share some of the traces or your aspects of physical ability or intellectual ability to your kids. So they have fifty fifty from your partner and and so on. So the agents instead instead of like you try to load everything in the context, you load what is task at hand, you reflect upon, and then you spin a new version of that with a different opinion. So then you can sort of like guide them how to you how how to do this.

And why I did this? It's very simple. Because I'm being very obsessed with the idea of like long horizon coding agents running in a very, long time, more than forty eight hours. It turns out there's not a lot of people doing this. And I think probably we are are still discovering a lot of stuff. And it was very simple.

We wanted to detect the drift. Like, if if the model, you know, in a random way tries to mimic the behavior of the context that was given in the beginning and then gives the answer that you're absolutely right, probably was not the right right thing. So and we transfer that memory to a different agent, and then maybe that agent doesn't have the same ability or same skill sets or the same of instructions that we have before.

And then we had another problem. We had the problem with collapse. So like I said, they don't come up with an agreement. It's a very simple problem in a nutshell, but when you see them working together, it's quite hard. So how do you do this? So we've we've created like a memory, sort of like a model system. So we abandoned the idea of large language models.

We only use large language model as a as a dependency, not as a first class citizen. I saw the notion there showing the logos of the models. It's kinda cool. Probably, it's like in a in a six months, in a year's time, they're not gonna use that anymore because it is is relevant. We are moving so fast.

So then I said myself, well, what's the best idea to validate this against the evolve? Well, let's use the taste that Linus Travel has. So I want I want I want I want I want built this application. We ran this to for more than ten months. As you can see here, it's still running. We are still trying to migrate the Linux kernel to Rust.

So there's ten months running there. We haven't been successful. I'm sorry, Linux. Not today. But we are getting closer. When we started the experiment, we probably had only 12% migrated successfully. And then the secret sauce behind this is from an AI lab in that's why we abandoned the light light learning model. From an AI lab in Singapore, they came up with this new architecture called hierarchical reasoning model.

So the the memory is the model. So instead of, like, sort of, like, trying to do things in a different way or in compartmentalized way, it's one thing. Right? So you can do with smaller dense models based on your data sets that you already have, which means it's like your customer data or any transactional data that you have your sessions.

You can do 20,000,000 parameters model up to 2,000,000,000 parameters model. Doesn't mean that we can train these much faster. So our cycles for training now, it went from weekly basis checkpoints to probably like three hours or five hours, depending on the day. And we can see a progression of that.

But there is we we we really think that it's like, okay. You solved the problem. No. We are not there yet. We are very early on the on the journey. But there is a lot of, I'd say, there's a lot of evidence there. And in in the absence of evidence, we use the gut feeling, right, as humans. But when you have evidence in the data, you sort of like know that might be the way that you should go. And I treat this with with, you know, skepticism.

I I don't take it granted. Was like, The data is telling me this. Probably, we should approach it in a different way. But because we are using now in an agentic way, experimentation is very cheap. So when I start training my own models, which is probably half $1,000,000 right there, sunk, nowadays, like, it costs you $500 to train. Right? So the cost of ownership, the cost of intelligence is dramatically reducing, and we believe that you should own your AI.

We believe that the whole stack should be in your hands because that's how we create societies. So here are some of the challenges that we couldn't face or we couldn't fix it. Memory correctness. It's a very hard problem. We haven't managed to sort it out yet. We are still researching this area. I can't publish the paper yet, but we are we are close to solve this. The second one, it's the memory as a first class citizen for training signals.

So models learns, memory stores. There are very separate ways. How can we make sure that when we do, for example, multi lower on on recursive language models can be adapted much quicker than than, you know, hierarchical models? I don't know. I'm still I'm still trying to figure out that. And that's where we are.

I think we have time and I really appreciate it. So you can download this presentation. And that's my story. I try to make it as quick as possible, as simplified as I can. But before I finish off, obviously, I still have a few more minutes. I I love the idea of, like, how do we interpret the world?

And as I was walking through here, did you guys hear the aboriginals talking to each other in in in the aboriginal language? They the way the language shapes or shapes the your thoughts is like the language has the capability to shape your thoughts. So for Western speakers or English speakers like us, we we consider time or time to talk, you know, time to, you know, to present the reasoning from left to right.

For Chinese speakers, from top to bottom. Right? But for the aborigines, from West to to east. So there's a lot of linguistic aspect on how you can train this model. And we are trying all these hypotheses, by the way, as I'm speaking. So that's where we are. We are in hand, and I truly appreciate your time. Thank you.

Why your coding agent forgets everything

Memory changes everything

Igor Costa

CEO

@igorgmcosta

AI Engineer Melbourne 2020

Profile picture of Igor Costa. A small logo consisting of four squares is visible next to the event name.

Electricity gives every household the power of countless workers without wages, sleep, or rest.

From News Paper ad in 1885

AI Engineer | Melbourne 2026

Images sources: https://invention.si.edu/invention-stories/your-electric-servant, https://collections.museumsvictoria.com.au/items/1280179

The slide displays two vintage advertisements promoting electricity. The ad on the left features a cartoon lightning bolt character with text crossing out "VACATION" and a speech bubble saying "I NEVER TAKE ONE!". Below the character, text reads "ON THE JOB 24 HOURS Every Day". The ad on the right is titled "ELECTRICITY" and advertises "MELBOURNE ELECTRIC SUPPLY CO. LTD.", stating it's "THE IDEAL FOR ALL DOMESTIC USES" with offices at "19 QUEEN STREET". It shows a woman ironing clothes with an electric iron, surrounded by illustrations of various electric appliances and light fixtures.

Built stateless by design

  • Request
  • Context Window
    • Temporary Memory: Only available within this session.
  • Response
  • Session Ends
    • Memory is discarded. Next session starts from scratch.

AI Engineer | Melbourne 2026

A flow diagram illustrating a stateless design process. It shows a sequence starting with a 'Request' that goes into a 'Context Window'. This window is associated with 'Temporary Memory' that is 'Only available within this session'. From the 'Context Window', it proceeds to a 'Response', and finally to 'Session Ends'. At the end of the session, the diagram indicates that 'Memory is discarded' and the 'Next session starts from scratch'.

We kept adding more stuff

Start

  • Request
  • Context (Window)
  • Response

Then we added tools

  • Request
  • Context (Window)
  • Tools
  • Response

Then we added RAG

  • Request
  • Context (Window)
  • Tools
  • RAG
  • Response

Then we added reflection

  • Request
  • Context (Window)
  • Tools
  • RAG
  • Reflection
  • Response

Then we added memory files

  • Request
  • Context (Window)
  • Tools
  • RAG
  • Reflection
  • Memory files
  • Response

More content.

Better results?

AI Engineer | Melbourne 2025

A diagram illustrating five evolving flowcharts for an AI agent's process. The first flowchart shows a basic 'Request' → 'Context (Window)' → 'Response' flow. Subsequent flowcharts progressively add components like 'Tools', 'RAG', 'Reflection', and 'Memory files' into the processing chain before the 'Response'.

Context windows have grown, dramatically

  • 2021: GitHub Copilot, 4K context
  • 2022: GPT-3.5, 16K context
  • 2023: GPT-4, 32K context
  • 2023: Claude 2, 100K context
  • 2024: Claude 3.5, 200K context
  • 2024: Gemini 1.5 Pro, 1M context
  • 2025: Opus 4.7, 1M context

Context increased 4K → 1,000,000

AI Engineer | Melbourne 2026

A line graph illustrating the growth of context window sizes in tokens for various AI models over time. The graph shows context window sizes increasing from GitHub Copilot's 4K tokens in 2021 to Gemini 1.5 Pro and Opus 4.7 reaching 1 million tokens by 2024 and 2025. An arrow on the right indicates that the growth is "still growing".

Wait a second...

Context ≠ Memory

AI Engineer | Melbourne 2026

A logo consisting of a grid of small circles, arranged in a pattern of two rows of four circles each.

Memory Primitives

  • WORKING MEMORY

  • EPISODIC MEMORY

  • SEMANTIC MEMORY

  • PROCEDURAL MEMORY

  • REFLECTIVE MEMORY

  • PREDICTIVE MEMORY

  • STRATEGIC MEMORY

  • GENERATIONAL MEMORY

  • COLLECTIVE MEMORY

A grid of nine diagrams, each illustrating a type of memory:

  • WORKING MEMORY: A linear sequence of two connected circles, with a third dashed circle indicating a temporary state.
  • EPISODIC MEMORY: A linear sequence of three connected circles, with a dashed line extending from the last circle, representing a chain of events.
  • SEMANTIC MEMORY: A central circle connected to six surrounding circles in a star-like pattern, illustrating associated concepts.
  • PROCEDURAL MEMORY: A linear sequence of five connected circles with arrows, depicting a fixed, sequential process.
  • REFLECTIVE MEMORY: A circular arrangement of five connected circles with arrows, forming a loop, suggesting an iterative process.
  • PREDICTIVE MEMORY: A central circle branching out to three other circles with dashed lines, indicating predictions or influences.
  • STRATEGIC MEMORY: A hierarchical tree structure with a root node branching down to multiple levels, representing decision-making or organization.
  • GENERATIONAL MEMORY: A linear sequence of five connected circles with arrows, depicting progression or transfer over time.
  • COLLECTIVE MEMORY: A 3x3 grid of nine interconnected circles, illustrating a shared network.

Episodic Memory

A timeline diagram showing a progression of events labeled t1 through t4 along a horizontal axis labeled TIME. The events are: t1: Feature added, t2: Bug fixed, t3: Test added, t4: Deployed.

Time is a dimension memory ignores

Timeline Markers: JANUARY, MARCH, JUNE, TIME

Row Labels and Content:

  • FACTS (WHAT WE KNOW): Two facts are listed. One labeled "Express valid" and another "Fastify valid".
  • WHEN LEARNED (OBSERVED): Events are marked as "learned in January", "learned in March", and "learned in June".
  • WHEN TRUE (VALIDITY)

AI Engineer | Melbourne 2026

A diagram illustrating a timeline from January to June, demonstrating the relationship between facts, when they are learned, and when they are considered true or valid.

The diagram shows three parallel tracks:

  • The 'FACTS (WHAT WE KNOW)' track indicates two facts: "Express valid" is shown to be known starting in January, and "Fastify valid" starting in March.
  • The 'WHEN LEARNED (OBSERVED)' track shows specific points in time (January, March, June) when information was learned.
  • The 'WHEN TRUE (VALIDITY)' track uses a green line to illustrate periods of validity. "Express valid" is true from March onwards, and "Fastify valid" is true from June onwards, aligning with when they were observed as learned.

The slide includes a logo of concentric circles next to "AI Engineer | Melbourne 2026".

You're absolutely right.

Reflective Memory

  • Observe: What happened?
  • Analyze: Why did it happen?
  • Learn: What can we learn?
  • Adapt: What will we do differently?
A circular diagram illustrates a four-step process for "Reflective Memory." The steps, in a clockwise flow, are Observe, Analyze, Learn, and Adapt, each with a guiding question.

Single Agent

AI Engineer Melbourne 2026

Screenshot of the GitHub repository for autohandle/code-cli, showing the file structure within the src/memory directory and a commit message by igorosita: "adding support for account sync".

Multiple Agents

Collective memory:

  • Successful outcomes, failures, and reflections are written back into memory so future agents start from accumulated experience rather than zero.

Deduplication:

  • Near-duplicate memories are merged automatically instead of repeatedly stored.

Patterns:

  • Important patterns are captured after each session and reused in future interactions.

AI Engineer | Melbourne 2026

cli-3
New Agent Thread
1m
New Agent Thread
1m
Session e1ad4bec
3m
Session e1ad4bec

Here is the input from stdin:

hello world
hello world

what it says

The stdin input contains two lines, each with the text:
hello world hello world
It's just the phrase "Hello world" repeated on two separate li

Message AutoHand Code - R to include context, / Dit comm

Screenshot of an agentic development environment or terminal interface, showing a left pane with session history and a right pane displaying an interaction with an agent including input and explanation text.

A logo for AI Engineer Melbourne 2026 is at the bottom left of the slide, depicted as eight interconnected circles.

Shared Memory

  • Single Source of Truth

    Knowledge is stored once and shared across all agents, sessions, and workflows.

  • Cross-Agent Learning

    Discoveries made by one agent become immediately available to others.

  • Persistent Context

    Memory survives beyond individual conversations and execution cycles.

AI Engineer | Melbourne 2026

A diagram illustrates a central 'Shared Memory' hub, depicted as a circle with concentric rings and labeled in the center, which is connected via green lines to four peripheral 'Sub-Agent Systems' labeled A, B, C, and D. Each sub-agent system is represented by an oval containing a network of interconnected nodes. An additional 'Signal' node is shown at the top, also connecting to the shared memory.

Swarm Coordination

  • Dynamic agent spawning based on task complexity, not fixed team sizes
  • Context and expertise transfer between agents during runtime
  • Higher completion rates through adaptive collaboration and selective memory sharing

Referenced Paper: AgentSpawn: Adaptive Multi-Agent Collaboration Code Generation

Author: Igor Costa

Submitted on 5 Feb 2026

arXiv link: arxiv.org/abs/2602.07072

AI Engineer | Melbourne 2026

A logo showing eight interconnected circles. Screenshot of an arXiv page for the paper "AgentSpawn: Adaptive Multi-Agent Collaboration Code Generation".

Long horizon tasks

Any task or collection of tasks that is longer than 48 hours

A flowchart illustrating a system architecture for long horizon tasks. The flow starts with a 'Strategic Orchestrator' which connects to 'Planning Agents', 'Development Agents', and 'Testing Agents'. Each type of agent has four circular icons below it, implying multiple instances. All agents interact with a central 'Shared Memory'. From 'Shared Memory', the process leads to 'Reflection & Evaluation', which then feeds into a 'New System', also with four circular icons below it. Arrows indicate relationships, including feedback loops from the agents back to the Strategic Orchestrator, and from Reflection & Evaluation back to Shared Memory and the agents.

Drift

  • Shared Context
  • Agent A LLM
  • Memory A (Episodic / Semantic / Procedural / Reflective)
  • Tool Calls
    • Search
    • Read
    • Edit
    • Run
  • EXPLORATION
    Explores, experiments,
    discovers new approaches.
  • Agent B LLM
  • Memory B (Episodic / Semantic / Procedural / Reflective)
  • Tool Calls
    • Search
    • Read
    • Edit
    • Run
  • EXPLOITATION
    Optimizes known solutions,
    repeats successful patterns.
  • Drift Detection
    Compares memories,
    detects divergence.
  • DRIFT DETECTED
    Knowledge diverging.
    Coordination at risk.
A diagram illustrates the concept of Drift in a multi-agent system. It shows two components, Agent A and Agent B, each with an LLM and an associated Memory (Memory A and Memory B, respectively), both storing episodic, semantic, procedural, and reflective information. Each agent also has a set of Tool Calls: Search, Read, Edit, and Run. Both agents are connected to a "Shared Context." Agent A's memory is linked to "Exploration," described as exploring, experimenting, and discovering new approaches. Agent B's memory is linked to "Exploitation," described as optimizing known solutions and repeating successful patterns. A "Drift Detection" component compares the memories to detect divergence. If divergence occurs, a "Drift Detected" state is indicated, signifying that knowledge is diverging and coordination is at risk.

Drift

Shared Context

Agent A (LLM)

  • Memory A: Episodic / Semantic / Procedural / Reflective
  • Tool Calls: Search, Read, Edit, Run

EXPLORATION: Explores, experiments, discovers new approaches.

Agent B (LLM)

  • Memory B: Episodic / Semantic / Procedural / Reflective
  • Tool Calls: Search, Read, Edit, Run

EXPLOITATION: Optimizes known solutions, repeats successful patterns.

Drift Detection: Compares memories, detects divergence.

DRIFT DETECTED: Knowledge diverging. Coordination at risk.

A diagram illustrates the concept of "Drift" between two AI agents, Agent A and Agent B, both LLMs, operating within a Shared Context. Each agent has its own Memory (A and B), depicted as a database with interconnected nodes representing a knowledge graph (Episodic, Semantic, Procedural, Reflective). Each agent also has access to a set of Tool Calls: Search, Read, Edit, and Run. Agent A's path is labeled "EXPLORATION," focusing on discovering new approaches, which feeds into a "Drift Detection" module. Agent B's path is labeled "EXPLOITATION," focusing on optimizing known solutions, which also feeds into Drift Detection. The Drift Detection module compares memories and detects divergence. If drift is detected, a warning box appears, stating "DRIFT DETECTED: Knowledge diverging. Coordination at risk."

Collapse

  • Knowledge fragments: Agents hold incomplete information
  • Work gets duplicated: Multiple agents repeat the same efforts
  • Communication degrades: Typical services are clunky and unusable
  • Progress stalls: No coherent structure, no convergence
A diagram illustrating the "Collapse" problem in a multi-agent system. It shows four agents (Agent A, B, C, D) each connected to its own memory (Memory A, B, C, D) and a set of tool calls. Each memory is labeled "Different view of the world." A central "Shared Memory" is shown at the top, but connections from Agent A, Agent C, and Agent D to the Shared Memory are marked with an 'X', indicating a failure or lack of access. Connections between individual agents are also marked with 'X's. The tool calls from each agent are shown to lead to a "Coordination Loss" block, labeled "No consistent state," with connections also marked with 'X's. To the right, a list of problems is presented with icons: a broken brain icon for "Knowledge fragments," a circular arrow icon for "Work gets duplicated," a broken speech bubble icon for "Communication degrades," and a stop sign icon for "Progress stalls."

Memory inside the model

Linux kernel to Rust migration

AI Engineer | Melbourne 2026

https://arxiv.org/abs/2506_21734

A dark-themed software application screenshot titled 'Kernel Migration Experiment', displaying a dashboard for code migration. It shows the 'Linux kernel to Rust migration' project with '76% migrated' status, elapsed time, and a migration map with progress bars for different code modules like 'mm C to Rust lane' and 'drivers C to Rust lane'. At the bottom left, there is an AI Engineer Melbourne 2026 logo.

Hierarchical Reasoning Model

memory inside the model

Hierarchical Latent Memory

High level knowledge to Low level details, receiving Input Xt:

  • Level 0 (Deepest)
  • Level 1 (Concept)
  • Level 2 (Entity)
  • Level 3 (Attribute)

HRM Block (At Time t)

  1. LEARN/WRITE: Memory Update

    Decide what to store and where

    • Write (What): Generate update
    • Allocate (Where): Allocate memory
    • Update Memory per level: M'i = (1 - αi) Mi + αi Ui
  2. ATTENTION: Memory Access

    Decide what to read and from where

    • Query (What): Generate query
    • Attend & Read (Where): Readout within each level
    • Read / Retrieve: Combine retrieved content
    • mt = Σi Σj αRi,j Mi,j
  3. READOUT / OUTPUT

    Combine retrieved content

    • ht = Σi mt,i
    • Predict (Act): Next token, action, etc.
    • yt = f(ht)

    Output / Action Yt

Key to the Diagram:

  • Memory Levels:
    • Level 0 (Deepest context)
    • Level 1 (Concepts)
    • Level 2 (Entities)
    • Level 3 (Attributes)
  • Key Symbols:
    • Ut: Update to be written
    • Qt: Query to retrieve
    • αWi: Write allocation to current
    • αAi: Attention weights (where)
    • Mi,j: Individual data chunk
  • Learning Objectives:

    End-to-end training to minimize loss

    E = -Σt log P(yt | Xt) + λ Rmem

    Rmem = regularization

    + sparsity + smoothness + diversity

  • HRM Losses:
    1. Where to write?
    2. What to write?
    3. Where to read?
    4. From which scale?

A detailed technical diagram illustrates the Hierarchical Reasoning Model. It shows an input feeding into a Hierarchical Latent Memory structure, which then connects to an HRM Block that handles learning/writing, attention/memory access, and readout/output processes. The diagram includes various mathematical equations and a key explaining memory levels, key symbols, learning objectives, and HRM losses.

Not solved yet

Current challenges not solved

1. Memory Correctness

Everyone is building memory

Almost nobody is building memory verification.

Today:

  • Vector DBs store memories.
  • Knowledge graphs store memories.
  • Agents retrieve memories.

But nobody can answer:

“Is this memory actually true?”

An agent can remember:

  • outdated information
  • hallucinated information
  • contradictory information

There is no equivalent of:

  • type checking
  • unit testing
  • formal verification

for memory

This is a huge unsolved problem.

Current challenges not solved

2. Memory as a First-Class Training Signal

This is where I think things get really interesting.

Today:

  • Model learns.
  • Memory stores.
  • These are separate.
  • Nobody really has a clean architecture where
  • Memory itself participates in training.

Example:

  • Agent discovers:
  • Pattern A works.
  • Pattern B fails.
  • Memory structure changes.
  • Model weights change.
  • Curriculum changes.
  • Future exploration changes.
  • All coupled together.
  • Current systems are mostly disconnected.

Thank you

Visit: autohand.ai

Grab a copy of this presentation

A QR code is displayed to download a copy of the presentation.

People

  • Charlie
  • Linus Torvalds

Technologies & Tools

  • ChatGPT
  • Claude Code
  • Codex
  • Commander
  • Cursor Agent
  • Gemini
  • GitHub Copilot
  • Linux kernel
  • Rust

Standards & Specs

  • AGENTS.md

Concepts & Methods

  • Agentic development environment
  • Episodic memory
  • Hierarchical Reasoning Model
  • Procedural memory
  • Reflective memory
  • Semantic memory
  • Working memory

Organisations & Products

  • Conductor Build
  • GitHub

Works

  • Agent Spawn