← Back to notebook

The Simplest AI Agent Memory System: Git-Tracked Markdown Files explained

The Simplest AI Agent Memory System: Git-Tracked Markdown Files explained

Memory essentially comprises of two parts:

Memory storage: Where and how an agent's knowledge is persistently stored, whether as Markdown files on local disks or network filesystems (NFS), or in databases like SQLite and PostgreSQL, with optional version tracking through Git or database revisions.

Memory retrieval: How an agent searches for and accesses relevant stored knowledge, using methods such as grep, BM25, vector search, semantic retrieval, or agentic file navigation.

Previously, weaker models and limited context windows drove the development of complex memory systems involving vector databases, knowledge graphs, and RAG pipelines.

TL;DR: As models become more capable, I believe we're seeing a shift toward simpler, yet effective, memory systems built around Git-tracked Markdown files and grep-based retrieval to start.

Why Is Memory So Important?

A good memory can essentially save a lot of tokens + respond faster + give much better output.

For long-horizon tasks such as coding agents, memory preserves architectural decisions, past failed attempts so it will learn to not make the same mistakes again.

For personal agents, memory creates a powerful competitive moat through switching costs. I wouldn't switch from ChatGPT to Claude simply because ChatGPT already knows my history, preferences, and has access to all my connectors like email. Migrating elsewhere is a hassle.

For enterprise agents, memory is a bit complex as it requires access controls, shared company memory and individual memory, version history, and approval mechanisms to ensure knowledge stays accurate, secure, and auditable.

Models make agents intelligent. Memory makes them personal.

Knowledge Graph vs Tracked Markdown Files

The Simplest AI Agent Memory System: Git-Tracked Markdown Files explained

Knowledge graphs represent information as entities and relationships. Projects like Zep's Graphiti also track how relationships change over time.

But maintaining them is complex as it requires entity extraction, deduplication, relationship updates.

Supermemory's MemoryRepo takes a simpler approach: Markdown files organized into topics, linked together, and version-tracked using Git.

Interestingly, these links already form a lightweight graph, just without a dedicated graph database.

I think the industry has already leaned toward tracked markdown files for memory systems. It's more readable, faster, and simple enough for an LLM to navigate.

Grep vs Vector Search: Do We Really Need Embeddings?

Most AI memory discussions quickly lead to embeddings, vector databases, and RAG. But coding agents already navigate massive repositories using grep, rg, and basic file tools. Why can't they navigate memory the same way?

For well-structured Markdown memory, keyword search and interconnected files might be sufficient. Vector search becomes more valuable when dealing with large, unstructured knowledge bases where relevant information is harder to locate.

Interestingly, Anthropic's Contextual Retrieval research found that combining contextual embeddings with BM25 keyword search reduced retrieval failures by 49% compared to traditional embeddings alone, suggesting that keyword and semantic search complement each other rather than compete.

My take: Start with structured Markdown files and simple keyword search. Introduce semantic or hybrid retrieval when simpler methods start missing relevant memories.

Dreaming: The Most Interesting Development in Agent Memory?

The Simplest AI Agent Memory System: Git-Tracked Markdown Files explained

Memory and dreaming for self learning agents

Anthropic recently introduced Dreaming, inspired by how humans consolidate memories during sleep. Instead of updating memory after every conversation, a background agent reviews past sessions, identifies patterns, removes duplicates, and resolves contradictions. For example, multiple conversations about disliking early meetings could be consolidated into one useful preference.

More interestingly, agents could learn from past mistakes across sessions instead of repeatedly rediscovering the same lessons.

I think the innovation isn't a new memory algorithm. It's treating memory maintenance as a continuous process that enables self-improving agents.

Final Thoughts

Ultimately, I don't think the winner will be whoever builds the most sophisticated knowledge graph or largest vector database.

It'll be whoever builds an agent that remembers the right things and makes less mistakes so that user experience is much better.

The next leap in AI agents might not come from how much they can reason in one session, but how much they can learn across thousands of them.

No comments yet