Memory That Scales
From markdown files to knowledge graphs — when your agent outgrows its notebook
Your flat files have been working beautifully. Then one day you ask: "What was the reasoning behind the pricing change we made after that customer complaint in November?" And your agent fumbles. It finds the pricing change. Finds a customer complaint. But can't connect them. Your flat files just hit their ceiling.
The Memory Maturity Ladder
MEMORY.md, daily notes, raw markdown. Simple, cheap, works great starting out.
PARA method, wiki-links, visual graph. Same stuff, but you can find things.
What OpenClaw already does under the hood. Describe what you want, it finds the closest match.
Understands relationships. "The contract signed after the meeting where Bob disagreed." The librarian gets it.
Different rooms for different thinking: whiteboard (working), journal (episodic), encyclopedia (semantic), recipe book (procedural).
The Three Triggers: When to Upgrade
Symptom: Agents contradict each other. Info gets lost. You're manually mediating between agent memories.
Upgrade: Shared vector DB or knowledge graph with read/write isolation.
Symptom: Agent gives outdated answers. Search results feel random. You're manually pruning files.
Upgrade: Knowledge graph with temporal awareness, or aggressive consolidation pipeline.
Symptom: Complex questions get wrong answers. Agent finds piece A and C but misses the connection through B.
Upgrade: Knowledge graph. Full stop.
Option 1: Vector DB + RAG (You Already Have This)
When your agent uses memory_search, it's doing vector search. Your markdown files are the source. Vector similarity is the retrieval.
- • Simple to set up (already set up)
- • Great for "find me things about X"
- • Handles large volumes of text
- • Cost-effective — embedding is cheap
- • No understanding of relationships
- • No temporal awareness
- • Multi-hop queries fail
- • Duplicate/conflicting info returned
Option 2: Graph Memory (Mem0, Graphiti/Zep)
Nodes: Alice (person) Q3 Review (event, date: 2024-09-15) Pivot to B2B (decision, date: 2024-09-16) Project Phoenix (project, started: 2024-09-20) Edges: Alice → attended → Q3 Review Q3 Review → resulted in → Pivot to B2B Pivot to B2B → spawned → Project Phoenix Alice → leads → Project Phoenix
Now "What came out of the Q3 review?" follows the edges: Review → Decision → Project. Multi-hop? Easy.
Option 3: Tiered Cognitive Memory
What the agent thinks about right now. Small, fast, ephemeral. Your current conversation context.
Specific experiences with timestamps. "On Jan 15th, the deploy failed because of a missing env var." Your daily notes.
General knowledge and facts. "The production DB is on AWS us-east-1." No timestamp needed. MEMORY.md, knowledge base.
How to do things. "To deploy, run git push origin main then check Vercel." AGENTS.md, skill files.
The Pragmatic Recommendation
Keep markdown + Obsidian + OpenClaw's vector search. Stop here. This handles 90% of use cases.
Add Mem0. Keep markdown as source of truth backup. Sweet spot for power users.
Add Graphiti/Zep for temporal reasoning. Implement tiered memory. Use a proper vector DB. Enterprise territory.
Setting Up Mem0
import { MemoryClient } from 'mem0ai';
const mem0 = new MemoryClient({
apiKey: process.env.MEM0_API_KEY,
});
// Store a memory
export async function addMemory(
content: string,
userId: string,
metadata?: Record<string, any>
) {
return await mem0.add(
[{ role: 'user', content }],
{ user_id: userId, metadata }
);
}
// Search memories
export async function searchMemory(query: string, userId: string) {
return await mem0.search(query, {
user_id: userId,
limit: 10,
});
}The Bridge Pattern: Gradual Migration
Don't rip out markdown to add a graph. Use the bridge pattern:
async function agentSearch(query: string, agentId: string) {
// Try graph first for complex queries
if (isComplexQuery(query)) {
try {
const graphResults = await searchMem0(query, agentId);
if (graphResults.length > 0) return graphResults;
} catch (e) {
console.warn('Graph search failed, falling back');
}
}
// Fall back to vector/markdown search
return await vectorSearch(query, agentId);
}
function isComplexQuery(query: string): boolean {
const indicators = [
'before', 'after', 'because', 'led to',
'resulted in', 'changed', 'decided',
'when did', 'why did', 'how did',
];
return indicators.some(i =>
query.toLowerCase().includes(i)
);
}Cost Comparison
Already done. Minimal maintenance. Good for single agent, <90 days.
30 min setup. Near zero maintenance. Good for 1-3 agents, production.
Half-day setup. High maintenance. Good for temporal reasoning, multi-agent.
1-2 weeks setup. Enterprise territory. Hire someone who's done this before.
The Decision Framework
Ask these in order. Stop at the first "yes":
Share this chapter
Chapter navigation
24 of 36