Building AgentMem: A Postgres-Native Memory Layer for AI Agents

Multi-agent systems have a memory problem. Each agent starts fresh. Pass context through prompts and you hit token limits. Bolt on a vector database and you add another managed service that has no idea what your other agents are doing.

I wanted something different: a single Postgres table that all agents in a system could read and write to, with smart retrieval built in.

Why Postgres?

Most teams already run Postgres. Adding pgvector is a one-line migration. You get ACID guarantees, row-level security, and a query language every developer already knows.

The alternative — spinning up Neo4j for the entity graph, Pinecone for semantic search, and Redis for caching — works, but it's three extra services, three billing accounts, and three things that can go down.

Hybrid Retrieval

The core of AgentMem is a retrieval pipeline that combines three signals:

1. Semantic search via pgvector cosine similarity on embeddings
2. BM25 keyword matching for exact-term recall (PostgreSQL full-text search)
3. Entity-graph traversal — memories are linked to named entities, so asking about "the Fincil project" surfaces everything tagged to that entity

Gemini re-ranks the combined results before returning them to the agent, so the most contextually relevant memory floats to the top.

CRDT Conflict Resolution

When two agents write to the same memory slot concurrently, AgentMem uses a last-write-wins CRDT with vector clocks. No locks, no blocking — agents keep moving and the system reconciles.

Shipping It

Shipping as both @agentmem/sdk (npm) and agentmem-py (PyPI) meant maintaining two clients against the same REST API. The TypeScript SDK came first; the Python port took a weekend.

The docs site is built with Nextra and deployed on Vercel — one git push and the docs update.

What I'd Do Differently

• Start with the Python SDK. Most AI engineers reach for Python first.
• Write integration tests against a real Postgres instance from day one. Mocks lied to me twice.
• Ship the docs before the SDK. Having something to point people to makes the launch feel real.

If you're building multi-agent systems and want persistent, queryable memory without a new database, check it out on npm.