Agentic Memory Explained (With Code) | db-agent

Most text-to-SQL agents live in a box. They see one database, answer one team's questions, and forget everything the moment the session ends. That's fine right up until you run more than one agent across an organization: an OLTP agent in front of SQL Server, an OLAP agent in front of Snowflake or Databricks, each behind a different network boundary, each blind to what the other has already figured out.

db-agent, our open-source text-to-SQL agent, now has a feature that closes part of that gap. We call it cross-platform contextual memory. This post covers three things: the core concepts you need to understand it (embeddings, vector similarity, retrieval-augmented generation, agentic memory), how the feature is actually built with real code from the repo, and where its limits are on purpose. Being upfront about the limits matters more to us than making it sound bigger than it is.

The problem it's solving

Say your OLTP agent gets asked "which customers ordered the most last quarter" fifty times a week. Somewhere else in the org, an analyst using a completely separate OLAP agent against a data warehouse is asking a related question. Neither agent knows the other exists, and there's no live connection between them. Different databases, different security boundaries, sometimes different teams entirely.

What can safely cross that boundary isn't the data. It's the shape of the question.

Core concepts, defined plainly

Before getting into the implementation, it's worth being precise about a few terms that get thrown around loosely in AI marketing. These are the definitions I'd want on hand before any serious conversation about vector infrastructure, including the one that prompted this post: a call with a MinIO solutions architect about whether they should build something like AWS S3 Vectors.

Embedding. A model takes a piece of text (a sentence, a paragraph, a question) and outputs a fixed-length list of numbers, typically hundreds to a few thousand of them. That list is a point in a high-dimensional space. Text with similar meaning ends up as points that are close together in that space, even if the words used are completely different. "Which customers spent the most" and "who are our top buyers" produce vectors that land near each other, despite sharing almost no words.

Cosine similarity. The most common way to measure how close two embeddings are. It measures the angle between two vectors rather than their raw distance, which makes it robust to differences in text length. A score of 1 means identical direction (very similar meaning), 0 means unrelated, negative means opposite.

Vector database (or vector index). A system built to store large numbers of embeddings and answer one specific question fast: given a new vector, which stored vectors are the closest matches? Doing this by brute force (compare against every stored vector) works fine for thousands of records. Past that, you need an approximate nearest-neighbor (ANN) index, which is what dedicated vector databases like Milvus, Pinecone, or AWS S3 Vectors actually provide.

RAG (Retrieval-Augmented Generation). The umbrella pattern that ties the above together: before asking an LLM to answer something, retrieve relevant text from an external source using embedding similarity, and stuff that retrieved text into the prompt. This is how a model can answer questions about your private documents or your database without those documents being part of its training data.

Agentic memory. The specific application of RAG to an agent's own history. Instead of retrieving from a fixed document set, the agent retrieves from a growing store of its own past interactions. The current industry framing (see AWS Bedrock AgentCore's documentation) splits this into short-term memory, meaning the raw context within one session, and long-term memory, meaning a distilled, consolidated summary extracted asynchronously and stored for later retrieval.

Redaction. In db-agent's case specifically, this means stripping literal data (row values, customer names, exact SQL) out of a memory record before it's stored, keeping only structural references (a table name, a type of entity) and a natural-language summary. This is what makes it safe to share a memory across a security boundary that the raw data itself could never cross.

With those defined, here's how they combine in the actual implementation.

How it works

After every question db-agent answers, a second LLM call turns that turn into a redacted summary. Not the SQL, not the rows, not even the raw question text. Just an insight, a list of entities referenced by type, and a handful of natural-language follow-up questions a different analyst on a different platform might reasonably ask next.

export async function summarizeForMemory(llm, output, { agentId, dbKind, ttlSeconds, model }) {
  // ... calls the LLM with a system prompt that forbids literal data ...
  return {
    recordId: crypto.randomUUID(),
    sourceAgent: agentId,
    sourceDbKind: dbKind,
    createdAt: new Date().toISOString(),
    ttlEpoch: Math.floor(now + ttlSeconds),
    entities: data.entities || [],           // e.g. "table:orders", "account_id:4471"
    insightSummary: data.insight_summary || "",
    suggestedFollowups: data.suggested_followups || [],
  };
}

Notice what's missing from that object. No SQL string, no row values, no customer names. The system prompt behind this call is explicit about it: entities are referenced by identifier and type only, never by the literal value inside the row.

That summary gets embedded (turned into a vector, using the concepts above) and written to a shared store, tagged with an agent ID and a TTL so old insights age out automatically. Other agents can then query that store using cosine similarity to surface anything relevant, while explicitly excluding their own agent ID:

export async function fetchRelevantMemories(queryText, cfg, topK = 3) {
  if (!cfg.memoryEnabled) return [];
  const vector = await embed(cfg.embeddingClient, queryText, cfg.embeddingModel);
  return await cfg.backend.query(vector, { excludeAgent: cfg.dbagentId, topK });
}

That excludeAgent line is doing real work. An agent never sees its own memories reflected back at it. It only ever sees what a different deployment learned.

One important design decision worth calling out

This does not make db-agent smarter about its own past. The retrieved memories surface in the UI as clickable suggestions for a human to consider, not as context injected back into the model's own reasoning during SQL generation. That's a deliberate choice, not an oversight. A suggestion a person can evaluate and choose to click is safer and more predictable than an agent silently reasoning off another agent's summarized insight with no one reviewing it.

If you've read anything about agentic memory patterns from AWS Bedrock AgentCore or Anthropic's memory tool, this is closer to a shared, AI-summarized activity feed with a human in the loop than to a closed-loop memory an agent autonomously draws on for its own next decision. We think that's the right tradeoff for a feature that touches cross-boundary data, even in redacted form, and it's worth being precise about that distinction rather than calling it "agentic memory" without qualification.

There's a feedback loop on top of this too. If a suggested follow-up turns out to be a bad question (a user thumbs it down, or a later benchmark run proves the SQL for it was wrong), that specific suggestion gets stripped out of shared memory immediately, without deleting the whole insight it was attached to:

export async function invalidateFollowup(questionText, cfg) {
  if (!cfg.memoryEnabled) return { removed: 0 };
  const vector = await embed(cfg.embeddingClient, questionText, cfg.embeddingModel);
  return await cfg.backend.invalidateFollowup(questionText, vector);
}

Pluggable by design

The storage layer behind this isn't locked to one vendor. There's a local JSON backend for zero-setup local development, an AWS S3 Vectors backend, and a Milvus backend, all behind the same three-method interface:

const REGISTRY = {
  local:     { create: (env, ctx) => new LocalJsonBackend(...) },
  s3vectors: { create: (env)      => new S3VectorsBackend(...) },
  milvus:    { create: (env)      => new MilvusBackend(...) },
};

Adding a new backend means writing one file that implements put, query, and invalidateFollowup, then adding one entry to that registry. Nothing else in the app changes. That matters because this feature is meant to run anywhere: on a laptop, on Kubernetes, on any cloud, not tied to a single provider's vector store.

This is also the exact reason a conversation with a MinIO solutions architect made sense. MinIO's AIStor is genuinely useful S3-compatible object storage, but as of this writing it has no native vector-similarity primitive of its own, unlike AWS S3 Vectors. Milvus fills that gap today for anyone who wants a self-hosted, open-source option. Whether MinIO builds something native in that space is an open question, and one worth asking directly rather than assuming.

What this is actually good for

Think of it less as "the agent remembers" and more as "the org's agents can tip each other off." A question explored on one platform surfaces as a suggestion on another, with none of the underlying data crossing the boundary. It's a small, deliberately scoped feature, and we would rather it be understood accurately than oversold as something it isn't.

What's next

Coming this week: a rebuilt benchmarking system focused on query accuracy and performance, plus a metrics dashboard showing how the agent is actually doing over time rather than just pass or fail on a fixed set of questions. That's the next real signal we want to give teams running this in production, and it will get its own writeup once it lands.

See it built live

This post covers what shipped. If you want to see it built from scratch, live, with the redaction step and the failure mode it caught explained in real time, that's the first session in our free Agentic Memory on Databricks webinar series. No slides, notebook included.

You can also try db-agent yourself or bring your team through it directly in our Databricks Genie & AI agents workshop.