Agent Memory Consolidation Scheduling: When to Merge, Summarize or Forget

Here’s the uncomfortable truth about long-running AI agents: most of them don’t fail because they forget too little. They fail because they forget nothing. An agent that dutifully stores every observation, every tool result, every user correction, and every half-finished thought will, within a few weeks, be drowning in its own past. Retrieval gets noisier, context windows fill with stale trivia, and the agent starts confidently citing something it “learned” three months ago that stopped being true two months ago.

That’s why agent memory consolidation scheduling matters. Consolidation isn’t a feature you bolt on — it’s a maintenance discipline. Biological memory systems don’t just store; they prune, merge, and abstract on a schedule, and the agents that hold up over months rather than hours tend to do the same. In this tutorial, we’ll build a concrete scheduling system that decides when an agent should merge related memories, when it should summarize them into higher-level knowledge, and when it should let them decay into oblivion.

This is a hands-on guide. You’ll get architecture, code, thresholds, and the pitfalls that will bite you if you skip the boring parts.

Prerequisites

Before you start, you should have:

• A working agent with persistent memory. Any stack works — a custom vector store, Mem0, Letta (formerly MemGPT), Zep, or a homegrown SQLite + embeddings setup. We’ll write the scheduling logic to be storage-agnostic.
• Comfort with Python. The examples are Python 3.10+. You don’t need to be a developer, but you should be able to read and adapt code.
• A scheduler primitive. Cron, APScheduler, Celery beat, or your cloud provider’s scheduled jobs. Consolidation runs are background jobs, not request-path operations.
• An LLM you can call for summarization. A cheap, fast model is fine for most merge and summarize operations. Reserve stronger models for high-stakes abstraction.
• A staging environment with real-ish data. Do not test consolidation policies on your production memory store first. More on this in the pitfalls section.

One conceptual prerequisite: you need to already distinguish between episodic memory (specific events and interactions), semantic memory (generalized facts and preferences), and procedural memory (how to do things). Consolidation policy is different for each. If your store is one undifferentiated blob of text, fix that first — you can’t schedule maintenance on a system you can’t categorize.

Why Consolidation Needs a Schedule (and Why “Just Retrieve Better” Isn’t Enough)

There’s a popular shortcut in agent design: store everything, retrieve with a good reranker, and let the model sort it out. This works for short-lived agents. It falls apart for anything that runs for weeks.

The failure modes are predictable:

1. Contradiction accumulation. The user said their timezone was UTC in January and Europe/Berlin in March. Both memories are “true” in the store. Retrieval surfaces both, and the agent picks wrong half the time.
2. Noise inflation. Every retrieval pulls in near-duplicate memories — five slightly different versions of the same preference. The context window fills with redundancy that crowds out actually relevant information.
3. Staleness without expiry. Facts that were true once (a project’s codebase language, a colleague’s role) persist forever with no confidence decay.
4. Cost creep. Bigger stores mean more embedding calls, more retrieval tokens, and more summarization overhead at query time.

The Generative Agents research from Stanford showed that memory retrieval quality depends heavily on recency, importance, and relevance scoring — and that agents which reflect on accumulated experience (synthesizing higher-level insights) behave far more coherently than those that just replay raw observations. The MemGPT work made a similar point from the systems side: treating the context window as managed memory rather than a dumping ground is what makes long-horizon agents viable.

So the schedule isn’t optional hygiene. It’s the mechanism that keeps memory useful. Let’s build it.

The Three Operations: Merge, Summarize, Forget

Before scheduling, define the operations precisely. Each has a different trigger, a different risk profile, and a different failure mode.

Merge: Deduplicate and Reconcile

Merging takes two or more memories that describe the same thing and produces one canonical record. This is the safest operation and the one you should run most frequently. Examples: collapsing five near-identical preference statements into one, or reconciling two versions of the same fact where one is clearly newer.

The critical sub-case is contradiction resolution. When two memories conflict, merging must record which one wins, why, and when. Never silently discard the loser — keep it as a superseded record with a timestamp, so the agent can answer “what did I used to think?” if asked.

Summarize: Abstract and Elevate

Summarization takes a cluster of episodic memories and produces a semantic or procedural abstraction. Ten episodes of “user asked for shorter answers, then edited my output to be shorter, then said ‘too long’ again” should become one semantic memory: “User strongly prefers concise responses; default to bullet points and cap at ~150 words unless asked otherwise.”

Summarization is where agents get genuinely smarter over time — and where they get genuinely wrong if you’re careless. A bad summary launders noise into confident-sounding fact. This is why summarization needs guardrails: minimum cluster size, source provenance, and a review path for high-impact abstractions.

Forget: Decay and Delete

Forgetting is the operation everyone skips and everyone regrets skipping. It has two flavors:

• Soft decay: reduce a memory’s retrieval weight and confidence over time without deleting it. The memory becomes less likely to surface unless strongly relevant.
• Hard deletion: remove the record entirely. Reserve this for low-value, high-volume data — raw tool outputs, verbose logs, transient scratchpad content.

Forgetting should be governed by explicit rules, not vibes. “Delete anything older than 90 days” is a rule. “The agent decides what to forget” is a recipe for inconsistency.

Designing the Consolidation Schedule

A consolidation schedule is a set of triggers, each tied to an operation, a scope, and a set of preconditions. Here’s a starter policy table you can adapt. Treat the numbers as starting points, not commandments — you’ll tune them against your own data.

Trigger	Operation	Scope	Preconditions	Frequency
Duplicate similarity > 0.92	Merge	Full store, per category	Same memory type; no unresolved conflict flag	Nightly
Contradiction detected	Merge + supersede	Conflicting pair	Newer timestamp wins; log both	Nightly
Cluster of ≥ 5 related episodes	Summarize	Episodic → semantic	Cluster coherence check passes	Weekly
Memory age > 90 days, never retrieved	Soft decay	Episodic, low-importance	Importance score < threshold	Weekly
Raw tool output > 30 days	Hard delete	Tool-result category	Not referenced by any summary	Monthly
Retrieval weight < 0.05 for 60 days	Hard delete	Any	Not flagged as pinned	Monthly
User correction received	Merge + supersede	Single memory	Immediate	Event-driven

A few principles embedded in that table:

• Frequency scales inversely with risk. Merging duplicates is low-risk, so it runs nightly. Hard deletion is irreversible, so it runs monthly and requires the memory to be unreferenced.
• Summarization is gated on cluster size. Summarizing two episodes produces a brittle, over-specific abstraction. Five or more is a reasonable floor; tune upward for high-stakes domains.
• Event-driven triggers exist alongside cron. A user correction should supersede the old memory immediately, not wait for the nightly job.
• Everything is scoped. A merge pass over episodic memory should never touch procedural memory. Category boundaries are safety rails.

Implementing the Scheduler

Here’s a storage-agnostic skeleton. It assumes you have a memory store object exposing query, update, and delete methods, and an LLM client for summarization. Adapt the interfaces to your stack.

from dataclasses import dataclass
from datetime import datetime, timedelta
from enum import Enum

class MemoryType(Enum):
    EPISODIC = "episodic"
    SEMANTIC = "semantic"
    PROCEDURAL = "procedural"

class Operation(Enum):
    MERGE = "merge"
    SUMMARIZE = "summarize"
    DECAY = "decay"
    DELETE = "delete"

@dataclass
class ConsolidationJob:
    operation: Operation
    memory_type: MemoryType
    preconditions: dict
    dry_run: bool = True  # default safe

def run_consolidation(job: ConsolidationJob, store, llm):
    candidates = store.query(
        memory_type=job.memory_type,
        **job.preconditions
    )
    if job.operation == Operation.MERGE:
        return merge_pass(candidates, store, llm, dry_run=job.dry_run)
    elif job.operation == Operation.SUMMARIZE:
        return summarize_pass(candidates, store, llm, dry_run=job.dry_run)
    elif job.operation == Operation.DECAY:
        return decay_pass(candidates, store, dry_run=job.dry_run)
    elif job.operation == Operation.DELETE:
        return delete_pass(candidates, store, dry_run=job.dry_run)

The dry_run default is deliberate. Every job should be runnable in report-only mode that logs what it would do. You’ll use this constantly.

The Merge Pass

Merging starts with candidate generation: find clusters of memories above a similarity threshold. Do this with your store’s native similarity search rather than brute-force comparisons — a naive pairwise scan over 50,000 memories is 1.25 billion comparisons.

def merge_pass(candidates, store, llm, dry_run=True):
    merged = []
    for mem in candidates:
        neighbors = store.similarity_search(
            mem.embedding, threshold=0.92, exclude_id=mem.id
        )
        if not neighbors:
            continue
        cluster = [mem] + neighbors
        # Resolve contradictions explicitly
        winner = max(cluster, key=lambda m: m.timestamp)
        merged_record = {
            "content": winner.content,
            "type": mem.type,
            "supersedes": [m.id for m in cluster if m.id != winner.id],
            "merged_at": datetime.utcnow().isoformat(),
        }
        if not dry_run:
            store.upsert(merged_record)
            store.flag_superseded([m.id for m in cluster if m.id != winner.id])
        merged.append(merged_record)
    return merged

Expected result: After a merge pass on a store with heavy duplication, you should see your memory count drop meaningfully — often 10–30% on messy stores — while retrieval quality (measured on a held-out eval set) improves. If count drops but quality doesn’t improve, your similarity threshold is too aggressive and you’re merging distinct memories.

The Summarize Pass

Summarization is the highest-leverage and highest-risk operation. The pattern: cluster related episodic memories, generate an abstraction with an LLM, validate it against the sources, and store it as a semantic memory with provenance links.

SUMMARY_PROMPT = """You are consolidating an agent's episodic memories.
Given the following episodes, produce ONE concise semantic fact or preference.
Rules:
- Only state what is consistently supported across episodes.
- If episodes conflict, state the most recent consistent pattern.
- Cite which episode IDs support each claim.
- If no consistent pattern exists, output: NO_CONSISTENT_PATTERN

Episodes:
{episodes}
"""

def summarize_pass(candidates, store, llm, dry_run=True):
    clusters = cluster_by_topic(candidates, min_size=5)
    results = []
    for cluster in clusters:
        prompt = SUMMARY_PROMPT.format(episodes=format_episodes(cluster))
        summary = llm.generate(prompt)
        if "NO_CONSISTENT_PATTERN" in summary:
            continue  # don't launder noise into fact
        record = {
            "content": summary,
            "type": MemoryType.SEMANTIC.value,
            "derived_from": [m.id for m in cluster],
            "confidence": coherence_score(cluster),
            "created_at": datetime.utcnow().isoformat(),
        }
        if not dry_run:
            store.upsert(record)
        results.append(record)
    return results

The NO_CONSISTENT_PATTERN escape hatch is essential. Without it, your model will confidently summarize five contradictory episodes into a tidy-sounding lie. The coherence score should penalize clusters where episodes disagree; gate storage on a minimum.

Expected result: Summarization should reduce the number of raw episodes the agent needs to retrieve while increasing the hit rate on preference-style queries. Track “retrieval precision on preference questions” before and after. If it goes down, your summaries are too aggressive or your clusterer is grouping unrelated episodes.

The Decay and Delete Pass

Decay is a weight adjustment, not a deletion. Delete is permanent, so it gets the strictest preconditions.

def decay_pass(candidates, store, dry_run=True):
    now = datetime.utcnow()
    for mem in candidates:
        age_days = (now - mem.created_at).days
        if age_days < 90:
            continue
        if mem.retrieval_count > 0:
            continue  # it's being used; leave it alone
        new_weight = mem.weight * 0.5
        if not dry_run:
            store.update_weight(mem.id, new_weight)
    return candidates

def delete_pass(candidates, store, dry_run=True):
    now = datetime.utcnow()
    for mem in candidates:
        age_days = (now - mem.created_at).days
        if mem.pinned:
            continue
        if mem.referenced_by_summary:
            continue  # provenance matters
        if mem.weight < 0.05 and age_days > 60:
            if not dry_run:
                store.delete(mem.id)

Notice the referenced_by_summary guard. If a summary was derived from an episode, deleting the episode breaks provenance. Either keep episodes that back summaries or archive them to cold storage. Deleting your sources is how you end up with confident abstractions and no way to audit them.

Expected result: Decay and delete should shrink storage costs and reduce retrieval noise over time without measurable quality loss. If quality drops after a delete pass, you deleted something load-bearing — restore from your archive and tighten the preconditions.

Tuning Thresholds Without Breaking Things

The numbers in the policy table are guesses. Here’s how to make them real:

1. Build an eval set first. Collect 50–100 representative queries with known-correct answers drawn from your memory store. This is your regression suite. Every consolidation policy change gets evaluated against it.
2. Run every job in dry-run against production data for a week. Log what would change. Manually inspect a sample of 20 proposed merges, 10 proposed summaries, and 20 proposed deletions. You will find bad merges and bad summaries. That’s the point.
3. Change one threshold at a time. Raise the merge similarity threshold from 0.92 to 0.95, re-run the eval, compare. If precision improves without recall collapsing, keep it. If not, revert.
4. Track the right metrics. Retrieval precision and recall on your eval set, memory count, storage cost, and — critically — a “contradiction rate” measuring how often the agent surfaces conflicting facts. Consolidation should push contradiction rate toward zero.

A useful heuristic for summarization cluster size: start at 5, and raise it for domains where being wrong is expensive (finance, health, legal) and lower it for low-stakes personalization. The cost of a bad summary scales with how much the agent relies on it.

Common Pitfalls (and How to Avoid Them)

Pitfall 1: Consolidating on the request path. Someone will be tempted to run a merge pass when the agent gets a query. Don’t. Consolidation is a batch operation; putting it on the request path adds latency and creates race conditions. Keep it in background jobs.

Pitfall 2: Summarizing noise. If your episodic store is full of low-signal chatter, summarization will produce confident-sounding garbage. Filter first: only summarize episodes that passed an importance threshold or were explicitly marked as significant by a reflection step.

Pitfall 3: Deleting sources of live summaries. Covered above, but it bears repeating because it’s the most common irreversible mistake. Maintain a provenance graph and never delete a node with inbound derived_from edges unless you archive it.

Pitfall 4: No audit trail. Every merge, summary, and deletion should write a log entry with the operation, the affected IDs, the trigger, and the timestamp. When the agent starts behaving strangely in week six, this log is the only way to reconstruct why.

Pitfall 5: Treating all memory types identically. Procedural memory (how to use a tool, what a workflow looks like) should almost never be decayed by age alone — it’s valuable precisely because it’s stable. Episodic memory should decay fastest. Semantic memory sits in between. Encode these differences in your policy.

Pitfall 6: Ignoring the user’s right to correct. When a user says “actually, I moved,” that correction must trigger an immediate supersede, not wait for the nightly job. Event-driven consolidation for explicit corrections is non-negotiable.

Pitfall 7: Forgetting to forget. Teams obsess over storing and retrieving and treat deletion as an afterthought. The agents that stay sharp are the ones whose operators treat forgetting as a first-class, scheduled, audited operation.

What a Healthy Consolidation Cadence Looks Like

If you implement the schedule above and tune it against real data, here’s what a healthy system looks like after a month of operation:

• Memory count grows sublinearly with interaction volume — new episodes are offset by merges and summaries.
• Retrieval precision on your eval set stays flat or improves.
• Contradiction rate trends toward zero as supersede operations resolve conflicts.
• Storage cost stabilizes rather than climbing indefinitely.
• The audit log shows a steady cadence of background jobs, each with a bounded blast radius.

If any of those signals are missing, the schedule needs adjustment — usually by tightening preconditions rather than adding more operations.

Takeaway: Schedule the Boring Maintenance

The agents that impress in demos are the ones with big context windows and clever retrieval. The agents that survive in production are the ones whose operators scheduled the unglamorous work: the nightly merges, the weekly summaries, the monthly decay passes, the audit logs nobody reads until something breaks.

Agent memory consolidation scheduling isn’t a feature. It’s the difference between an agent that accumulates wisdom and one that accumulates clutter. Start with the policy table above, run everything in dry-run first, build your eval set before you touch a single threshold, and treat forgetting as seriously as you treat remembering. Your agent’s future self — the one running six months from now — depends on the maintenance you schedule today.

Your next step: pick one operation — merging duplicates is the safest — and run it in dry-run mode against a copy of your production memory store this week. Inspect twenty proposed merges by hand. You’ll learn more about your agent’s memory quality from that hour than from any architecture diagram.