Auto-consolidation: automatic memory compression for agents

Auto-consolidation compresses daily session logs into curated topic files on a 24-hour cadence, keeping your agent’s memory index lean and fast to read.

As your project accumulates session logs, the raw append-only files grow large. Auto-consolidation solves this by periodically merging those logs into structured topic files — stripping duplicates, converting relative dates to absolute ones, and keeping the index under a hard line and size cap. The process runs automatically when three gates all pass; you can also trigger it manually at any time.

When consolidation triggers

Consolidation runs when all three conditions are true simultaneously:

• Time gate: At least 24 hours have passed since the last consolidation
• Session gate: At least 5 sessions have accumulated since the last consolidation
• Lock gate: No other consolidation is currently in progress

The lock file at .agents/.consolidation-lock tracks all three. If a lock is more than 30 minutes old, it is considered stale and consolidation proceeds.

The four-phase process

1. Orient

   The agent reads .agents/local.md (the index) and the existing topic files to understand current organization before making any changes.

2. Gather

   The agent scans recent daily logs for new signal: patterns that recurred across multiple sessions, gotchas that appeared more than once, stable preferences, and dead ends worth remembering.

   One-time events, transient debugging state, inline code snippets, and secrets are skipped.

3. Consolidate

   For each piece of signal, the agent checks whether it already exists in a topic file. Duplicates are merged into the existing entry. Truly new, recurring items get a new topic entry. All relative dates (“yesterday”, “last week”) are converted to absolute dates.

4. Prune

   The index is updated to point to newly important entries. Stale or contradicted pointers are removed. The index stays under 200 lines and ~25KB.

Validated compression ratio: ~9:1 (107 KB of raw logs → 11.6 KB of curated topic files on first run).

File structure

.agents/
├── local.md                    # Index (200 lines max, ~25KB cap)
├── .consolidation-lock         # Lock file (multi-process safety)
├── logs/
│   ├── 2026-03-25.md           # Daily log (append-only, raw)
│   └── 2026-03-26.md
└── topics/
    ├── patterns.md             # Curated patterns
    ├── gotchas.md              # Curated gotchas
    └── preferences.md          # Curated preferences

Daily log format

Daily logs are append-only. Each session appends a new block to the file for that day.

# Daily Log — 2026-03-25

## Session 14:32 EDT

### What Changed
- Added feature X

### What Worked
- Pattern Z applied successfully

### What Didn't
- Approach A failed (use B instead)

### Patterns Learned
- When X happens, do Y

Never edit past daily logs. They are a raw stream — correctness depends on their append-only nature. If you need to correct something, note the correction in a new session entry.

Index format

The index file (.agents/local.md) is a pointer, not content storage. Each entry is one line with a link to the relevant topic file and a short hook.

# Memory Index

## Patterns
- [Result<T> Pattern](topics/patterns.md#result-pattern) — Error handling without throw (seen 5 sessions)
- [Barrel Exports](topics/patterns.md#barrel-exports) — Module organization (seen 3 sessions)

## Gotchas
- [pnpm build hides errors](topics/gotchas.md#pnpm-build) — Always use --noEmit flag
- [DB_URL in unit tests](topics/gotchas.md#db-url) — Set even for mocked tests

## Preferences
- [Dark mode everywhere](topics/preferences.md#dark-mode) — All editors, terminals

Scripts

Check if consolidation should trigger

./scripts/auto-consolidation-trigger.sh --check

Run --dry-run before your first consolidation to see exactly which logs will be read and what the prompt looks like. It exits without writing any files.

Migrating from the old structure

If your project uses the monolithic .agents.local.md file instead of daily logs, the migration script handles the conversion.

1. Run the migration script

   ./scripts/migrate-to-daily-logs.sh
   

   This creates .agents/logs/, .agents/topics/, and .agents/local.md in the new index format. Your original .agents.local.md is backed up to .agents.local.md.backup — nothing is deleted.

2. Update the memory system section in AGENTS.md

   Replace any references to .agents.local.md in the memory workflow with the new daily log path (.agents/logs/YYYY-MM-DD.md).

3. Start logging to daily logs

   At each session end, append to .agents/logs/YYYY-MM-DD.md instead of .agents.local.md.

4. Wait for first auto-consolidation

   After 24 hours and 5 sessions, consolidation triggers automatically. You can also force it immediately with --force.

5. Review the results

   Check .agents/topics/*.md for curated content and .agents/local.md for the updated index. Verify the compression ratio (target: 5:1 minimum, 9:1 ideal).

Lock file reference

The lock file at .agents/.consolidation-lock is a JSON object:

{
  "last_consolidated_at": 1774952228616,
  "locked_by": "auto-consolidation",
  "locked_at": 1774952228616
}

A lock older than 30 minutes is treated as stale and cleared automatically. If you need to clear a stuck lock manually, delete the file — consolidation will recreate it on the next run.