rollmem

Standalone, dependency-free rolling conversation memory for LLM apps — a running summary plus a recent-message buffer, inspired by LangChain's ConversationSummaryBufferMemory, but with no LangChain (or any) dependency.

Handy for conversation memory, context compression, summarization, and gist-style long-chat handling — a tiny LangChain alternative for the common conversation-memory patterns. It ships four interchangeable compression strategies (see Memory strategies), from keeping every turn verbatim to keeping only a running summary.

Why

ConversationSummaryBufferMemory is a great pattern: keep recent turns verbatim, fold older turns into a running summary so context stays bounded. But pulling in all of LangChain just for that is heavy. rollmem extracts the idea into a tiny, provider-agnostic package. You inject how to summarize and how to count tokens — rollmem stays neutral.

Install

pip install rollmem

Requires Python 3.9+. Zero runtime dependencies, and fully typed (ships py.typed, so your type checker sees the annotations).

Memory strategies

rollmem ships four conversation-compression strategies. They share one Message type and one serialization format, so state moves freely between them — pick whichever fits, and switch later without reworking your storage.

Class	Strategy	What it keeps
RollingMemory / AsyncRollingMemory	summary + recent buffer	recent turns verbatim; older turns folded into a running summary once the buffer passes max_tokens
BufferMemory	buffer-only	the entire conversation verbatim — no summary, no eviction
WindowMemory	sliding window	only the most recent max_messages turns — older ones dropped, no summary
SummaryMemory / AsyncSummaryMemory	summary-only	only a running summary — no verbatim buffer; every turn folded in on arrival

The examples below walk through each. RollingMemory is the default choice; the API section lists every class's signature.

Usage

from rollmem import RollingMemory

def summarize(existing_summary, messages):
    # plug in any LLM here; return the new summary string
    folded = " ".join(m.content for m in messages)
    return (existing_summary + " " + folded).strip()

mem = RollingMemory(
    max_tokens=2000,
    summarize_fn=summarize,   # optional; without it, evicted turns are dropped
    # token_counter=...       # optional; defaults to a word-count estimate.
    #                         # In production inject a model-accurate counter, e.g.
    #                         # token_counter=lambda text: len(enc.encode(text))
)

mem.add_user_message("Hi, I'm planning a trip to Korea.")
mem.add_assistant_message("Great! When are you going?")

print(mem.get_context())    # -> str: summary (as a system turn) + buffer, joined
print(mem.get_messages())   # -> list[Message]: summary prepended as a system turn

Agentic conversations work too — messages can carry tool calls, an id, and opaque metadata:

from rollmem import ASSISTANT, ToolCall

mem.add_message(
    ASSISTANT,
    "",
    tool_calls=[ToolCall(id="c1", name="get_weather", arguments='{"city": "Seoul"}')],
)
mem.add_tool_message("sunny, 23C", tool_call_id="c1")

A tool call and its results are an atomic unit: pruning evicts them together or keeps them together. The eviction boundary is also aligned so that after pruning the buffer never starts with an assistant turn, a tool turn, or an orphaned tool result — openings most provider APIs reject (Anthropic, for example, requires the first message to use the user role).

In asyncio applications, use AsyncRollingMemory — same behaviour and serialization format, but the add_* methods are coroutines and summarize_fn may be a coroutine function:

from rollmem import AsyncRollingMemory

async def summarize(existing_summary, messages):
    folded = " ".join(m.content for m in messages)
    return (existing_summary + " " + folded).strip()   # await your LLM here

mem = AsyncRollingMemory(max_tokens=2000, summarize_fn=summarize)

await mem.add_user_message("Hi, I'm planning a trip to Korea.")

AsyncRollingMemory is safe for concurrent use by multiple asyncio tasks on one event loop — pruning is serialized internally, and a summarizer failure never loses turns. It is not thread-safe, and one instance must stay on one event loop.

To keep the entire conversation verbatim — no summary, no eviction — use BufferMemory (the buffer-only strategy, LangChain's ConversationBufferMemory). It takes no parameters: every turn is simply appended and kept, so context grows with the conversation and trimming is up to you. It shares the same serialization format, so state moves freely between it and RollingMemory.

from rollmem import BufferMemory

mem = BufferMemory()
mem.add_user_message("Hi, I'm planning a trip to Korea.")
mem.add_assistant_message("Great! When are you going?")

print(mem.get_messages())   # the full history, verbatim

To keep only the most recent messages — no summary, oldest dropped — use WindowMemory (the sliding-window strategy, LangChain's ConversationBufferWindowMemory). It takes a single max_messages argument: after each turn the buffer is trimmed back to at most that many messages, so context size stays predictable without any LLM call. Trimming stays atomic over tool-call units — a tool call and its result are dropped together or kept together — and it shares the same serialization format as the other classes.

from rollmem import WindowMemory

mem = WindowMemory(max_messages=6)
mem.add_user_message("Hi, I'm planning a trip to Korea.")
mem.add_assistant_message("Great! When are you going?")

print(mem.get_messages())   # only the last few turns, verbatim

To keep only a running summary — no verbatim buffer at all — use SummaryMemory (the summary-only strategy, LangChain's ConversationSummaryMemory). It requires a summarize_fn and folds every turn into the summary as it arrives, so each add_* call is one summarizer round trip. AsyncSummaryMemory is the asyncio variant, accepting a coroutine summarize_fn; both share the serialization format with the other classes.

from rollmem import SummaryMemory

def summarize_fn(existing_summary, messages):
    ...   # call your LLM; return the updated summary

mem = SummaryMemory(summarize_fn=summarize_fn)
mem.add_user_message("Hi, I'm planning a trip to Korea.")
mem.add_assistant_message("Great! When are you going?")

print(mem.get_messages())   # a single system turn holding the summary

max_tokens is the budget for the verbatim recent-message buffer — not the running summary, and not a model's generation max_tokens (output limit). When the buffer exceeds it, the oldest turns are folded into the summary.

token_counter takes a single message's text (str) and returns an int. The default is a crude word count — fine for demos, but pass a model-accurate counter (such as tiktoken) for real token budgets. The text it receives is Message.token_text() — the content plus any tool-call names, arguments, and linkage — so tool payloads count toward the budget.

Persistence

to_dict() / from_dict() serialize the memory state (running summary plus buffer) to and from a plain dict — you choose the storage format:

import json

raw = json.dumps(mem.to_dict())   # save anywhere: file, DB column, cache...

mem = RollingMemory.from_dict(
    json.loads(raw),
    max_tokens=2000,
    summarize_fn=summarize,        # callbacks are NOT serialized — re-inject them
    # token_counter=...
)

max_tokens and the callbacks are runtime configuration, not saved state, so you pass them again on restore. The buffer is restored verbatim; the token budget is re-applied on the next added message.

The other strategies share this exact format — only the from_dict arguments differ: BufferMemory.from_dict(data), WindowMemory.from_dict(data, *, max_messages=...), and SummaryMemory.from_dict(data, *, summarize_fn=...). State saved by any class loads in another; a saved summary is surfaced as a leading system turn.

How it works

This describes RollingMemory, the summary-plus-buffer strategy; the other three are simpler variants of the same machinery (see Memory strategies).

• New turns go into buffer.
• When buffer exceeds max_tokens, the oldest turns are folded into summary via summarize_fn (or dropped if none is provided). Eviction is atomic over tool-call units — an assistant message with tool_calls and its linked tool results always travel together — and boundary-aligned, so the buffer is kept from starting on a response-like turn whenever possible.
• get_messages() -> list[Message] returns the buffer with the summary prepended as a system turn. get_context() -> str is the string form of the same thing (prompt-ready), so the two never diverge. Neither adds a language-specific label — relabel the summary in your own prompt assembly if you need to.

API

RollingMemory(max_tokens=2000, summarize_fn=None, token_counter=None)

• add_message(role, content, *, id=None, tool_calls=(), tool_call_id=None, metadata=None) — append a turn with any role string, optionally carrying tool calls, an id, and metadata.
• add_user_message / add_assistant_message / add_system_message / add_tool_message — convenience wrappers over add_message using the USER / ASSISTANT / SYSTEM / TOOL role constants (add_tool_message also takes tool_call_id=).
• get_messages() -> list[Message] / get_context() -> str — read the state back (see How it works).
• to_dict() / from_dict(data, *, max_tokens=..., summarize_fn=..., token_counter=...) — serialize and restore (see Persistence).
• clear() — reset the summary and buffer.
• summary: str and buffer: list[Message] — the live state, exposed as plain public attributes.

AsyncRollingMemory(max_tokens=2000, summarize_fn=None, token_counter=None) is the asyncio variant: the same API, but the add_* methods are coroutines and summarize_fn may be a regular or coroutine function (AsyncSummarizeFn). Reads, clear(), and serialization stay synchronous, and state saved by either class loads in the other. Passing a coroutine function as summarize_fn to the synchronous RollingMemory raises TypeError instead of failing silently.

The other three strategies share the same add_*, reads (get_messages() / get_context()), clear(), and summary / buffer attributes; only the constructor and from_dict differ:

• BufferMemory() — buffer-only: appends and keeps every turn, no summary and no eviction. No max_tokens or summarize_fn; restore with from_dict(data).
• WindowMemory(max_messages) — sliding window: after each turn trims the buffer to at most max_messages messages, atomic over tool-call units and boundary-aligned like RollingMemory. Restore with from_dict(data, *, max_messages=...).
• SummaryMemory(summarize_fn) / AsyncSummaryMemory(summarize_fn) — summary-only: summarize_fn is required and folds every turn into the running summary on arrival; there is no verbatim buffer. Restore with from_dict(data, *, summarize_fn=...). The async variant's add_* are coroutines and accept a coroutine summarize_fn; the sync one rejects a coroutine function with TypeError.

All four share the serialization format, so state saved by one loads in another.

Message(role, content, id=None, tool_calls=(), tool_call_id=None, metadata={}) is the provider-neutral turn type: a frozen dataclass with to_dict() / from_dict(), a "role: content" string form, and token_text() — the canonical text used for token counting (content plus tool-call payloads). tool_calls holds ToolCall(id, name, arguments) entries on assistant turns; tool_call_id links a tool-result turn back to its call; metadata is opaque and round-tripped verbatim. The exported role constants are USER, ASSISTANT, SYSTEM, and TOOL — but any string is accepted as a role.

Limitations

These concern summarization, so they apply to RollingMemory and SummaryMemory. BufferMemory and WindowMemory never summarize — so the lossy-summary points below don't apply to them (WindowMemory simply drops the oldest turns; keep its max_messages large enough to retain what you need).

• Lossy by design. Older turns are folded into the summary repeatedly, so each pass can blur or drop detail (a "telephone game" effect). Keep max_tokens large enough that anything you can't afford to lose stays in the verbatim buffer.
• The summary is not bounded for you. max_tokens limits only the verbatim buffer, not the running summary. rollmem hands your summarize_fn the current summary plus the evicted turns and stores whatever it returns — so keeping the summary compact is your summarize_fn's job. If it merely concatenates, the summary (and thus get_context()) grows without limit. Prompt it to compress, or cap the summary length inside the callback.
• Only as accurate as your counter. The default token counter is a rough word count; inject a model-accurate one (e.g. tiktoken) for real budgets.
• In-memory by default. State lives in memory, but to_dict() / from_dict() let you persist and restore it (see Persistence). Callbacks are not serialized and must be re-injected on restore.

Development

pip install -e ".[dev]"   # editable install with dev tools (pytest, build, twine)
pytest                    # run the test suite

License

MIT