From 3be2a2064cbbe1bb449ed70584dc8f2ec8531074 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Tue, 28 Jul 2026 18:17:00 -0500 Subject: [PATCH 01/32] docs: add ADR 0032 for durable thread compaction --- .../0032-durable-thread-compaction.md | 272 ++++++++++++++++++ 1 file changed, 272 insertions(+) create mode 100644 docs/decisions/0032-durable-thread-compaction.md diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md new file mode 100644 index 0000000..cda5b8a --- /dev/null +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -0,0 +1,272 @@ +--- +# These are optional elements. Feel free to remove any of them. +status: proposed +contact: ahmedmuhsin +date: 2026-07-27 +deciders: ahmedmuhsin +consulted: eavanvalkenburg +informed: +--- + +# Thread Compaction for Durable Agents and Workflows + +## Context and Problem Statement + +Long-running **durable** agents and workflows accumulate conversation history in durable +storage and replay it on every turn. Durable agents persist a full `ConversationHistory` in +entity state (`AgentEntity` → `DurableAgentState`); durable workflows persist inter-executor +messages (`AgentExecutor.full_conversation`) as checkpointed envelopes. Unlike an in-memory agent +— whose history lives in process RAM (gigabytes) and disappears when the process recycles — this +history is **persisted, reloaded every turn, and permanent**. + +It helps to separate **three distinct pressures**, because they have different owners: + +| Pressure | What bounds it | Same in core? | Owner | +| --- | --- | --- | --- | +| **Context window** — the model's max input per call | the model | **Yes** — identical in core and durable | Compaction (in-run filter) | +| **Token cost / latency** — resending history each turn | tokens billed / round-trip | **Yes** — same mechanism | Compaction (in-run filter) | +| **Storage capacity** — the cumulative persisted state | backend state-size limit | **No** — durable-only | Storage backend (built-in limit or external store) | + +The first two are **per-operation** (what a single turn sends to the model) and are **identical in +core and durable** — the model's context window is the same regardless of runtime. The third is +**cumulative across all runs**: `ConversationHistory` is a single blob appended to every turn and +re-persisted whole, so it is bounded by the durable backend's state-size limit (backend-specific; +e.g. classic Azure Storage ~1 MB/entity), whereas a core process is bounded only by RAM and resets +on restart. **Storage capacity is an infrastructure concern, not a context-window concern** — it is +relieved by raising the limit or moving to an external store, not by trimming what the model sees. + +Core MAF already has a compaction system ([ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md); +.NET `Microsoft.Agents.AI.Compaction`; Python `agent_framework._compaction`) with **two hooks**: + +1. **In-run filter** — a `CompactionProvider` (`AIContextProvider`) / `compaction_strategy` runs + before each model call. It is **non-lossy**: it filters the projection sent to the model and + stores incremental group state in the `AgentSession.StateBag`; the underlying store is untouched. +2. **Store reducer** — an `IChatReducer` on a `ChatHistoryProvider` (e.g. `InMemoryChatHistoryProvider`) + **lossily** rewrites the stored conversation. `strategy.AsChatReducer()` bridges any core strategy + into this hook, so it is the **same strategies** applied at the store instead of the model call. + +The durable layer benefits from **neither** today, because `AgentEntity` **bypasses the +`ChatHistoryProvider`**: it creates a fresh session per operation (so the StateBag — and any history +provider store or reducer in it — is discarded) and feeds `ConversationHistory` directly as input +messages. So both the in-run filter's incremental state and the store reducer are thrown away each +turn. + +The goal is **configuration parity**: a user's core compaction config must carry over to a durable +entity or workflow **unchanged**, reusing the same strategies and hooks on the durable runtime, +without a parallel durable-only API. + +**How should core compaction (both hooks) be reused on the durable runtime, in both .NET and +Python, so that the model input is bounded identically to core and the persisted store can be +bounded when the user opts into it?** + +## Decision Drivers + +- **Configuration parity** — the same core compaction config (strategies, `CompactionProvider`, + `IChatReducer`) must apply unchanged when moving core → durable entity → durable workflow. No + parallel durable-only API. +- **Reuse existing core hooks** — do not reinvent triggers/strategies/grouping; reuse the in-run + filter and the store reducer. +- **Separate storage capacity from context management** — bound the model input with compaction + (parity with core); relieve persisted-storage capacity with infrastructure (backend limits / + external stores), not by silently trimming. +- **No silent data loss in the durable record** — a durable system of record must not quietly + truncate history; lossy reduction is explicit opt-in, and hard capacity limits should surface a + clear error/warning. +- **Determinism / idempotency** — durable entity operations can be retried; a lossy reducer + (especially LLM summarization) must not corrupt or diverge persisted state across retries. +- **Message-list correctness** — preserve atomic groups (assistant tool-call + tool-result, and + reasoning pairings) so the model input stays valid. +- **Cover both surfaces** — durable agents **and** durable workflows, in **both** languages. +- **No-op for service-managed storage** — when the service owns the conversation (a + `ConversationId`/`service_session_id` is set), the client has no history to compact. + +## Considered Options + +- **Option 1 — In-run filter only.** Register the core `CompactionProvider` / `compaction_strategy` + on the inner agent; change nothing else in the durable layer. +- **Option 2 — Bespoke pre-write compaction in the agent entity.** Add durable-specific code that + compacts `ConversationHistory` inside the entity operation before checkpoint. +- **Option 3 — On-storage maintenance compaction.** Compact persisted history from a separate + entity signal/operation, decoupled from the request path. +- **Option 4 — Workflow-level compaction hook.** Apply a strategy at the `AgentExecutor` + `context_mode` / `context_filter` boundary that governs the `full_conversation` chained between + agent executors. +- **Option 5 — Auto-derive a durable store reducer.** When only an in-run filter is configured, + automatically derive a lossy store reducer (`strategy.AsChatReducer()`) so durable storage is + bounded even without an explicit reducer. +- **Option 6 — Durable store as a `ChatHistoryProvider` (chosen).** Back the durable entity's + persisted conversation with a core `ChatHistoryProvider` implementation, so **both** core hooks + apply on the durable runtime unchanged: the in-run filter runs in the agent pipeline (L1), and a + user-configured `IChatReducer` bounds the store (L2, opt-in). The same seam makes external storage + backends (Cosmos, Valkey, blob) pluggable for capacity. + +## Decision Outcome + +Chosen option: **Option 6 — express durable conversation storage as a core `ChatHistoryProvider`**, +combined with the workflow hook (Option 4). This makes core's two compaction hooks apply on the +durable runtime with **no config change**, and cleanly separates context management from storage +capacity. + +Compaction applies at **three layers**, mapped directly onto the core hooks: + +| Layer | Core mechanism reused | Lossy? | Role | +| --- | --- | --- | --- | +| **L1 — in-run filter** | `CompactionProvider` in the agent pipeline | No | Always-on. Bounds the **model input** (context window, token cost). Identical to core. | +| **L2 — store reducer** | `IChatReducer` on the durable `ChatHistoryProvider` | Yes | **Opt-in.** Bounds the **persisted store**, only when the user configures a reducer. Identical to core. | +| **L3 — workflow hook** | the same strategy as the `AgentExecutor` `context_filter` | Yes | Bounds the inter-executor `full_conversation`. | + +**Two accumulation surfaces:** + +| Surface | Where it accumulates | Covered by | +| --- | --- | --- | +| **In-agent** | the agent's model input, and the persisted `AgentEntity` store | L1 (filter) + L2 (reducer, opt-in) | +| **Inter-executor (workflow)** | `AgentExecutor.full_conversation`, checkpointed as envelopes | L3 | + +**Why Option 6 over bespoke entity compaction (Option 2).** Making the durable store a +`ChatHistoryProvider` means L2 is core's existing `IChatReducer` path — not new compaction code — +and the same abstraction is the seam for **external storage backends** (Cosmos/Valkey/blob) that +relieve capacity. One abstraction delivers both the opt-in reducer and pluggable storage, all +reused from core. + +**Strict parity — no auto-derive (Option 5 rejected).** Durable honors exactly the hooks the user +configured. If only an in-run filter is configured, durable trims the model input just like core +and the store still grows — because the context window (which compaction addresses) is identical in +both runtimes, and storage capacity is a separate concern. Auto-deriving a lossy reducer would use a +context-window tool to solve a storage problem and **silently destroy the durable record**, breaking +both the "no data loss" driver and parity. Storage capacity is instead addressed by the backend: +the built-in store enforces a limit (surface a clear error/warning as it is approached), and an +external `ChatHistoryProvider` raises the ceiling for those who need unbounded durable records. + +**Ideal durable default:** keep the full record in a (possibly external) durable `ChatHistoryProvider` +and apply the L1 in-run filter to the model input — never lose the record, always bound what the +model sees. A lossy L2 reducer is a deliberate opt-in, not a durable surprise. + +**Why workflows largely come "for free."** Durable workflow agent execution +(`DurableExecutorDispatcher.ExecuteAgentAsync`) runs an agent through the same +`DurableAIAgent → AgentEntity → inner agent` path as standalone durable agents, so **L1 and L2 are +inherited by workflow agent executors**. The workflow's own `full_conversation` between executors +does not pass through the agent, so it needs the separate **L3** hook. + +**Service-managed storage** remains out of scope (mirrors ADR-0019): when the service owns the +conversation, the client holds no history to compact. + +### Consequences + +- Good: **configuration parity** — the same core strategies/hooks apply on the durable runtime with + no changes; the model input is bounded identically to core. +- Good: **no reinvention** — L2 is core's `IChatReducer` path; the `ChatHistoryProvider` seam also + makes external storage backends pluggable for capacity. +- Good: **no silent data loss** — the durable record is only reduced when the user opts into a + reducer; capacity limits surface explicitly. +- Good: durable workflows inherit L1+L2; L3 reuses the existing `context_filter` seam. +- Neutral: making the durable store a `ChatHistoryProvider` is a larger change to the entity than a + bespoke compaction pass would be, and must preserve the existing `ConversationHistory` consumer + contract (`AgentRunHandle` response polling, audit/replay, TTL). +- Bad: an opt-in LLM-based reducer runs inside the entity operation and re-runs on retry; mitigated + by stable summary identity and (optionally) Option 3 to move heavy summarization off the request + path. + +### Validation + +- **Unit tests (both languages):** a core `CompactionProvider` on a durable agent bounds the model + input; a configured `IChatReducer` bounds the persisted store; with no reducer the store is not + silently truncated; atomic groups preserved; reducer idempotent across simulated entity retries; + service-managed sessions skipped. +- **Integration tests:** the same agent config produces equivalent compaction behavior in core and + durable; a fan-out/chained durable workflow keeps `full_conversation` bounded via L3; an external + `ChatHistoryProvider` stores history beyond the built-in limit. + +## Pros and Cons of the Options + +### Option 1 — In-run filter only + +- Good, because it is the existing core feature with (almost) no new code, and bounds the model + input within a run (including long tool loops). +- Good, because it applies to workflow agent executors too (shared agent path). +- Neutral, because it is non-lossy — by design it does not bound the persisted store. +- Bad, because the persisted `ConversationHistory` still grows and its incremental StateBag is + discarded each operation (recomputed every turn), so on its own it does not address storage. + +### Option 2 — Bespoke pre-write compaction in the agent entity + +- Good, because it directly bounds persisted state and can reuse the static `CompactAsync`. +- Neutral, because it requires a `DurableAgentStateMessage` ⇄ `ChatMessage` conversion. +- Bad, because it is **new durable-specific code** that duplicates what core's `IChatReducer` path + already does, and it does not give external-storage pluggability. + +### Option 3 — On-storage maintenance compaction + +- Good, because it keeps expensive summarization off the request/response path and maps to the + "on existing storage" point from ADR-0019. +- Neutral, because it can layer on top of Option 6 later without rework. +- Bad, because it adds scheduling/trigger machinery and a window where state is temporarily + un-compacted; on its own it does not bound in-turn growth. + +### Option 4 — Workflow-level compaction hook + +- Good, because it bounds the inter-executor `full_conversation` that agent-level compaction never + sees, reusing the existing `context_filter` seam. +- Neutral, because it is only relevant to multi-agent workflows. +- Bad, because a naive filter could break atomic groups if it does not reuse the core grouping. + +### Option 5 — Auto-derive a durable store reducer + +- Good, because it would bound durable storage automatically even for in-run-filter-only configs. +- Bad, because it **conflates storage with context management** — using a lossy tool to solve a + capacity problem — and **silently truncates the durable record**, breaking parity and the + no-data-loss driver. Rejected. + +### Option 6 — Durable store as a `ChatHistoryProvider` (chosen) + +- Good, because **both** core hooks apply unchanged: L1 filter in the pipeline, L2 reducer on the + store — full configuration parity. +- Good, because the same abstraction makes external storage backends (Cosmos/Valkey/blob) pluggable, + relieving capacity without touching compaction. +- Good, because it is core reuse rather than durable-specific compaction code. +- Neutral, because L2 is opt-in — a store is only reduced when the user configures a reducer. +- Bad, because it is a larger entity change and must preserve the `ConversationHistory` consumer + contract (response polling, audit, TTL). + +## Cross-Cutting Design Details + +- **Configuration parity (discovery over new API).** The durable runtime honors the compaction the + user already configured on the agent — the `CompactionProvider` in the pipeline (L1) and any + `IChatReducer` on the history provider (L2). A durable-specific option exists at most as an + optional override, never as the required path. Moving core → durable entity → durable workflow + requires no reconfiguration. +- **Two hooks, mapped.** In-run filter (`CompactionProvider`) → L1, non-lossy, bounds the model + input. Store reducer (`IChatReducer` on the durable `ChatHistoryProvider`) → L2, lossy, opt-in, + bounds the persisted store. Both accept the same `CompactionStrategy` (via `strategy.AsChatReducer()`). +- **Reducer trigger.** Honor the configured `ReducerTriggerEvent`; `AfterMessageAdded` + (compact-on-write, before checkpoint) is the natural durable default so the checkpoint is already + bounded. `BeforeMessagesRetrieval` also works (reduce-on-load, then persist). +- **Storage capacity is separate.** The built-in entity store is bounded by the backend state-size + limit; approaching it should surface a clear error/warning, not silent truncation. An external + `ChatHistoryProvider` (Cosmos/Valkey/blob) raises the ceiling for unbounded durable records and + is enabled by the same Option 6 seam. +- **Determinism & idempotency.** An opt-in lossy reducer runs inside the entity operation and + re-runs on retry. Give any generated summary a **stable identity** (derived from the ids of the + messages it replaces) so retries do not re-summarize or duplicate. Reduced content becomes + **permanent** durable state (same indirect-prompt-injection caution core flags on + `ChatReducerCompactionStrategy` / `SummarizationCompactionStrategy`). +- **Message-list correctness.** Reuse core grouping so atomic tool-call/result and reasoning + pairings are preserved at every layer. +- **Token counting.** Triggers must work without a live model call; use the estimator tokenizer + (`CharacterEstimatorTokenizer` / equivalent) unless a real tokenizer is supplied. +- **Placement.** The durable `ChatHistoryProvider` backs `AgentEntity` (.NET) / `AgentEntity` in + `_entities.py` (Python). L3 lives in the `AgentExecutor` context handling in both languages. + +## More Information + +- Builds on [ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md) (context compaction strategy), + which defines the in-run / pre-write / on-existing-storage compaction points and the atomic-group + constraint. +- Core reference mechanisms reused: `CompactionProvider` (in-run filter), `InMemoryChatHistoryProvider` + + `IChatReducer` (store reducer), `strategy.AsChatReducer()` bridge, and the existing external + `ChatHistoryProvider` implementations (`CosmosChatHistoryProvider`, `ValkeyChatHistoryProvider`). +- Relevant durable code: `AgentEntity` and `DurableAgentState` (durable agents), + `DurableExecutorDispatcher.ExecuteAgentAsync` (durable workflow agent execution), and + `AgentExecutor` (`context_mode` / `context_filter`, `full_conversation`). +- Suggested realization order: express the durable store as a `ChatHistoryProvider` (Option 6) → + verify L1 filter parity → wire L3 workflow hook → add external storage backends → evaluate + Option 3 for heavy summarization. From 63604bc427d8f9ae9292436363445afe04b1b67e Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 19:45:29 -0500 Subject: [PATCH 02/32] feat: back durable agent history with a core HistoryProvider (ADR 0032) Adds DurableHistoryProvider, a core HistoryProvider whose store is the agent's durable entity state. Because it is an ordinary provider, a CompactionProvider configured the normal way runs against durable history unchanged. Compaction is reconciled by message id rather than by position, since strategies may insert messages (summaries) as well as annotate them. That required persisting message ids and making DurableAgentStateMessage serialization symmetric: extension_data was read on load but silently dropped on save, so compaction annotations were destroyed on every turn. The ADR records the core interface gaps found while doing this. --- .../0032-durable-thread-compaction.md | 33 ++ .../agent_framework_durabletask/__init__.py | 3 + .../agent_framework_durabletask/_constants.py | 3 + .../_durable_agent_state.py | 19 +- .../agent_framework_durabletask/_entities.py | 78 ++++- .../_history_provider.py | 294 +++++++++++++++++ .../tests/test_durable_history_provider.py | 304 ++++++++++++++++++ 7 files changed, 724 insertions(+), 10 deletions(-) create mode 100644 python/packages/durabletask/agent_framework_durabletask/_history_provider.py create mode 100644 python/packages/durabletask/tests/test_durable_history_provider.py diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index cda5b8a..50c8c28 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -256,6 +256,39 @@ conversation, the client holds no history to compact. - **Placement.** The durable `ChatHistoryProvider` backs `AgentEntity` (.NET) / `AgentEntity` in `_entities.py` (Python). L3 lives in the `AgentExecutor` context handling in both languages. +## Core Interface Gaps for Pluggable History Providers + +Prototyping the Python `DurableHistoryProvider` surfaced three places where the current contracts +assume a *session-state-backed* history provider. They are recorded here because they affect **any** +external provider (Cosmos, Valkey, durable), not just this one. The prototype works around them; the +cleaner fix is upstream. + +1. **Compaction bypasses the provider.** `CompactionProvider.after_run` reads stored messages + directly from `session.state[history_source_id]["messages"]` rather than asking the provider. + A provider whose store is *not* session state therefore gets no post-run compaction - L2 silently + no-ops. *Workaround:* the provider publishes its loaded messages as a working buffer under that + key. *Upstream fix:* have compaction request messages from the history provider. + +2. **`save_messages()` is append-only.** It receives only the newly produced messages, so mutations + that compaction applies to *already stored* messages (setting `_excluded`, inserting a summary) + have no defined path back to the store. *Workaround (implemented):* the provider overrides + `after_run` and reconciles the working buffer itself **by `message_id`**, updating annotations on + known messages and inserting ones compaction added. This required persisting `messageId` in + durable state, which also gives summaries the **stable identity** the idempotency requirement + needs. *Upstream fix:* add an explicit replace/flush operation alongside append so every external + provider does not have to re-implement this reconciliation. + +3. **Message-level metadata was not persisted (durable schema).** `DurableAgentStateMessage.to_dict()` + dropped `extension_data` while `from_dict()` read it - a write-lossy asymmetry that silently + discarded compaction annotations on every state round-trip. Since annotations are what carry + compaction state, this had to be fixed for any of this to work. The Python side now serializes it; + **.NET and the shared state schema need the same treatment** for cross-language parity. + +Consequence for ordering: core runs `before_run` forward and `after_run` in **reverse**. With +`[history, compaction]`, compaction annotates the buffer *before* the history provider flushes it +(convenient), but it sees history only as of the **previous** turn - so context reaches a steady +state rather than shrinking immediately. This is expected, not a defect. + ## More Information - Builds on [ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md) (context compaction strategy), diff --git a/python/packages/durabletask/agent_framework_durabletask/__init__.py b/python/packages/durabletask/agent_framework_durabletask/__init__.py index a3e2727..fecc925 100644 --- a/python/packages/durabletask/agent_framework_durabletask/__init__.py +++ b/python/packages/durabletask/agent_framework_durabletask/__init__.py @@ -50,6 +50,7 @@ ) from ._entities import AgentEntity, AgentEntityStateProviderMixin from ._executors import DurableAgentExecutor +from ._history_provider import DurableHistoryBinding, DurableHistoryProvider from ._models import AgentSessionId, DurableAgentSession, RunRequest from ._orchestration_context import DurableAIAgentOrchestrationContext from ._response_utils import ensure_response_format, load_agent_response @@ -159,6 +160,8 @@ def __dir__() -> list[str]: "DurableAgentStateUriContent", "DurableAgentStateUsage", "DurableAgentStateUsageContent", + "DurableHistoryBinding", + "DurableHistoryProvider", "DurableStateFields", "DurableTaskWorkflowContext", "DurableWorkflowClient", diff --git a/python/packages/durabletask/agent_framework_durabletask/_constants.py b/python/packages/durabletask/agent_framework_durabletask/_constants.py index 9e48b51..e1542dc 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_constants.py +++ b/python/packages/durabletask/agent_framework_durabletask/_constants.py @@ -131,6 +131,9 @@ class DurableStateFields: # History field CONVERSATION_HISTORY: Final[str] = "conversationHistory" + # Stable per-message identity (used for compaction reconciliation and idempotency) + MESSAGE_ID: Final[str] = "messageId" + class ContentTypes: """Content type discriminator values for the $type field. diff --git a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py index f1fb577..0bf9a94 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py +++ b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py @@ -722,13 +722,19 @@ class DurableAgentStateMessage: contents: List of content items (text, function calls, errors, etc.) author_name: Optional name of the message author (typically set for assistant messages) created_at: Optional timestamp when the message was created - extension_data: Optional additional metadata (not serialized per schema) + message_id: Optional stable identifier for the message. Persisted so context-management + state (for example compaction summaries that reference the messages they replace) + can be reconciled across entity operations. + extension_data: Optional additional metadata. Carries a message's + ``additional_properties``, including compaction annotations, so that context + management state survives across entity operations. """ role: str contents: list[DurableAgentStateContent] author_name: str | None = None created_at: datetime | None = None + message_id: str | None = None extension_data: dict[str, Any] | None = None def __init__( @@ -738,11 +744,13 @@ def __init__( author_name: str | None = None, created_at: datetime | None = None, extension_data: dict[str, Any] | None = None, + message_id: str | None = None, ) -> None: self.role = role self.contents = contents self.author_name = author_name self.created_at = created_at + self.message_id = message_id self.extension_data = extension_data def to_dict(self) -> dict[str, Any]: @@ -763,6 +771,10 @@ def to_dict(self) -> dict[str, Any]: result[DurableStateFields.CREATED_AT] = self.created_at.isoformat() if self.author_name is not None: result[DurableStateFields.AUTHOR_NAME] = self.author_name + if self.message_id is not None: + result[DurableStateFields.MESSAGE_ID] = self.message_id + if self.extension_data: + result[DurableStateFields.EXTENSION_DATA] = self.extension_data return result @classmethod @@ -775,6 +787,7 @@ def from_dict(cls, data: dict[str, Any]) -> DurableAgentStateMessage: contents=_parse_contents(data), author_name=data.get(DurableStateFields.AUTHOR_NAME), created_at=created_at, + message_id=data.get(DurableStateFields.MESSAGE_ID), extension_data=data.get(DurableStateFields.EXTENSION_DATA), ) @@ -820,6 +833,7 @@ def from_chat_message(chat_message: Message) -> DurableAgentStateMessage: role=chat_message.role if hasattr(chat_message.role, "value") else str(chat_message.role), contents=contents_list, author_name=chat_message.author_name, + message_id=getattr(chat_message, "message_id", None), extension_data=dict(chat_message.additional_properties) if chat_message.additional_properties else None, ) @@ -841,6 +855,9 @@ def to_chat_message(self) -> Any: if self.author_name is not None: kwargs["author_name"] = self.author_name + if self.message_id is not None: + kwargs["message_id"] = self.message_id + if self.extension_data is not None: kwargs["additional_properties"] = self.extension_data diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 63f7098..833734e 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -7,6 +7,7 @@ import inspect import logging import warnings +from collections.abc import Sequence from datetime import datetime, timezone from typing import Any, cast @@ -28,6 +29,12 @@ DurableAgentStateRequest, DurableAgentStateResponse, ) +from ._history_provider import ( + DurableHistoryBinding, + DurableHistoryProvider, + bind_durable_history, + unbind_durable_history, +) from ._models import RunRequest logger = logging.getLogger("agent_framework.durabletask") @@ -171,16 +178,41 @@ async def run( state_request = DurableAgentStateRequest.from_run_request(run_request) self.state.data.conversation_history.append(state_request) - try: - chat_messages: list[Message] = [ - replayable_message - for entry in self.state.data.conversation_history - if not self._is_error_response(entry) - for m in entry.messages - if (replayable_message := self._to_replayable_message(m)) is not None - ] + durable_history = self._find_durable_history_provider() + binding_token = ( + bind_durable_history( + DurableHistoryBinding(state_provider=self._state_provider, correlation_id=correlation_id) + ) + if durable_history is not None + else None + ) - run_kwargs: dict[str, Any] = {"messages": chat_messages, "options": options} + try: + if durable_history is not None: + # Provider-backed path: the DurableHistoryProvider loads prior turns straight + # from durable entity state, so history lives in exactly one place and only the + # newly received request messages are passed as run input. Core context providers + # (history and compaction) therefore work unchanged on the durable runtime. + chat_messages = [ + replayable_message + for m in state_request.messages + if (replayable_message := self._to_replayable_message(m)) is not None + ] + run_kwargs: dict[str, Any] = { + "messages": chat_messages, + "session": self._create_session(), + "options": options, + } + else: + # Legacy path: replay the full persisted conversation on every turn. + chat_messages = [ + replayable_message + for entry in self.state.data.conversation_history + if not self._is_error_response(entry) + for m in entry.messages + if (replayable_message := self._to_replayable_message(m)) is not None + ] + run_kwargs = {"messages": chat_messages, "options": options} agent_run_response: AgentResponse = await self._invoke_agent( run_kwargs=run_kwargs, @@ -213,6 +245,34 @@ async def run( return error_response + finally: + if binding_token is not None: + unbind_durable_history(binding_token) + + def _find_durable_history_provider(self) -> DurableHistoryProvider | None: + """Return the agent's :class:`DurableHistoryProvider`, if it is configured with one.""" + providers = getattr(self.agent, "context_providers", None) + if not isinstance(providers, (list, tuple)): + return None + for provider in cast("Sequence[Any]", providers): + if isinstance(provider, DurableHistoryProvider): + return provider + return None + + def _create_session(self) -> Any: + """Create a fresh session for a provider-backed run. + + No session state needs to persist: conversation history and any compaction + annotations live in durable entity state, loaded by the history provider. + """ + create_session = getattr(self.agent, "create_session", None) + if not callable(create_session): + raise TypeError( + f"Agent {type(self.agent).__name__} is configured with a DurableHistoryProvider " + "but does not support create_session()." + ) + return create_session() + @staticmethod def _to_replayable_message(message: DurableAgentStateMessage) -> Message | None: """Convert persisted history into a message safe to replay into chat clients.""" diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py new file mode 100644 index 0000000..1ad11ae --- /dev/null +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -0,0 +1,294 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""A core ``HistoryProvider`` backed by durable entity state. + +This lets the durable runtime plug into the Agent Framework context-provider pipeline +instead of managing conversation history itself. Because the agent's own history +provider supplies context, core compaction (``CompactionProvider``) works unchanged and +its annotations are persisted alongside the messages in durable entity state - a single +stored copy, no side-car session blob. + +See ADR-0032 (durable thread compaction). +""" + +from __future__ import annotations + +import logging +from collections.abc import Iterator, Sequence +from contextvars import ContextVar, Token +from dataclasses import dataclass +from typing import TYPE_CHECKING, Any, cast + +from agent_framework import HistoryProvider, Message + +from ._durable_agent_state import DurableAgentStateEntry, DurableAgentStateMessage, DurableAgentStateResponse + +if TYPE_CHECKING: + from ._entities import AgentEntityStateProviderMixin + +logger = logging.getLogger("agent_framework.durabletask") + +WORKING_BUFFER_KEY = "messages" +POSITIONS_KEY = "_positions" +EXCLUDED_KEY = "_excluded" + + +@dataclass +class DurableHistoryBinding: + """Per-operation binding between a durable entity and the history provider.""" + + state_provider: AgentEntityStateProviderMixin + """The entity state provider whose conversation history backs the agent.""" + + correlation_id: str | None = None + """Correlation id of the in-flight request, whose entry is excluded from loaded history.""" + + +_current_binding: ContextVar[DurableHistoryBinding | None] = ContextVar( + "durable_history_binding", + default=None, +) + + +def bind_durable_history(binding: DurableHistoryBinding) -> Token[DurableHistoryBinding | None]: + """Bind the durable entity state for the current operation. + + Returns a token that must be passed to :func:`unbind_durable_history`. + """ + return _current_binding.set(binding) + + +def unbind_durable_history(token: Token[DurableHistoryBinding | None]) -> None: + """Release a binding created by :func:`bind_durable_history`.""" + _current_binding.reset(token) + + +def current_durable_history_binding() -> DurableHistoryBinding | None: + """Return the binding for the current durable operation, if any.""" + return _current_binding.get() + + +class DurableHistoryProvider(HistoryProvider): + """History provider whose store is the durable entity's conversation history. + + The durable entity remains the writer of record for requests and responses, so this + provider does not append messages itself (``store_inputs``/``store_outputs`` are off). + What it does provide is: + + * **load** - flattens persisted conversation history into ``Message`` objects, restoring + any compaction annotations that were stored with them. + * **flush** - writes annotations that compaction applied during the run back into the + persisted messages, so compaction state survives across entity operations. + + Attributes: + skip_excluded: When True, messages marked ``_excluded`` by compaction are omitted + from the context loaded for the model. The messages remain in durable storage. + prune_excluded: When True, excluded messages are physically removed from durable + storage on flush. This is **lossy** and opt-in - it is what actually bounds the + size of persisted state. + """ + + DEFAULT_SOURCE_ID = "durable_history" + + def __init__( + self, + source_id: str | None = None, + *, + skip_excluded: bool = True, + prune_excluded: bool = False, + ) -> None: + """Initialize the durable history provider. + + Args: + source_id: Unique identifier for this provider instance. + skip_excluded: Omit compaction-excluded messages from loaded context. + prune_excluded: Physically delete excluded messages from durable storage + on flush. Lossy; disabled by default. + """ + super().__init__( + source_id=source_id or self.DEFAULT_SOURCE_ID, + load_messages=True, + # The durable entity owns appends to conversation history. + store_inputs=False, + store_outputs=False, + ) + self.skip_excluded = skip_excluded + self.prune_excluded = prune_excluded + + def _binding(self) -> DurableHistoryBinding | None: + binding = current_durable_history_binding() + if binding is None: + logger.warning( + "[DurableHistoryProvider] No durable binding is active; the provider yields no history. " + "This provider only works inside a durable agent entity operation." + ) + return binding + + def _replayable_entries(self, binding: DurableHistoryBinding) -> Iterator[tuple[DurableAgentStateEntry, int]]: + """Yield (entry, message_index) pairs that participate in model context.""" + for entry in binding.state_provider.state.data.conversation_history: + if isinstance(entry, DurableAgentStateResponse) and entry.is_error: + continue + if binding.correlation_id is not None and entry.correlation_id == binding.correlation_id: + # The in-flight request is delivered as run input, not as history. + continue + for index in range(len(entry.messages)): + yield entry, index + + @staticmethod + def _to_message(stored: DurableAgentStateMessage) -> Message | None: + """Convert a persisted message into one that is safe to replay to a chat client.""" + chat_message: Message = stored.to_chat_message() + replayable = [content for content in chat_message.contents if content.type != "reasoning"] + if not replayable: + return None + return Message( + role=chat_message.role, + contents=replayable, + author_name=chat_message.author_name, + message_id=stored.message_id, + additional_properties=chat_message.additional_properties, + ) + + async def get_messages( + self, session_id: str | None, *, state: dict[str, Any] | None = None, **kwargs: Any + ) -> list[Message]: + """Load conversation history from durable entity state.""" + binding = self._binding() + if binding is None: + return [] + + loaded: list[Message] = [] + id_map: dict[str, tuple[DurableAgentStateEntry, int]] = {} + for entry, index in self._replayable_entries(binding): + stored = entry.messages[index] + message = self._to_message(stored) + if message is None: + continue + if not message.message_id: + # Give every loaded message a stable identity so compaction results can be + # reconciled back onto durable state on flush. + message.message_id = f"durable_{id(entry):x}_{index}" + stored.message_id = message.message_id + loaded.append(message) + id_map[message.message_id] = (entry, index) + + if state is not None: + # Expose the loaded messages as the working buffer so CompactionProvider's + # after_strategy can annotate them (core reads session.state[source_id]["messages"]). + state[WORKING_BUFFER_KEY] = loaded + state[POSITIONS_KEY] = id_map + + if self.skip_excluded: + return [m for m in loaded if not m.additional_properties.get(EXCLUDED_KEY)] + return list(loaded) + + async def save_messages( + self, + session_id: str | None, + messages: Sequence[Message], + *, + state: dict[str, Any] | None = None, + **kwargs: Any, + ) -> None: + """No-op: the durable entity appends requests and responses to its own state.""" + return + + async def after_run( + self, + *, + agent: Any, + session: Any, + context: Any, + state: dict[str, Any], + ) -> None: + """Flush compaction annotations from the working buffer into durable state.""" + self.flush(state) + + def flush(self, state: dict[str, Any]) -> None: + """Persist compaction results back into durable entity state. + + Reconciliation is by ``message_id`` rather than position, so strategies that + *insert* messages (for example ``ToolResultCompactionStrategy``, which replaces a + tool-call group with a summary) are handled as well as ones that only annotate. + + Args: + state: The provider-scoped session state holding the working buffer. + """ + binding = current_durable_history_binding() + if binding is None: + return + + raw_buffer = state.get(WORKING_BUFFER_KEY) + raw_positions = state.get(POSITIONS_KEY) + if not isinstance(raw_buffer, list) or not isinstance(raw_positions, dict): + return + buffer = cast("list[Message]", raw_buffer) + stored_by_id = cast("dict[str, tuple[DurableAgentStateEntry, int]]", raw_positions) + + pruned: list[tuple[DurableAgentStateEntry, int]] = [] + # Messages that compaction added (summaries) are inserted right after the last + # known message so ordering in durable state matches the compacted conversation. + last_known: tuple[DurableAgentStateEntry, int] | None = None + + for message in buffer: + annotations = dict(message.additional_properties) if message.additional_properties else None + position = stored_by_id.get(message.message_id) if message.message_id else None + + if position is None: + inserted = self._insert_new_message(binding, message, after=last_known) + if inserted is not None: + last_known = inserted + continue + + entry, index = position + stored = entry.messages[index] + stored.extension_data = annotations + last_known = position + if self.prune_excluded and annotations and annotations.get(EXCLUDED_KEY): + pruned.append(position) + + if pruned: + self._prune(binding, pruned) + + binding.state_provider.persist_state() + + @staticmethod + def _insert_new_message( + binding: DurableHistoryBinding, + message: Message, + *, + after: tuple[DurableAgentStateEntry, int] | None, + ) -> tuple[DurableAgentStateEntry, int] | None: + """Persist a message that compaction produced (for example a summary).""" + stored = DurableAgentStateMessage.from_chat_message(message) + if after is not None: + entry, index = after + entry.messages.insert(index + 1, stored) + return entry, index + 1 + + history = binding.state_provider.state.data.conversation_history + if not history: + return None + first = history[0] + first.messages.insert(0, stored) + return first, 0 + + @staticmethod + def _prune(binding: DurableHistoryBinding, pruned: list[tuple[DurableAgentStateEntry, int]]) -> None: + """Physically remove excluded messages (and any entries left empty).""" + by_entry: dict[int, list[int]] = {} + for entry, index in pruned: + by_entry.setdefault(id(entry), []).append(index) + + for entry, _ in pruned: + indexes = by_entry.pop(id(entry), None) + if indexes is None: + continue + for index in sorted(indexes, reverse=True): + del entry.messages[index] + + history = binding.state_provider.state.data.conversation_history + remaining = [entry for entry in history if entry.messages] + if len(remaining) != len(history): + history[:] = remaining diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py new file mode 100644 index 0000000..1ecdb0e --- /dev/null +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -0,0 +1,304 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Tests for :class:`DurableHistoryProvider` (ADR-0032 Option 6). + +The provider makes durable entity state the store behind core's ``HistoryProvider`` +interface, so conversation history is persisted exactly once and core compaction +plugs in unchanged. +""" + +from collections.abc import AsyncIterable, Awaitable, Sequence +from typing import Any + +from agent_framework import ( + Agent, + ChatResponse, + ChatResponseUpdate, + CompactionProvider, + Content, + InMemoryHistoryProvider, + Message, + ResponseStream, +) + +from agent_framework_durabletask import ( + AgentEntity, + AgentEntityStateProviderMixin, + DurableHistoryProvider, +) + +KEEP_LAST_MESSAGES = 2 + + +class RecordingChatClient: + """Minimal chat client that records the message list it receives per call.""" + + def __init__(self) -> None: + self.additional_properties: dict[str, Any] = {} + self.received_messages: list[list[Message]] = [] + self._counter = 0 + + def get_response( + self, + messages: str | Message | list[str] | list[Message], + *, + stream: bool = False, + options: dict[str, Any] | None = None, + **kwargs: Any, + ) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]: + options = options or {} + normalized = [m for m in messages if isinstance(m, Message)] if isinstance(messages, list) else [] + self.received_messages.append(normalized) + + if stream: + return self._stream(options) + + async def _get() -> ChatResponse: + self._counter += 1 + return ChatResponse(messages=Message(role="assistant", contents=[f"reply-{self._counter}"])) + + return _get() + + def _stream(self, options: dict[str, Any]) -> ResponseStream[ChatResponseUpdate, ChatResponse]: + async def _updates() -> AsyncIterable[ChatResponseUpdate]: + self._counter += 1 + yield ChatResponseUpdate(contents=[Content.from_text(f"reply-{self._counter}")], role="assistant") + + def _finalize(updates: Sequence[ChatResponseUpdate]) -> ChatResponse: + return ChatResponse.from_updates(updates, output_format_type=options.get("response_format")) + + return ResponseStream(_updates(), finalizer=_finalize) + + +class _InMemoryStateProvider(AgentEntityStateProviderMixin): + """Test-only state provider that keeps the serialized entity state in memory.""" + + def __init__(self, *, session_id: str = "durable-history-session") -> None: + self._session_id = session_id + self._state_dict: dict[str, Any] = {} + + def _get_state_dict(self) -> dict[str, Any]: + return self._state_dict + + def _set_state_dict(self, state: dict[str, Any]) -> None: + self._state_dict = state + + def _get_session_id_from_entity(self) -> str: + return self._session_id + + +async def _keep_last_messages(messages: list[Message]) -> bool: + """Compaction strategy: mark everything except the most recent messages as excluded.""" + if len(messages) <= KEEP_LAST_MESSAGES: + return False + changed = False + for message in messages[:-KEEP_LAST_MESSAGES]: + if not message.additional_properties.get("_excluded"): + message.additional_properties["_excluded"] = True + changed = True + return changed + + +async def _summarize_oldest(messages: list[Message]) -> bool: + """Strategy that *inserts* a summary message, mimicking ToolResultCompactionStrategy. + + Uses a stable summary id derived from the messages it replaces, so re-running it must + not create duplicates. + """ + if len(messages) <= KEEP_LAST_MESSAGES: + return False + + older = [m for m in messages[:-KEEP_LAST_MESSAGES] if not m.additional_properties.get("_excluded")] + if not older: + return False + + summary_id = "summary_" + "_".join(sorted(m.message_id or "" for m in older)) + if any(m.message_id == summary_id for m in messages): + return False + + for message in older: + message.additional_properties["_excluded"] = True + message.additional_properties["_summarized_by_summary_id"] = summary_id + + summary = Message( + role="assistant", + contents=[f"[summary of {len(older)} messages]"], + message_id=summary_id, + additional_properties={"_summary_of_message_ids": [m.message_id for m in older]}, + ) + messages.insert(messages.index(older[-1]) + 1, summary) + return True + + +def _build_agent( + client: RecordingChatClient, + *, + with_compaction: bool = False, + prune_excluded: bool = False, + strategy: Any = None, +) -> Agent: + history = DurableHistoryProvider(prune_excluded=prune_excluded) + providers: list[Any] = [history] + if with_compaction: + providers.append( + CompactionProvider( + after_strategy=strategy or _keep_last_messages, + history_source_id=history.source_id, + ) + ) + return Agent(client=client, name="assistant", context_providers=providers) + + +def _make_entity(agent: Agent, provider: _InMemoryStateProvider) -> AgentEntity: + return AgentEntity(agent, state_provider=provider) + + +async def _run_turns(entity: AgentEntity, prompts: list[str]) -> None: + for index, prompt in enumerate(prompts): + await entity.run({"message": prompt, "correlationId": f"corr-{index}"}) + + +def _stored_messages(entity: AgentEntity) -> list[Any]: + return [m for entry in entity.state.data.conversation_history for m in entry.messages] + + +class TestDurableHistoryProvider: + """Durable entity state is the single store behind core's HistoryProvider.""" + + async def test_history_is_stored_once(self) -> None: + """No side-car session blob: messages live only in conversation history.""" + client = RecordingChatClient() + provider = _InMemoryStateProvider() + entity = _make_entity(_build_agent(client), provider) + + await _run_turns(entity, ["first", "second"]) + + persisted = provider._get_state_dict()["data"] + assert "sessionState" not in persisted + assert list(persisted.keys()) == ["conversationHistory"] + assert len(entity.state.data.conversation_history) == 4 + + async def test_provider_supplies_history_across_turns(self) -> None: + """Prior turns are loaded from durable state, not replayed by the entity.""" + client = RecordingChatClient() + entity = _make_entity(_build_agent(client), _InMemoryStateProvider()) + + await _run_turns(entity, ["first", "second", "third"]) + + assert len(client.received_messages[0]) == 1 + assert len(client.received_messages[1]) > len(client.received_messages[0]) + assert len(client.received_messages[2]) > len(client.received_messages[1]) + assert client.received_messages[1][0].text == "first" + + async def test_no_duplicate_of_in_flight_request(self) -> None: + """The in-flight request is delivered as input, not also loaded as history.""" + client = RecordingChatClient() + entity = _make_entity(_build_agent(client), _InMemoryStateProvider()) + + await _run_turns(entity, ["only-once"]) + + texts = [m.text for m in client.received_messages[0]] + assert texts.count("only-once") == 1 + + async def test_compaction_annotations_persist_in_durable_state(self) -> None: + """Core compaction plugs in and its annotations are stored with the messages.""" + client = RecordingChatClient() + entity = _make_entity(_build_agent(client, with_compaction=True), _InMemoryStateProvider()) + + await _run_turns(entity, ["t1", "t2", "t3", "t4", "t5"]) + + excluded = [m for m in _stored_messages(entity) if (m.extension_data or {}).get("_excluded")] + assert excluded, "expected compaction annotations persisted in conversation history" + + # Annotations survive a full serialize/deserialize round-trip of entity state. + from agent_framework_durabletask import DurableAgentState + + restored = DurableAgentState.from_dict(entity.state.to_dict()) + restored_excluded = [ + m + for entry in restored.data.conversation_history + for m in entry.messages + if (m.extension_data or {}).get("_excluded") + ] + assert len(restored_excluded) == len(excluded) + + async def test_compaction_bounds_model_input(self) -> None: + """Excluded messages are withheld from the model, so context stops growing.""" + turns = ["t1", "t2", "t3", "t4", "t5", "t6"] + + plain_client = RecordingChatClient() + await _run_turns(_make_entity(_build_agent(plain_client), _InMemoryStateProvider()), turns) + + compacted_client = RecordingChatClient() + await _run_turns( + _make_entity(_build_agent(compacted_client, with_compaction=True), _InMemoryStateProvider()), + turns, + ) + + assert len(compacted_client.received_messages[-1]) < len(plain_client.received_messages[-1]) + + async def test_prune_excluded_bounds_persisted_state(self) -> None: + """Opt-in pruning physically shrinks durable storage (the lossy L2 step).""" + turns = ["t1", "t2", "t3", "t4", "t5", "t6"] + + kept_entity = _make_entity(_build_agent(RecordingChatClient(), with_compaction=True), _InMemoryStateProvider()) + await _run_turns(kept_entity, turns) + + pruned_entity = _make_entity( + _build_agent(RecordingChatClient(), with_compaction=True, prune_excluded=True), + _InMemoryStateProvider(), + ) + await _run_turns(pruned_entity, turns) + + assert len(_stored_messages(pruned_entity)) < len(_stored_messages(kept_entity)) + # Nothing marked excluded is left behind in storage. + assert not [m for m in _stored_messages(pruned_entity) if (m.extension_data or {}).get("_excluded")] + + async def test_summarizing_strategy_persists_inserted_messages(self) -> None: + """Strategies that insert a summary (not just annotate) are reconciled by message id.""" + client = RecordingChatClient() + entity = _make_entity( + _build_agent(client, with_compaction=True, strategy=_summarize_oldest), + _InMemoryStateProvider(), + ) + + await _run_turns(entity, ["t1", "t2", "t3", "t4"]) + + stored = _stored_messages(entity) + summaries = [m for m in stored if m.message_id and m.message_id.startswith("summary_")] + assert summaries, "expected the inserted summary message to be persisted" + + # Identity and annotations survive a durable state round-trip. + from agent_framework_durabletask import DurableAgentState + + restored = DurableAgentState.from_dict(entity.state.to_dict()) + restored_ids = [ + m.message_id + for entry in restored.data.conversation_history + for m in entry.messages + if m.message_id and m.message_id.startswith("summary_") + ] + assert restored_ids == [m.message_id for m in summaries] + + async def test_summary_is_not_duplicated_across_turns(self) -> None: + """Re-running compaction with a stable summary id must not append duplicates.""" + client = RecordingChatClient() + entity = _make_entity( + _build_agent(client, with_compaction=True, strategy=_summarize_oldest), + _InMemoryStateProvider(), + ) + + await _run_turns(entity, ["t1", "t2", "t3", "t4", "t5", "t6"]) + + ids = [m.message_id for m in _stored_messages(entity) if m.message_id] + assert len(ids) == len(set(ids)), f"duplicate message ids persisted: {ids}" + + async def test_without_durable_provider_legacy_replay_is_used(self) -> None: + """Agents without the provider keep the original full-replay behavior.""" + client = RecordingChatClient() + agent = Agent(client=client, name="assistant", context_providers=[InMemoryHistoryProvider()]) + entity = _make_entity(agent, _InMemoryStateProvider()) + + await _run_turns(entity, ["first", "second"]) + + assert len(entity.state.data.conversation_history) == 4 From 9a2dfc320670086cc3724d107dac4b8e57777901 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 11:16:21 -0500 Subject: [PATCH 03/32] feat: forward workflow conversation context to durable agents (ADR 0032 L3) --- .../0032-durable-thread-compaction.md | 26 +++ .../_durable_agent_state.py | 10 +- .../agent_framework_durabletask/_entities.py | 24 +++ .../agent_framework_durabletask/_executors.py | 4 + .../_history_provider.py | 21 ++ .../agent_framework_durabletask/_models.py | 15 +- .../agent_framework_durabletask/_shim.py | 8 + .../_workflows/context.py | 10 +- .../_workflows/dt_context.py | 10 +- .../_workflows/orchestrator.py | 52 ++++- .../tests/test_durable_history_provider.py | 31 +++ .../tests/test_workflow_context_parity.py | 191 ++++++++++++++++++ 12 files changed, 394 insertions(+), 8 deletions(-) create mode 100644 python/packages/durabletask/tests/test_workflow_context_parity.py diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index 50c8c28..00c11e7 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -289,6 +289,32 @@ Consequence for ordering: core runs `before_run` forward and `after_run` in **re (convenient), but it sees history only as of the **previous** turn - so context reaches a steady state rather than shrinking immediately. This is expected, not a defect. +## L3 Realization: Workflow Context Parity + +In-process workflows give a downstream `AgentExecutor` the upstream conversation through +`AgentExecutorResponse.full_conversation`, governed by `context_mode` (`full` | `last_agent` | +`custom` + `context_filter`). The durable orchestrator previously flattened that to the **last +message's text**, so a downstream agent lost everything earlier nodes produced. + +Durable now projects the same conversation and delivers it to the agent entity: + +- The orchestrator reads the executor's `context_mode`/`context_filter` and projects + `full_conversation` accordingly. +- The projection travels as `RunRequest.context_messages` (serialized `Message` values) and becomes + the request entry's messages, so it is persisted like any other conversation content and is + visible to compaction. +- A node that runs more than once (a cycle) receives the whole upstream conversation again, so the + entity **drops messages whose id it has already recorded**, keeping at least the latest message so + the agent always has an input. This relies on the persisted `messageId` described above. + +Behavior difference that remains, by design: each agent node also keeps its **own durable history** +(keyed by workflow instance + executor), so per-agent memory survives restarts and is compacted +independently - a superset of the in-process behavior rather than a strict match. + +**Service-managed sessions** are a no-op at every layer: when a session carries a +`service_session_id` the model service owns the conversation, so the durable history provider +neither loads nor flushes. + ## More Information - Builds on [ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md) (context compaction strategy), diff --git a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py index 0bf9a94..321181b 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py +++ b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py @@ -611,10 +611,18 @@ def from_dict(cls, data: dict[str, Any]) -> DurableAgentStateRequest: @staticmethod def from_run_request(request: RunRequest) -> DurableAgentStateRequest: + # A workflow may deliver the upstream conversation instead of a single message. + if request.context_messages: + messages = [ + DurableAgentStateMessage.from_chat_message(Message.from_dict(raw)) for raw in request.context_messages + ] + else: + messages = [DurableAgentStateMessage.from_run_request(request)] + # Determine response_type based on response_format return DurableAgentStateRequest( correlation_id=request.correlation_id, - messages=[DurableAgentStateMessage.from_run_request(request)], + messages=messages, created_at=_parse_created_at(request.created_at), response_type=request.request_response_format, response_schema=serialize_response_format(request.response_format), diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 833734e..641e0aa 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -176,6 +176,8 @@ async def run( logger.debug("[AgentEntity.run] Received SessionId %s Message: %s", session_id, run_request) state_request = DurableAgentStateRequest.from_run_request(run_request) + if run_request.context_messages: + state_request.messages = self._drop_already_stored(state_request.messages) self.state.data.conversation_history.append(state_request) durable_history = self._find_durable_history_provider() @@ -249,6 +251,28 @@ async def run( if binding_token is not None: unbind_durable_history(binding_token) + def _drop_already_stored(self, messages: list[DurableAgentStateMessage]) -> list[DurableAgentStateMessage]: + """Filter out upstream context messages this entity has already recorded. + + A workflow node that runs more than once (for example in a cycle) receives the whole + upstream conversation each time. Messages carrying an id that is already in this + entity's history are dropped so the conversation is not duplicated. The final message + is always kept so the agent still receives an input. + """ + known_ids = { + stored.message_id + for entry in self.state.data.conversation_history + for stored in entry.messages + if stored.message_id + } + if not known_ids: + return messages + + deduped = [m for m in messages if not m.message_id or m.message_id not in known_ids] + if not deduped and messages: + return [messages[-1]] + return deduped + def _find_durable_history_provider(self) -> DurableHistoryProvider | None: """Return the agent's :class:`DurableHistoryProvider`, if it is configured with one.""" providers = getattr(self.agent, "context_providers", None) diff --git a/python/packages/durabletask/agent_framework_durabletask/_executors.py b/python/packages/durabletask/agent_framework_durabletask/_executors.py index eea17ef..1b97b08 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_executors.py +++ b/python/packages/durabletask/agent_framework_durabletask/_executors.py @@ -160,6 +160,7 @@ def get_run_request( message: str, *, options: dict[str, Any] | None = None, + context_messages: list[dict[str, Any]] | None = None, ) -> RunRequest: """Create a RunRequest from message and options.""" correlation_id = self.generate_unique_id() @@ -179,6 +180,7 @@ def get_run_request( wait_for_response=wait_for_response, correlation_id=correlation_id, options=opts, + context_messages=context_messages, ) def _create_acceptance_response(self, correlation_id: str) -> AgentResponse: @@ -454,6 +456,7 @@ def get_run_request( message: str, *, options: dict[str, Any] | None = None, + context_messages: list[dict[str, Any]] | None = None, ) -> RunRequest: """Get the current run request from the orchestration context. @@ -463,6 +466,7 @@ def get_run_request( request = super().get_run_request( message, options=options, + context_messages=context_messages, ) request.orchestration_id = self._context.instance_id return request diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py index 1ad11ae..e93b60f 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -194,6 +194,25 @@ async def save_messages( """No-op: the durable entity appends requests and responses to its own state.""" return + @staticmethod + def _is_service_managed(session: Any) -> bool: + """Return whether the conversation is stored by the model service, not by us.""" + return bool(getattr(session, "service_session_id", None)) + + async def before_run( + self, + *, + agent: Any, + session: Any, + context: Any, + state: dict[str, Any], + ) -> None: + """Load durable history into context, unless the service owns the conversation.""" + if self._is_service_managed(session): + logger.debug("[DurableHistoryProvider] Session is service-managed; skipping durable history load.") + return + await super().before_run(agent=agent, session=session, context=context, state=state) + async def after_run( self, *, @@ -203,6 +222,8 @@ async def after_run( state: dict[str, Any], ) -> None: """Flush compaction annotations from the working buffer into durable state.""" + if self._is_service_managed(session): + return self.flush(state) def flush(self, state: dict[str, Any]) -> None: diff --git a/python/packages/durabletask/agent_framework_durabletask/_models.py b/python/packages/durabletask/agent_framework_durabletask/_models.py index e8eabca..f6ac97d 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_models.py +++ b/python/packages/durabletask/agent_framework_durabletask/_models.py @@ -109,6 +109,11 @@ class RunRequest: created_at: Optional timestamp when the request was created orchestration_id: Optional ID of the orchestration that initiated this request options: Optional options dictionary forwarded to the agent + context_messages: Optional upstream conversation (serialized ``Message`` dicts) that should + be delivered to the agent as the request's messages. Workflows use this to give a + downstream agent the conversation produced by upstream nodes, matching the in-process + ``AgentExecutor`` context behavior. When set, it replaces ``message`` as the + request payload; ``message`` still carries the latest text for logging. """ message: str @@ -121,6 +126,7 @@ class RunRequest: created_at: datetime | None = None orchestration_id: str | None = None options: dict[str, Any] = field(default_factory=lambda: {}) + context_messages: list[dict[str, Any]] | None = None def __init__( self, @@ -134,6 +140,7 @@ def __init__( created_at: datetime | None = None, orchestration_id: str | None = None, options: dict[str, Any] | None = None, + context_messages: list[dict[str, Any]] | None = None, ) -> None: self.message = message self.correlation_id = correlation_id @@ -145,6 +152,7 @@ def __init__( self.created_at = created_at if created_at is not None else datetime.now(tz=timezone.utc) self.orchestration_id = orchestration_id self.options = options if options is not None else {} + self.context_messages = context_messages @staticmethod def coerce_role(value: str | None) -> str: @@ -158,7 +166,7 @@ def coerce_role(value: str | None) -> str: def to_dict(self) -> dict[str, Any]: """Convert to dictionary for JSON serialization.""" - result = { + result: dict[str, Any] = { "message": self.message, "enable_tool_calls": self.enable_tool_calls, "wait_for_response": self.wait_for_response, @@ -173,6 +181,8 @@ def to_dict(self) -> dict[str, Any]: result["created_at"] = self.created_at.isoformat() if self.orchestration_id: result["orchestrationId"] = self.orchestration_id + if self.context_messages: + result["contextMessages"] = self.context_messages return result @classmethod @@ -200,6 +210,8 @@ def from_dict(cls, data: dict[str, Any]) -> RunRequest: raise ValueError("correlationId is required in RunRequest data") options = data.get("options") + raw_context = data.get("contextMessages") + context_messages = cast("list[dict[str, Any]]", raw_context) if isinstance(raw_context, list) else None return cls( message=data.get("message", ""), @@ -212,6 +224,7 @@ def from_dict(cls, data: dict[str, Any]) -> RunRequest: created_at=created_at, orchestration_id=data.get("orchestrationId"), options=cast(dict[str, Any], options) if isinstance(options, dict) else {}, + context_messages=context_messages, ) diff --git a/python/packages/durabletask/agent_framework_durabletask/_shim.py b/python/packages/durabletask/agent_framework_durabletask/_shim.py index e6e9f5d..6340033 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_shim.py +++ b/python/packages/durabletask/agent_framework_durabletask/_shim.py @@ -92,6 +92,7 @@ def run( # type: ignore[override] stream: Literal[False] = False, session: AgentSession | None = None, options: dict[str, Any] | None = None, + context_messages: list[dict[str, Any]] | None = None, ) -> TaskT: """Execute the agent via the injected provider. @@ -103,6 +104,9 @@ def run( # type: ignore[override] options: Optional options dictionary. Supported keys include ``response_format``, ``enable_tool_calls``, and ``wait_for_response``. Additional keys are forwarded to the agent execution. + context_messages: Optional upstream conversation (serialized ``Message`` dicts) + delivered to the agent as prior context. Workflows use this to give a + downstream agent the conversation produced by upstream nodes. Note: This method overrides SupportsAgentRun.run() with a different return type: @@ -122,9 +126,13 @@ def run( # type: ignore[override] raise ValueError("DurableAIAgent does not support streaming mode (stream must be False)") message_str = self._normalize_messages(messages) + # Only forward context messages when a workflow supplied them, so executors that do + # not implement the parameter keep working unchanged. + extra: dict[str, Any] = {"context_messages": context_messages} if context_messages else {} run_request = self._executor.get_run_request( message=message_str, options=options, + **extra, ) return self._executor.run_durable_agent( diff --git a/python/packages/durabletask/agent_framework_durabletask/_workflows/context.py b/python/packages/durabletask/agent_framework_durabletask/_workflows/context.py index d757d00..d129148 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_workflows/context.py +++ b/python/packages/durabletask/agent_framework_durabletask/_workflows/context.py @@ -73,13 +73,21 @@ def current_utc_datetime(self) -> datetime: """The current replay-safe UTC datetime.""" ... - def prepare_agent_task(self, executor_id: str, message: str, orchestration_instance_id: str) -> Any: + def prepare_agent_task( + self, + executor_id: str, + message: str, + orchestration_instance_id: str, + context_messages: list[dict[str, Any]] | None = None, + ) -> Any: """Create a yieldable task that runs an agent executor. Args: executor_id: Agent name / executor ID. message: The text message to send to the agent. orchestration_instance_id: Instance ID used as the entity session key. + context_messages: Optional upstream conversation (serialized ``Message`` dicts) + delivered to the agent as prior context. Returns: A yieldable task whose result is an ``AgentResponse``. diff --git a/python/packages/durabletask/agent_framework_durabletask/_workflows/dt_context.py b/python/packages/durabletask/agent_framework_durabletask/_workflows/dt_context.py index 7388a0a..4892b31 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_workflows/dt_context.py +++ b/python/packages/durabletask/agent_framework_durabletask/_workflows/dt_context.py @@ -57,11 +57,17 @@ def current_utc_datetime(self) -> datetime: # -- Agent / Activity dispatch -------------------------------------------- - def prepare_agent_task(self, executor_id: str, message: str, orchestration_instance_id: str) -> Any: + def prepare_agent_task( + self, + executor_id: str, + message: str, + orchestration_instance_id: str, + context_messages: list[dict[str, Any]] | None = None, + ) -> Any: session_id = AgentSessionId(name=executor_id, key=orchestration_instance_id) session = DurableAgentSession(durable_session_id=session_id) agent = DurableAIAgent(self._executor, executor_id) - return agent.run(message, session=session) + return agent.run(message, session=session, context_messages=context_messages) def prepare_activity_task(self, activity_name: str, input_json: str) -> Any: return cast(Any, self._context.call_activity(activity_name, input=input_json)) diff --git a/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py b/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py index 3116ab9..d5f4340 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py +++ b/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py @@ -246,8 +246,38 @@ def build_agent_executor_response( # ============================================================================ +def _build_context_messages(executor: AgentExecutor, message: Any) -> list[dict[str, Any]] | None: + """Project the upstream conversation into messages for a downstream agent. + + Mirrors the in-process :class:`AgentExecutor` context behavior so a workflow behaves the + same way durably: ``full`` forwards the whole upstream conversation, ``last_agent`` only the + previous agent's messages, and ``custom`` applies the executor's ``context_filter``. + + Returns ``None`` when there is no upstream conversation to forward (for example the first + node in a workflow, which receives the raw input instead). + """ + if not isinstance(message, AgentExecutorResponse): + return None + + mode = getattr(executor, "_context_mode", "full") + if mode == "last_agent": + selected = list(message.agent_response.messages) if message.agent_response else [] + elif mode == "custom": + context_filter = getattr(executor, "_context_filter", None) + if context_filter is None: + return None + selected = list(context_filter(list(message.full_conversation))) + else: + selected = list(message.full_conversation) + + if not selected: + return None + return [m.to_dict() for m in selected] + + def _prepare_agent_task( ctx: WorkflowOrchestrationContext, + executor: AgentExecutor, executor_id: str, message: Any, workflow_name: str, @@ -259,10 +289,14 @@ def _prepare_agent_task( executor id dispatch to distinct entities (the entity layer prefixes this with ``dafx-``). The session *key* stays the orchestration instance id, so conversation state remains isolated per run. + + Any upstream conversation is forwarded as context messages so a downstream agent sees + what earlier nodes produced, matching in-process workflow behavior. """ message_content = _extract_message_content(message) + context_messages = _build_context_messages(executor, message) scoped_id = workflow_scoped_executor_id(workflow_name, executor_id) - return ctx.prepare_agent_task(scoped_id, message_content, ctx.instance_id) + return ctx.prepare_agent_task(scoped_id, message_content, ctx.instance_id, context_messages) def _prepare_activity_task( @@ -945,7 +979,13 @@ def _prepare_all_tasks( remaining = messages_list[1:] logger.debug("Preparing agent task: %s", executor_id) - task = _prepare_agent_task(ctx, first_msg[0], first_msg[1], workflow.name) + task = _prepare_agent_task( + ctx, + cast(AgentExecutor, workflow.executors[first_msg[0]]), + first_msg[0], + first_msg[1], + workflow.name, + ) all_tasks.append(task) task_metadata_list.append( TaskMetadata( @@ -1159,7 +1199,13 @@ def publish_live_status( # Phase 3: Process sequential agent messages for executor_id, message, _source_executor_id in remaining_agent_messages: logger.debug("Processing sequential message for agent: %s", executor_id) - task = _prepare_agent_task(ctx, executor_id, message, workflow.name) + task = _prepare_agent_task( + ctx, + cast(AgentExecutor, workflow.executors[executor_id]), + executor_id, + message, + workflow.name, + ) agent_response: AgentResponse = yield task logger.debug("Agent %s sequential response completed", executor_id) diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py index 1ecdb0e..c058d97 100644 --- a/python/packages/durabletask/tests/test_durable_history_provider.py +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -293,6 +293,37 @@ async def test_summary_is_not_duplicated_across_turns(self) -> None: ids = [m.message_id for m in _stored_messages(entity) if m.message_id] assert len(ids) == len(set(ids)), f"duplicate message ids persisted: {ids}" + async def test_service_managed_session_is_skipped(self) -> None: + """When the model service owns the conversation, the provider must not participate.""" + from types import SimpleNamespace + + from agent_framework_durabletask._history_provider import ( + DurableHistoryBinding, + bind_durable_history, + unbind_durable_history, + ) + + client = RecordingChatClient() + provider = _InMemoryStateProvider() + entity = _make_entity(_build_agent(client), provider) + await _run_turns(entity, ["first", "second"]) + + history = DurableHistoryProvider() + token = bind_durable_history(DurableHistoryBinding(state_provider=provider)) + try: + state: dict[str, Any] = {} + context = SimpleNamespace(session_id="s", extend_messages=lambda *_: None) + service_session = SimpleNamespace(service_session_id="svc-123", state={}) + + await history.before_run(agent=None, session=service_session, context=context, state=state) + # Nothing was loaded, so no working buffer was published. + assert "messages" not in state + + # Flushing is likewise a no-op and must not raise. + await history.after_run(agent=None, session=service_session, context=context, state=state) + finally: + unbind_durable_history(token) + async def test_without_durable_provider_legacy_replay_is_used(self) -> None: """Agents without the provider keep the original full-replay behavior.""" client = RecordingChatClient() diff --git a/python/packages/durabletask/tests/test_workflow_context_parity.py b/python/packages/durabletask/tests/test_workflow_context_parity.py new file mode 100644 index 0000000..71893b0 --- /dev/null +++ b/python/packages/durabletask/tests/test_workflow_context_parity.py @@ -0,0 +1,191 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Tests for workflow context parity (ADR-0032 L3). + +In-process workflows hand a downstream ``AgentExecutor`` the upstream conversation via +``AgentExecutorResponse.full_conversation``. These tests cover the durable equivalent: +the orchestrator projects that conversation into ``RunRequest.context_messages`` honoring +``context_mode``/``context_filter``, and the entity records it without duplication. +""" + +from typing import Any + +from agent_framework import ( + AgentExecutor, + AgentExecutorResponse, + AgentResponse, + Message, +) + +from agent_framework_durabletask import ( + AgentEntity, + AgentEntityStateProviderMixin, + DurableAgentStateRequest, + RunRequest, +) +from agent_framework_durabletask._workflows.orchestrator import _build_context_messages + + +class _StubAgent: + """Minimal agent stand-in for constructing an AgentExecutor.""" + + def __init__(self, name: str = "stub") -> None: + self.name = name + self.id = name + self.description = None + + async def run(self, messages: Any = None, **kwargs: Any) -> AgentResponse: + return AgentResponse(messages=[Message(role="assistant", contents=["ok"])]) + + def create_session(self, **kwargs: Any) -> Any: + from agent_framework import AgentSession + + return AgentSession() + + +class _InMemoryStateProvider(AgentEntityStateProviderMixin): + def __init__(self, *, session_id: str = "wf-session") -> None: + self._session_id = session_id + self._state_dict: dict[str, Any] = {} + + def _get_state_dict(self) -> dict[str, Any]: + return self._state_dict + + def _set_state_dict(self, state: dict[str, Any]) -> None: + self._state_dict = state + + def _get_session_id_from_entity(self) -> str: + return self._session_id + + +def _upstream_response(*, texts: list[str], agent_text: str) -> AgentExecutorResponse: + conversation = [Message(role="user", contents=[t], message_id=f"m{i}") for i, t in enumerate(texts)] + agent_message = Message(role="assistant", contents=[agent_text], message_id="agent-msg") + conversation.append(agent_message) + return AgentExecutorResponse( + executor_id="upstream", + agent_response=AgentResponse(messages=[agent_message]), + full_conversation=conversation, + ) + + +class TestContextProjection: + """The orchestrator projects upstream conversation per context_mode.""" + + def test_full_mode_forwards_entire_conversation(self) -> None: + executor = AgentExecutor(_StubAgent(), id="downstream") + upstream = _upstream_response(texts=["first", "second"], agent_text="reply") + + projected = _build_context_messages(executor, upstream) + + assert projected is not None + assert len(projected) == 3 + + def test_last_agent_mode_forwards_only_agent_messages(self) -> None: + executor = AgentExecutor(_StubAgent(), id="downstream", context_mode="last_agent") + upstream = _upstream_response(texts=["first", "second"], agent_text="reply") + + projected = _build_context_messages(executor, upstream) + + assert projected is not None + assert len(projected) == 1 + + def test_custom_mode_uses_context_filter(self) -> None: + executor = AgentExecutor( + _StubAgent(), + id="downstream", + context_mode="custom", + context_filter=lambda messages: messages[-2:], + ) + upstream = _upstream_response(texts=["first", "second"], agent_text="reply") + + projected = _build_context_messages(executor, upstream) + + assert projected is not None + assert len(projected) == 2 + + def test_non_agent_input_has_no_upstream_context(self) -> None: + """The first node receives raw input, so there is no conversation to forward.""" + executor = AgentExecutor(_StubAgent(), id="downstream") + + assert _build_context_messages(executor, "plain input") is None + + +class TestEntityContextIngestion: + """The entity records forwarded context and does not duplicate it.""" + + def _request(self, messages: list[Message], correlation_id: str) -> RunRequest: + return RunRequest( + message=messages[-1].text or "", + correlation_id=correlation_id, + context_messages=[m.to_dict() for m in messages], + ) + + def test_context_messages_become_request_messages(self) -> None: + messages = [ + Message(role="user", contents=["hello"], message_id="m0"), + Message(role="assistant", contents=["hi"], message_id="m1"), + ] + + entry = DurableAgentStateRequest.from_run_request(self._request(messages, "corr-0")) + + assert [m.message_id for m in entry.messages] == ["m0", "m1"] + + def test_repeated_context_is_not_duplicated(self) -> None: + """A node that runs twice in a cycle must not re-record the same conversation.""" + provider = _InMemoryStateProvider() + entity = AgentEntity(_StubAgent(), state_provider=provider) + + first = [Message(role="user", contents=["hello"], message_id="m0")] + entity.state.data.conversation_history.append( + DurableAgentStateRequest.from_run_request(self._request(first, "corr-0")) + ) + + repeated = [ + Message(role="user", contents=["hello"], message_id="m0"), + Message(role="assistant", contents=["new"], message_id="m1"), + ] + entry = DurableAgentStateRequest.from_run_request(self._request(repeated, "corr-1")) + entry.messages = entity._drop_already_stored(entry.messages) + + assert [m.message_id for m in entry.messages] == ["m1"] + + def test_fully_duplicate_context_keeps_last_message(self) -> None: + """The agent must always receive at least one input message.""" + provider = _InMemoryStateProvider() + entity = AgentEntity(_StubAgent(), state_provider=provider) + + messages = [Message(role="user", contents=["hello"], message_id="m0")] + entity.state.data.conversation_history.append( + DurableAgentStateRequest.from_run_request(self._request(messages, "corr-0")) + ) + + entry = DurableAgentStateRequest.from_run_request(self._request(messages, "corr-1")) + entry.messages = entity._drop_already_stored(entry.messages) + + assert [m.message_id for m in entry.messages] == ["m0"] + + +class TestRunRequestRoundTrip: + """context_messages survives the entity wire format.""" + + def test_context_messages_round_trip(self) -> None: + messages = [Message(role="user", contents=["hello"], message_id="m0")] + request = RunRequest( + message="hello", + correlation_id="corr-0", + context_messages=[m.to_dict() for m in messages], + ) + + restored = RunRequest.from_dict(request.to_dict()) + + assert restored.context_messages is not None + assert len(restored.context_messages) == 1 + + def test_absent_context_messages_stay_none(self) -> None: + request = RunRequest(message="hello", correlation_id="corr-0") + + restored = RunRequest.from_dict(request.to_dict()) + + assert restored.context_messages is None + assert "contextMessages" not in request.to_dict() From d2054161a96fa165f2674015f3ddfd89c150be35 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 13:15:13 -0500 Subject: [PATCH 04/32] feat: back registered agents with durable history automatically (ADR 0032) --- .../0032-durable-thread-compaction.md | 34 ++++ .../agent_framework_durabletask/_entities.py | 5 +- .../_history_provider.py | 81 ++++++++- .../tests/test_durable_history_autoswap.py | 163 ++++++++++++++++++ .../tests/test_durable_history_provider.py | 10 +- 5 files changed, 289 insertions(+), 4 deletions(-) create mode 100644 python/packages/durabletask/tests/test_durable_history_autoswap.py diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index 00c11e7..d81143c 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -315,6 +315,40 @@ independently - a superset of the in-process behavior rather than a strict match `service_session_id` the model service owns the conversation, so the durable history provider neither loads nor flushes. +## Zero-Configuration Registration + +The parity goal is only met if a user can take an agent that **already works in core**, register it +with `AgentFunctionApp` (or the worker, or as a workflow node), and get durable behavior with **no +edits to the agent**. Requiring them to add a durable-specific provider would just relocate the +configuration burden. + +So the durable entity substitutes the history provider at construction time - covering every +registration path, since both the worker and the Azure Functions host build the same entity. The +agent is never mutated: when a substitution is needed, a shallow copy with its own provider list is +used, so the caller's agent still behaves normally in-process. + +| User configured | Durable behavior | +| --- | --- | +| Nothing | Inject a durable history provider, using the `source_id` core's auto-injected provider would have - so default-wired compaction still resolves. No compaction by default (same as core). | +| `InMemoryHistoryProvider` (± compaction) | Replace with the durable provider, **preserving `source_id` and `skip_excluded`** so any attached `CompactionProvider` keeps working untouched. | +| Cosmos / Redis / file / custom provider | **Leave alone.** The user chose where their conversation lives; durable still supplies execution durability. | +| Service-managed history | **Leave alone.** The model service owns the conversation. | +| Agent without the core context pipeline | **Leave alone.** Falls back to replaying persisted history. | + +Preserving `source_id` is the load-bearing detail: `CompactionProvider` locates history through +`history_source_id` (default `"in_memory"`), so a provider swapped in under the same id is invisible +to the rest of the user's configuration. Because the injected provider is a `HistoryProvider` with +`load_messages=True`, core's own auto-injection sees a provider present and stands down - no +duplicate provider. + +An explicit `DurableHistoryProvider` remains supported as an advanced escape hatch, for example to +enable `prune_excluded`. + +**Side effect worth noting:** passing a session is what re-engages the context-provider pipeline, so +external history providers (Cosmos, Redis, file) now function under the durable runtime as well - +previously they were silently ignored because no session was ever created. Store-side compaction +still no-ops for those providers (core interface gap 1 below); only the in-run filter applies. + ## More Information - Builds on [ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md) (context compaction strategy), diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 641e0aa..0d4d65d 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -33,6 +33,7 @@ DurableHistoryBinding, DurableHistoryProvider, bind_durable_history, + ensure_durable_history, unbind_durable_history, ) from ._models import RunRequest @@ -125,7 +126,9 @@ def __init__( *, state_provider: AgentEntityStateProviderMixin, ) -> None: - self.agent = agent + # Back the agent's conversation history with durable entity state so an agent that + # already works in core runs durably without any configuration change. + self.agent = ensure_durable_history(agent) self.callback = callback self._state_provider = state_provider diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py index e93b60f..9ec996c 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -13,13 +13,14 @@ from __future__ import annotations +import copy import logging from collections.abc import Iterator, Sequence from contextvars import ContextVar, Token from dataclasses import dataclass from typing import TYPE_CHECKING, Any, cast -from agent_framework import HistoryProvider, Message +from agent_framework import HistoryProvider, InMemoryHistoryProvider, Message, SupportsAgentRun from ._durable_agent_state import DurableAgentStateEntry, DurableAgentStateMessage, DurableAgentStateResponse @@ -313,3 +314,81 @@ def _prune(binding: DurableHistoryBinding, pruned: list[tuple[DurableAgentStateE remaining = [entry for entry in history if entry.messages] if len(remaining) != len(history): history[:] = remaining + + +def _service_stores_history(agent: Any) -> bool: + """Return whether the agent's client keeps conversation history server-side.""" + client = getattr(agent, "client", None) + return bool(getattr(client, "STORES_BY_DEFAULT", False)) + + +def ensure_durable_history(agent: SupportsAgentRun) -> SupportsAgentRun: + """Back an agent's conversation history with durable entity state. + + Lets a user register an agent that already works in core and get durable behavior with no + configuration change. The agent is never mutated: when a substitution is needed a shallow + copy is returned with its own provider list. + + The rules mirror what core would do, so behavior stays predictable: + + * **No history provider** - a :class:`DurableHistoryProvider` is added. It uses the same + ``source_id`` core's auto-injected provider would have, so a ``CompactionProvider`` left on + its defaults still finds it. + * **In-memory history** - replaced by a :class:`DurableHistoryProvider` carrying the *same* + ``source_id`` and ``skip_excluded``, so any compaction wired to it keeps working untouched. + * **Any other history provider** (Cosmos, Redis, file, custom) - left alone. The user chose + where their conversation lives; durable still provides execution durability. + * **Service-managed history** - left alone. The model service owns the conversation. + * **Agents without the core context pipeline** - left alone; the entity falls back to + replaying its own persisted history. + + Args: + agent: The agent being registered with the durable runtime. + + Returns: + The agent to run, either unchanged or a shallow copy with durable-backed history. + """ + providers = getattr(agent, "context_providers", None) + if not isinstance(providers, (list, tuple)): + return agent + + if _service_stores_history(agent): + logger.debug( + "[DurableHistoryProvider] Agent %s stores history service-side; leaving providers unchanged.", + getattr(agent, "name", type(agent).__name__), + ) + return agent + + provider_list = list(cast("Sequence[Any]", providers)) + existing = next( + (p for p in provider_list if isinstance(p, HistoryProvider) and p.load_messages), + None, + ) + + if existing is None: + # Match the source_id core's auto-injected provider would use so default-wired + # compaction keeps resolving. + updated = [DurableHistoryProvider(source_id=InMemoryHistoryProvider.DEFAULT_SOURCE_ID), *provider_list] + elif isinstance(existing, InMemoryHistoryProvider): + replacement = DurableHistoryProvider( + source_id=existing.source_id, + skip_excluded=existing.skip_excluded, + ) + updated = [replacement if p is existing else p for p in provider_list] + else: + # A deliberate storage choice (external or custom); do not override it. + return agent + + try: + clone = copy.copy(agent) + clone.context_providers = updated # type: ignore[attr-defined] + except Exception: + logger.warning( + "[DurableHistoryProvider] Could not attach durable history to agent %s; " + "falling back to replaying persisted history.", + getattr(agent, "name", type(agent).__name__), + exc_info=True, + ) + return agent + + return clone diff --git a/python/packages/durabletask/tests/test_durable_history_autoswap.py b/python/packages/durabletask/tests/test_durable_history_autoswap.py new file mode 100644 index 0000000..b1a9d86 --- /dev/null +++ b/python/packages/durabletask/tests/test_durable_history_autoswap.py @@ -0,0 +1,163 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Tests for automatic durable history backing (ADR-0032). + +A user should be able to take an agent that already works in core, register it with the +durable runtime, and get durable conversation history with no configuration change. +These tests cover the substitution rules and confirm the user's agent is never mutated. +""" + +from typing import Any + +from agent_framework import ( + Agent, + HistoryProvider, + InMemoryHistoryProvider, + Message, +) + +from agent_framework_durabletask import AgentEntity, AgentEntityStateProviderMixin, DurableHistoryProvider +from agent_framework_durabletask._history_provider import ensure_durable_history + + +class _StubClient: + """Chat client stand-in that stores history locally (the common case).""" + + STORES_BY_DEFAULT = False + + def __init__(self) -> None: + self.additional_properties: dict[str, Any] = {} + + +class _ServiceStoringClient(_StubClient): + """Chat client whose service keeps the conversation server-side.""" + + STORES_BY_DEFAULT = True + + +class _ExternalHistoryProvider(HistoryProvider): + """Stand-in for Cosmos/Redis/file-backed history the user chose deliberately.""" + + def __init__(self) -> None: + super().__init__(source_id="external") + + async def get_messages(self, session_id: str | None, **kwargs: Any) -> list[Message]: + return [] + + async def save_messages(self, session_id: str | None, messages: Any, **kwargs: Any) -> None: + return None + + +class _InMemoryStateProvider(AgentEntityStateProviderMixin): + def __init__(self, *, session_id: str = "autoswap-session") -> None: + self._session_id = session_id + self._state_dict: dict[str, Any] = {} + + def _get_state_dict(self) -> dict[str, Any]: + return self._state_dict + + def _set_state_dict(self, state: dict[str, Any]) -> None: + self._state_dict = state + + def _get_session_id_from_entity(self) -> str: + return self._session_id + + +def _history_providers(agent: Any) -> list[Any]: + return [p for p in agent.context_providers if isinstance(p, HistoryProvider)] + + +class TestAutomaticDurableHistory: + """The durable runtime substitutes durable-backed history where appropriate.""" + + def test_agent_without_providers_gets_durable_history(self) -> None: + agent = Agent(client=_StubClient(), name="a") + + prepared = ensure_durable_history(agent) + + providers = _history_providers(prepared) + assert len(providers) == 1 + assert isinstance(providers[0], DurableHistoryProvider) + # Uses the source id core's auto-injected provider would have, so a + # default-configured CompactionProvider still resolves it. + assert providers[0].source_id == InMemoryHistoryProvider.DEFAULT_SOURCE_ID + + def test_in_memory_history_is_replaced_preserving_source_id(self) -> None: + agent = Agent( + client=_StubClient(), + name="a", + context_providers=[InMemoryHistoryProvider(source_id="custom_slot", skip_excluded=True)], + ) + + prepared = ensure_durable_history(agent) + + providers = _history_providers(prepared) + assert len(providers) == 1 + replacement = providers[0] + assert isinstance(replacement, DurableHistoryProvider) + # Preserving these is what keeps an existing CompactionProvider wired up. + assert replacement.source_id == "custom_slot" + assert replacement.skip_excluded is True + + def test_external_history_provider_is_left_alone(self) -> None: + """The user deliberately chose their own storage; durable must not override it.""" + external = _ExternalHistoryProvider() + agent = Agent(client=_StubClient(), name="a", context_providers=[external]) + + prepared = ensure_durable_history(agent) + + assert prepared is agent + assert _history_providers(prepared) == [external] + + def test_service_managed_history_is_left_alone(self) -> None: + agent = Agent(client=_ServiceStoringClient(), name="a") + + prepared = ensure_durable_history(agent) + + assert prepared is agent + assert not _history_providers(prepared) + + def test_existing_durable_provider_is_untouched(self) -> None: + """Explicit configuration (for example to enable pruning) wins.""" + explicit = DurableHistoryProvider(prune_excluded=True) + agent = Agent(client=_StubClient(), name="a", context_providers=[explicit]) + + prepared = ensure_durable_history(agent) + + assert prepared is agent + assert _history_providers(prepared) == [explicit] + + def test_agent_without_context_pipeline_is_left_alone(self) -> None: + """Custom agents that do not expose context_providers keep legacy replay.""" + + class _CustomAgent: + name = "custom" + + async def run(self, *args: Any, **kwargs: Any) -> Any: ... + + agent = _CustomAgent() + + assert ensure_durable_history(agent) is agent # type: ignore[arg-type] + + +class TestUserAgentIsNotMutated: + """Substitution must not change the object the caller handed us.""" + + def test_original_agent_keeps_its_providers(self) -> None: + original_provider = InMemoryHistoryProvider() + agent = Agent(client=_StubClient(), name="a", context_providers=[original_provider]) + original_list = agent.context_providers + + prepared = ensure_durable_history(agent) + + assert prepared is not agent + assert agent.context_providers is original_list + assert agent.context_providers == [original_provider] + + def test_entity_construction_does_not_mutate_the_agent(self) -> None: + agent = Agent(client=_StubClient(), name="a", context_providers=[InMemoryHistoryProvider()]) + + entity = AgentEntity(agent, state_provider=_InMemoryStateProvider()) + + assert isinstance(_history_providers(entity.agent)[0], DurableHistoryProvider) + assert isinstance(_history_providers(agent)[0], InMemoryHistoryProvider) diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py index c058d97..c74790f 100644 --- a/python/packages/durabletask/tests/test_durable_history_provider.py +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -324,12 +324,18 @@ async def test_service_managed_session_is_skipped(self) -> None: finally: unbind_durable_history(token) - async def test_without_durable_provider_legacy_replay_is_used(self) -> None: - """Agents without the provider keep the original full-replay behavior.""" + async def test_core_configured_agent_gets_durable_history_automatically(self) -> None: + """An agent configured the ordinary core way runs durably with no changes.""" client = RecordingChatClient() agent = Agent(client=client, name="assistant", context_providers=[InMemoryHistoryProvider()]) entity = _make_entity(agent, _InMemoryStateProvider()) await _run_turns(entity, ["first", "second"]) + # The entity swapped in durable-backed history without the user asking. + assert any(isinstance(p, DurableHistoryProvider) for p in entity.agent.context_providers) + # The caller's agent is untouched. + assert any(isinstance(p, InMemoryHistoryProvider) for p in agent.context_providers) + # History is served from durable state, so turn 2 sees turn 1. + assert len(client.received_messages[1]) > len(client.received_messages[0]) assert len(entity.state.data.conversation_history) == 4 From 406611daf0b692643b7df73f7df79dad5431bd51 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 13:51:31 -0500 Subject: [PATCH 05/32] fix: correct history ownership for external and service-managed agents; add prune_history --- .../0032-durable-thread-compaction.md | 41 ++++++- .../agent_framework_durabletask/_constants.py | 3 + .../_durable_agent_state.py | 9 ++ .../agent_framework_durabletask/_entities.py | 61 +++++++--- .../_history_provider.py | 16 ++- .../agent_framework_durabletask/_worker.py | 19 ++- .../tests/test_durable_history_autoswap.py | 111 ++++++++++++++++++ 7 files changed, 237 insertions(+), 23 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index d81143c..1d83c89 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -341,13 +341,42 @@ to the rest of the user's configuration. Because the injected provider is a `His `load_messages=True`, core's own auto-injection sees a provider present and stands down - no duplicate provider. -An explicit `DurableHistoryProvider` remains supported as an advanced escape hatch, for example to -enable `prune_excluded`. +An explicit `DurableHistoryProvider` remains supported as an advanced escape hatch, and takes +precedence over anything the runtime would inject. -**Side effect worth noting:** passing a session is what re-engages the context-provider pipeline, so -external history providers (Cosmos, Redis, file) now function under the durable runtime as well - -previously they were silently ignored because no session was ever created. Store-side compaction -still no-ops for those providers (core interface gap 1 below); only the in-run filter applies. +### When the entity manages history itself + +Two distinct decisions drive the entity, and conflating them caused bugs: + +1. **Who supplies conversation context?** If the agent exposes core's context-provider pipeline, + the providers do - so the entity passes a session and delivers **only the new messages**. This + holds whether history lives in durable state, an external store, or the model service. +2. **Should durable state be bound?** Only when a `DurableHistoryProvider` is present. + +The entity therefore replays its own persisted history in exactly one case: an agent that does not +expose the context pipeline at all (for example a fully custom agent). Routing external-store or +service-backed agents down that path was incorrect - it either bypassed their provider entirely or +re-sent history the service already had. + +**Consequence:** passing a session is what re-engages the pipeline, so external history providers +(Cosmos, Redis, file) now function under the durable runtime - previously they were silently +ignored because no session was ever created. Store-side compaction still no-ops for them (core +interface gap 1 below); only the in-run filter applies. + +### Service-managed conversations + +When the model service stores the conversation, it identifies the thread with an id. The entity +creates a fresh session per operation, so that id is **persisted in durable state and restored on +the next turn**; without it the service would start a new thread every turn. The durable history +provider additionally no-ops (neither loading nor flushing) for service-managed sessions. + +### Retention is a deployment policy, not agent configuration + +Compaction annotates; it does not delete. Physically deleting excluded messages bounds durable +storage but is **lossy**, so it is opt-in via `prune_history` at **registration** (app-level default +with a per-agent override) rather than on the agent. This keeps the agent definition portable - the +same agent runs in-memory, where a retention policy would be meaningless - and places the setting +next to its natural sibling, entity lifetime/TTL. ## More Information diff --git a/python/packages/durabletask/agent_framework_durabletask/_constants.py b/python/packages/durabletask/agent_framework_durabletask/_constants.py index e1542dc..03398a4 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_constants.py +++ b/python/packages/durabletask/agent_framework_durabletask/_constants.py @@ -134,6 +134,9 @@ class DurableStateFields: # Stable per-message identity (used for compaction reconciliation and idempotency) MESSAGE_ID: Final[str] = "messageId" + # Service-issued conversation id, for agents whose provider stores history server-side + SERVICE_SESSION_ID: Final[str] = "serviceSessionId" + class ContentTypes: """Content type discriminator values for the $type field. diff --git a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py index 321181b..cd19973 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py +++ b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py @@ -326,25 +326,31 @@ class DurableAgentStateData: Attributes: conversation_history: Ordered list of conversation entries (requests and responses) + service_session_id: Conversation id issued by a model service that stores history + server-side. Persisted so later turns continue the same service thread. extension_data: Optional dictionary for custom metadata (not part of core schema) """ conversation_history: list[DurableAgentStateEntry] + service_session_id: str | None extension_data: dict[str, Any] | None def __init__( self, conversation_history: list[DurableAgentStateEntry] | None = None, extension_data: dict[str, Any] | None = None, + service_session_id: str | None = None, ) -> None: """Initialize the data container. Args: conversation_history: Initial conversation history (defaults to empty list) extension_data: Optional custom metadata + service_session_id: Optional service-issued conversation id """ self.conversation_history = conversation_history or [] self.extension_data = extension_data + self.service_session_id = service_session_id def to_dict(self) -> dict[str, Any]: result: dict[str, Any] = { @@ -352,6 +358,8 @@ def to_dict(self) -> dict[str, Any]: } if self.extension_data is not None: result[DurableStateFields.EXTENSION_DATA] = self.extension_data + if self.service_session_id is not None: + result[DurableStateFields.SERVICE_SESSION_ID] = self.service_session_id return result @classmethod @@ -359,6 +367,7 @@ def from_dict(cls, data_dict: dict[str, Any]) -> DurableAgentStateData: return cls( conversation_history=_parse_history_entries(data_dict), extension_data=data_dict.get(DurableStateFields.EXTENSION_DATA), + service_session_id=data_dict.get(DurableStateFields.SERVICE_SESSION_ID), ) diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 0d4d65d..7db7efc 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -125,10 +125,11 @@ def __init__( callback: AgentResponseCallbackProtocol | None = None, *, state_provider: AgentEntityStateProviderMixin, + prune_history: bool = False, ) -> None: # Back the agent's conversation history with durable entity state so an agent that # already works in core runs durably without any configuration change. - self.agent = ensure_durable_history(agent) + self.agent = ensure_durable_history(agent, prune_history=prune_history) self.callback = callback self._state_provider = state_provider @@ -184,6 +185,7 @@ async def run( self.state.data.conversation_history.append(state_request) durable_history = self._find_durable_history_provider() + uses_context_pipeline = self._has_context_pipeline() binding_token = ( bind_durable_history( DurableHistoryBinding(state_provider=self._state_provider, correlation_id=correlation_id) @@ -193,11 +195,12 @@ async def run( ) try: - if durable_history is not None: - # Provider-backed path: the DurableHistoryProvider loads prior turns straight - # from durable entity state, so history lives in exactly one place and only the - # newly received request messages are passed as run input. Core context providers - # (history and compaction) therefore work unchanged on the durable runtime. + if uses_context_pipeline: + # The agent's own context providers supply prior turns - durable-backed history, + # an external store (Cosmos/Redis/file), or the model service itself. Only the + # newly received request messages are passed as run input, so history lives in + # exactly one place and core providers work unchanged on the durable runtime. + session = self._create_session() chat_messages = [ replayable_message for m in state_request.messages @@ -205,11 +208,13 @@ async def run( ] run_kwargs: dict[str, Any] = { "messages": chat_messages, - "session": self._create_session(), + "session": session, "options": options, } else: - # Legacy path: replay the full persisted conversation on every turn. + # Fallback for agents without the core context pipeline (for example a fully + # custom agent): the entity replays the persisted conversation on every turn. + session = None chat_messages = [ replayable_message for entry in self.state.data.conversation_history @@ -228,6 +233,7 @@ async def run( state_response = DurableAgentStateResponse.from_run_response(correlation_id, agent_run_response) self.state.data.conversation_history.append(state_response) + self._capture_service_session(session) self.persist_state() return agent_run_response @@ -254,6 +260,27 @@ async def run( if binding_token is not None: unbind_durable_history(binding_token) + def _has_context_pipeline(self) -> bool: + """Whether the agent exposes core's context-provider pipeline. + + When it does, the providers own conversation context and the entity delivers only the + new messages. Agents without it fall back to replaying persisted history. + """ + return isinstance(getattr(self.agent, "context_providers", None), (list, tuple)) + + def _capture_service_session(self, session: Any) -> None: + """Persist a service-issued conversation id so later turns continue the same thread. + + Service-backed agents keep the conversation on the service side and identify it with an + id. The entity creates a fresh session per operation, so without persisting this the + service would start a new thread on every turn. + """ + if session is None: + return + service_session_id = getattr(session, "service_session_id", None) + if isinstance(service_session_id, str) and service_session_id: + self.state.data.service_session_id = service_session_id + def _drop_already_stored(self, messages: list[DurableAgentStateMessage]) -> list[DurableAgentStateMessage]: """Filter out upstream context messages this entity has already recorded. @@ -287,18 +314,24 @@ def _find_durable_history_provider(self) -> DurableHistoryProvider | None: return None def _create_session(self) -> Any: - """Create a fresh session for a provider-backed run. + """Create the session for this operation. - No session state needs to persist: conversation history and any compaction - annotations live in durable entity state, loaded by the history provider. + Conversation history lives in the agent's context providers (durable entity state, an + external store, or the model service), so a fresh session per operation is enough. Any + previously issued service conversation id is restored so service-backed agents continue + the same thread. """ create_session = getattr(self.agent, "create_session", None) if not callable(create_session): raise TypeError( - f"Agent {type(self.agent).__name__} is configured with a DurableHistoryProvider " - "but does not support create_session()." + f"Agent {type(self.agent).__name__} exposes context providers but does not support create_session()." ) - return create_session() + session: Any = create_session() + + service_session_id = self.state.data.service_session_id + if service_session_id and getattr(session, "service_session_id", None) is None: + session.service_session_id = service_session_id + return session @staticmethod def _to_replayable_message(message: DurableAgentStateMessage) -> Message | None: diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py index 9ec996c..56ddff3 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -322,7 +322,7 @@ def _service_stores_history(agent: Any) -> bool: return bool(getattr(client, "STORES_BY_DEFAULT", False)) -def ensure_durable_history(agent: SupportsAgentRun) -> SupportsAgentRun: +def ensure_durable_history(agent: SupportsAgentRun, *, prune_history: bool = False) -> SupportsAgentRun: """Back an agent's conversation history with durable entity state. Lets a user register an agent that already works in core and get durable behavior with no @@ -345,6 +345,11 @@ def ensure_durable_history(agent: SupportsAgentRun) -> SupportsAgentRun: Args: agent: The agent being registered with the durable runtime. + Keyword Args: + prune_history: When True, the injected provider physically deletes messages that + compaction excluded, bounding durable storage. This is a **lossy retention policy** + and is off by default. It only affects providers this function creates. + Returns: The agent to run, either unchanged or a shallow copy with durable-backed history. """ @@ -368,11 +373,18 @@ def ensure_durable_history(agent: SupportsAgentRun) -> SupportsAgentRun: if existing is None: # Match the source_id core's auto-injected provider would use so default-wired # compaction keeps resolving. - updated = [DurableHistoryProvider(source_id=InMemoryHistoryProvider.DEFAULT_SOURCE_ID), *provider_list] + updated = [ + DurableHistoryProvider( + source_id=InMemoryHistoryProvider.DEFAULT_SOURCE_ID, + prune_excluded=prune_history, + ), + *provider_list, + ] elif isinstance(existing, InMemoryHistoryProvider): replacement = DurableHistoryProvider( source_id=existing.source_id, skip_excluded=existing.skip_excluded, + prune_excluded=prune_history, ) updated = [replacement if p is existing else p for p in provider_list] else: diff --git a/python/packages/durabletask/agent_framework_durabletask/_worker.py b/python/packages/durabletask/agent_framework_durabletask/_worker.py index 3eed81f..63f4fe8 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_worker.py +++ b/python/packages/durabletask/agent_framework_durabletask/_worker.py @@ -78,15 +78,21 @@ def __init__( self, worker: TaskHubGrpcWorker, callback: AgentResponseCallbackProtocol | None = None, + *, + prune_history: bool = False, ): """Initialize the worker wrapper. Args: worker: The durabletask worker instance to wrap callback: Optional callback for agent response notifications + prune_history: Default retention policy for registered agents. When True, messages + that compaction excluded are physically deleted from durable state, bounding + stored size. This is lossy and off by default. """ self._worker = worker self._callback = callback + self._prune_history = prune_history self._registered_agents: dict[str, SupportsAgentRun] = {} self._workflows: dict[str, Workflow] = {} # Every workflow whose orchestration has been registered (top-level plus nested @@ -102,6 +108,7 @@ def add_agent( callback: AgentResponseCallbackProtocol | None = None, *, entity_id: str | None = None, + prune_history: bool | None = None, ) -> None: """Register an agent with the worker. @@ -115,6 +122,8 @@ def add_agent( entity_id: Optional identity to register the entity under instead of ``agent.name``. Workflow hosting passes the executor's ``id`` so the entity matches the identity the orchestrator dispatches to. + prune_history: Per-agent retention override. When None, the worker-level + ``prune_history`` setting is used. Raises: ValueError: If the agent doesn't have a name or is already registered @@ -137,7 +146,12 @@ def add_agent( effective_callback = callback or self._callback # Create a configured entity class using the factory - entity_class = self.__create_agent_entity(agent, effective_callback, entity_id=registration_name) + entity_class = self.__create_agent_entity( + agent, + effective_callback, + entity_id=registration_name, + prune_history=(self._prune_history if prune_history is None else prune_history), + ) # Register the entity class with the worker # The worker.add_entity method takes a class @@ -356,6 +370,7 @@ def __create_agent_entity( callback: AgentResponseCallbackProtocol | None = None, *, entity_id: str | None = None, + prune_history: bool = False, ) -> type[DurableTaskEntityStateProvider]: """Factory function to create a DurableEntity class configured with an agent. @@ -368,6 +383,7 @@ def __create_agent_entity( entity_id: Optional identity to register the entity under instead of ``agent.name`` (used by workflow hosting to key entities by executor id). + prune_history: Whether excluded messages are physically deleted from durable state. Returns: A new DurableEntity subclass configured for this agent @@ -385,6 +401,7 @@ def __init__(self) -> None: agent=agent, callback=callback, state_provider=self, + prune_history=prune_history, ) logger.debug( "[ConfiguredAgentEntity] Initialized entity for agent: %s (entity name: %s)", diff --git a/python/packages/durabletask/tests/test_durable_history_autoswap.py b/python/packages/durabletask/tests/test_durable_history_autoswap.py index b1a9d86..6b3cd47 100644 --- a/python/packages/durabletask/tests/test_durable_history_autoswap.py +++ b/python/packages/durabletask/tests/test_durable_history_autoswap.py @@ -161,3 +161,114 @@ def test_entity_construction_does_not_mutate_the_agent(self) -> None: assert isinstance(_history_providers(entity.agent)[0], DurableHistoryProvider) assert isinstance(_history_providers(agent)[0], InMemoryHistoryProvider) + + +class TestPruneHistoryOptIn: + """Pruning is a deployment-level retention policy, set at registration.""" + + def test_off_by_default(self) -> None: + agent = Agent(client=_StubClient(), name="a") + + prepared = ensure_durable_history(agent) + + assert _history_providers(prepared)[0].prune_excluded is False + + def test_enabled_via_registration(self) -> None: + agent = Agent(client=_StubClient(), name="a", context_providers=[InMemoryHistoryProvider()]) + + prepared = ensure_durable_history(agent, prune_history=True) + + assert _history_providers(prepared)[0].prune_excluded is True + + def test_entity_forwards_the_flag(self) -> None: + agent = Agent(client=_StubClient(), name="a") + + entity = AgentEntity(agent, state_provider=_InMemoryStateProvider(), prune_history=True) + + assert _history_providers(entity.agent)[0].prune_excluded is True + + def test_explicit_provider_configuration_wins(self) -> None: + """A hand-configured provider is never overridden by the registration flag.""" + explicit = DurableHistoryProvider(prune_excluded=False) + agent = Agent(client=_StubClient(), name="a", context_providers=[explicit]) + + prepared = ensure_durable_history(agent, prune_history=True) + + assert _history_providers(prepared)[0] is explicit + assert explicit.prune_excluded is False + + +class TestServiceManagedSessions: + """Service-backed agents let the service own the conversation.""" + + async def test_only_new_messages_are_sent(self) -> None: + """History must not be replayed locally when the service already holds it.""" + recorded: list[list[Message]] = [] + + class _ServiceAgent: + name = "svc" + client = _ServiceStoringClient() + context_providers: list[Any] = [] + + def create_session(self, **kwargs: Any) -> Any: + from agent_framework import AgentSession + + return AgentSession() + + async def run(self, messages: Any = None, *, stream: bool = False, **kwargs: Any) -> Any: + from agent_framework import AgentResponse + + if stream: + raise TypeError("stream is not supported") + recorded.append(list(messages or [])) + return AgentResponse(messages=[Message(role="assistant", contents=["ok"])]) + + entity = AgentEntity(_ServiceAgent(), state_provider=_InMemoryStateProvider()) # type: ignore[arg-type] + + await entity.run({"message": "first", "correlationId": "c0"}) + await entity.run({"message": "second", "correlationId": "c1"}) + + # Each turn delivers only its own message; the service supplies the rest. + assert len(recorded[1]) == 1 + assert recorded[1][0].text == "second" + + async def test_service_conversation_id_is_persisted_and_restored(self) -> None: + """Without this the service would start a new thread on every turn.""" + seen_ids: list[str | None] = [] + + class _ThreadingAgent: + name = "svc" + client = _ServiceStoringClient() + context_providers: list[Any] = [] + + def create_session(self, **kwargs: Any) -> Any: + from agent_framework import AgentSession + + return AgentSession() + + async def run( + self, + messages: Any = None, + *, + stream: bool = False, + session: Any = None, + **kwargs: Any, + ) -> Any: + from agent_framework import AgentResponse + + if stream: + raise TypeError("stream is not supported") + seen_ids.append(getattr(session, "service_session_id", None)) + # The service issues (or confirms) the thread id on the session. + session.service_session_id = "svc-thread-1" + return AgentResponse(messages=[Message(role="assistant", contents=["ok"])]) + + provider = _InMemoryStateProvider() + entity = AgentEntity(_ThreadingAgent(), state_provider=provider) # type: ignore[arg-type] + + await entity.run({"message": "first", "correlationId": "c0"}) + await entity.run({"message": "second", "correlationId": "c1"}) + + assert seen_ids[0] is None # first turn has no thread yet + assert seen_ids[1] == "svc-thread-1" # second turn continues the same thread + assert provider._get_state_dict()["data"]["serviceSessionId"] == "svc-thread-1" From 4639bd4f4fc035d26fa1bab2204e0b90b7458652 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 14:07:34 -0500 Subject: [PATCH 06/32] feat: extend prune_history and workflow context forwarding to the Functions host --- .../agent_framework_azurefunctions/_app.py | 27 ++++++++++++++++--- .../_entities.py | 8 +++++- .../_orchestration.py | 6 ++++- .../_workflow_af_context.py | 10 +++++-- .../packages/azurefunctions/tests/test_app.py | 4 +-- 5 files changed, 46 insertions(+), 9 deletions(-) diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py index dc90d13..c7c3723 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py @@ -244,6 +244,7 @@ def __init__( poll_interval_seconds: float = DEFAULT_POLL_INTERVAL_SECONDS, enable_mcp_tool_trigger: bool = False, default_callback: AgentResponseCallbackProtocol | None = None, + prune_history: bool = False, ): """Initialize the AgentFunctionApp. @@ -263,6 +264,10 @@ def __init__( :param poll_interval_seconds: Delay in seconds between polling attempts. Defaults to ``DEFAULT_POLL_INTERVAL_SECONDS``. :param default_callback: Optional callback invoked for agents without specific callbacks. + :param prune_history: Default conversation-retention policy for agents hosted by this app + (including agents inside hosted workflows). When True, messages that compaction + excluded are physically deleted from durable state, bounding stored size. This is + lossy and off by default; ``add_agent`` can override it per agent. :note: If no agents are provided, they can be added later using :meth:`add_agent`. """ @@ -283,6 +288,7 @@ def __init__( self.enable_http_endpoints = enable_http_endpoints self.enable_mcp_tool_trigger = enable_mcp_tool_trigger self.default_callback = default_callback + self._prune_history = prune_history try: retries = int(max_poll_retries) @@ -826,6 +832,7 @@ def add_agent( enable_mcp_tool_trigger: bool | None = None, *, entity_id: str | None = None, + prune_history: bool | None = None, ) -> None: """Add an agent to the function app after initialization. @@ -842,6 +849,8 @@ def add_agent( durable entity (and the ``agents`` / ``get_agent`` key) matches the identity the orchestrator dispatches to. Mirrors ``DurableAIAgentWorker.add_agent(entity_id=...)``. + prune_history: Per-agent conversation-retention override. When None, the app-level + ``prune_history`` setting is used. Raises: ValueError: If the agent doesn't have a 'name' attribute. @@ -890,9 +899,15 @@ def add_agent( ) effective_callback = callback or self.default_callback + effective_prune_history = self._prune_history if prune_history is None else prune_history self._setup_agent_functions( - agent, registration_name, effective_callback, effective_enable_http_endpoint, effective_enable_mcp_endpoint + agent, + registration_name, + effective_callback, + effective_enable_http_endpoint, + effective_enable_mcp_endpoint, + prune_history=effective_prune_history, ) logger.debug(f"[AgentFunctionApp] Agent '{registration_name}' added successfully") @@ -937,6 +952,8 @@ def _setup_agent_functions( callback: AgentResponseCallbackProtocol | None, enable_http_endpoint: bool, enable_mcp_tool_trigger: bool, + *, + prune_history: bool = False, ) -> None: """Set up the HTTP trigger, entity, and MCP tool trigger for a specific agent. @@ -946,6 +963,7 @@ def _setup_agent_functions( callback: Optional callback to receive response updates enable_http_endpoint: Whether to create HTTP endpoint enable_mcp_tool_trigger: Whether to create MCP tool trigger + prune_history: Whether excluded messages are deleted from durable state. """ logger.debug(f"[AgentFunctionApp] Setting up functions for agent '{agent_name}'...") @@ -956,7 +974,7 @@ def _setup_agent_functions( "[AgentFunctionApp] HTTP run route disabled for agent '%s'", agent_name, ) - self._setup_agent_entity(agent, agent_name, callback) + self._setup_agent_entity(agent, agent_name, callback, prune_history=prune_history) if enable_mcp_tool_trigger: agent_description = agent.description @@ -1098,6 +1116,8 @@ def _setup_agent_entity( agent: SupportsAgentRun, agent_name: str, callback: AgentResponseCallbackProtocol | None, + *, + prune_history: bool = False, ) -> None: """Register the durable entity responsible for agent state. @@ -1105,6 +1125,7 @@ def _setup_agent_entity( agent: The agent instance agent_name: The agent name (used for both entity identification and function naming) callback: Optional callback for response updates + prune_history: Whether excluded messages are deleted from durable state. """ # Use the prefixed entity name for both registration and function naming entity_name_with_prefix = AgentSessionId.to_entity_name(agent_name) @@ -1117,7 +1138,7 @@ def entity_function(context: df.DurableEntityContext) -> None: - run_agent: (Deprecated) Execute the agent with a message - reset: Clear conversation history """ - entity_handler = create_agent_entity(agent, callback) + entity_handler = create_agent_entity(agent, callback, prune_history=prune_history) entity_handler(context) # Set function name for Azure Functions (used in function.json generation) diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py index 83ad50a..c69697e 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py @@ -51,6 +51,8 @@ def _get_session_id_from_entity(self) -> str: def create_agent_entity( agent: SupportsAgentRun, callback: AgentResponseCallbackProtocol | None = None, + *, + prune_history: bool = False, ) -> Callable[[df.DurableEntityContext], None]: """Factory function to create an agent entity class. @@ -58,6 +60,10 @@ def create_agent_entity( agent: The Microsoft Agent Framework agent instance (must implement SupportsAgentRun) callback: Optional callback invoked during streaming and final responses + Keyword Args: + prune_history: When True, messages that compaction excluded are physically deleted + from durable state. Lossy retention policy; off by default. + Returns: Entity function configured with the agent """ @@ -69,7 +75,7 @@ async def _entity_coroutine(context: df.DurableEntityContext) -> None: logger.debug("[entity_function] Operation: %s", context.operation_name) state_provider = AzureFunctionEntityStateProvider(context) - entity = AgentEntity(agent, callback, state_provider=state_provider) + entity = AgentEntity(agent, callback, state_provider=state_provider, prune_history=prune_history) operation = context.operation_name diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_orchestration.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_orchestration.py index cbbd134..be4df10 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_orchestration.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_orchestration.py @@ -154,6 +154,7 @@ def get_run_request( message: str, *, options: dict[str, Any] | None = None, + context_messages: list[dict[str, Any]] | None = None, ) -> RunRequest: """Get the current run request from the orchestration context. @@ -162,13 +163,16 @@ def get_run_request( options: Optional options dictionary. Supported keys include ``response_format``, ``enable_tool_calls``, and ``wait_for_response``. Additional keys are forwarded to the agent execution. + context_messages: Optional upstream conversation (serialized ``Message`` dicts) + delivered to the agent as prior context. Workflows use this to give a + downstream agent the conversation produced by upstream nodes. Returns: RunRequest: The current run request """ # Create a copy to avoid modifying the caller's dict - request = super().get_run_request(message, options=options) + request = super().get_run_request(message, options=options, context_messages=context_messages) request.orchestration_id = self.context.instance_id return request diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_workflow_af_context.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_workflow_af_context.py index eaf99a5..9da8a6c 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_workflow_af_context.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_workflow_af_context.py @@ -56,12 +56,18 @@ def current_utc_datetime(self) -> datetime: # -- Agent / Activity dispatch -------------------------------------------- - def prepare_agent_task(self, executor_id: str, message: str, orchestration_instance_id: str) -> Any: + def prepare_agent_task( + self, + executor_id: str, + message: str, + orchestration_instance_id: str, + context_messages: list[dict[str, Any]] | None = None, + ) -> Any: session_id = AgentSessionId(name=executor_id, key=orchestration_instance_id) session = DurableAgentSession(durable_session_id=session_id) az_executor = AzureFunctionsAgentExecutor(self._context) agent = DurableAIAgent(az_executor, executor_id) - return agent.run(message, session=session) + return agent.run(message, session=session, context_messages=context_messages) def prepare_activity_task(self, activity_name: str, input_json: str) -> Any: orchestration_context: Any = self._context diff --git a/python/packages/azurefunctions/tests/test_app.py b/python/packages/azurefunctions/tests/test_app.py index 15fff90..185b2c2 100644 --- a/python/packages/azurefunctions/tests/test_app.py +++ b/python/packages/azurefunctions/tests/test_app.py @@ -269,7 +269,7 @@ def test_agent_override_enables_http_route_when_app_disabled(self) -> None: app.add_agent(mock_agent, enable_http_endpoint=True) http_route_mock.assert_called_once_with("OverrideAgent") - agent_entity_mock.assert_called_once_with(mock_agent, "OverrideAgent", ANY) + agent_entity_mock.assert_called_once_with(mock_agent, "OverrideAgent", ANY, prune_history=False) assert app._agent_metadata["OverrideAgent"].http_endpoint_enabled is True def test_agent_override_disables_http_route_when_app_enabled(self) -> None: @@ -286,7 +286,7 @@ def test_agent_override_disables_http_route_when_app_enabled(self) -> None: app.add_agent(mock_agent, enable_http_endpoint=False) http_route_mock.assert_not_called() - agent_entity_mock.assert_called_once_with(mock_agent, "DisabledOverride", ANY) + agent_entity_mock.assert_called_once_with(mock_agent, "DisabledOverride", ANY, prune_history=False) assert app._agent_metadata["DisabledOverride"].http_endpoint_enabled is False def test_multiple_apps_independent(self) -> None: From 62b6ce3ee355f37d6db2c65dc64eb536337f8ab5 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 14:20:04 -0500 Subject: [PATCH 07/32] refactor: remove duplicated host-adapter logic and enforce the context protocol on both hosts --- .../_orchestration.py | 28 ++-------------- .../_workflow_af_context.py | 19 +++++++---- .../agent_framework_durabletask/__init__.py | 3 +- .../agent_framework_durabletask/_executors.py | 30 +++++++---------- .../agent_framework_durabletask/_shim.py | 32 ++++++++++++++++++- .../_workflows/dt_context.py | 14 ++++---- 6 files changed, 67 insertions(+), 59 deletions(-) diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_orchestration.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_orchestration.py index be4df10..98fa06e 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_orchestration.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_orchestration.py @@ -149,32 +149,8 @@ def __init__(self, context: AgentOrchestrationContextType): def generate_unique_id(self) -> str: return str(self.context.new_uuid()) - def get_run_request( - self, - message: str, - *, - options: dict[str, Any] | None = None, - context_messages: list[dict[str, Any]] | None = None, - ) -> RunRequest: - """Get the current run request from the orchestration context. - - Args: - message: The message to send to the agent - options: Optional options dictionary. Supported keys include - ``response_format``, ``enable_tool_calls``, and ``wait_for_response``. - Additional keys are forwarded to the agent execution. - context_messages: Optional upstream conversation (serialized ``Message`` dicts) - delivered to the agent as prior context. Workflows use this to give a - downstream agent the conversation produced by upstream nodes. - - Returns: - RunRequest: The current run request - """ - # Create a copy to avoid modifying the caller's dict - - request = super().get_run_request(message, options=options, context_messages=context_messages) - request.orchestration_id = self.context.instance_id - return request + def _orchestration_id(self) -> str | None: + return self.context.instance_id def run_durable_agent( self, diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_workflow_af_context.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_workflow_af_context.py index 9da8a6c..96fe027 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_workflow_af_context.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_workflow_af_context.py @@ -12,7 +12,7 @@ from datetime import datetime from typing import Any -from agent_framework_durabletask import AgentSessionId, DurableAgentSession, DurableAIAgent +from agent_framework_durabletask import WorkflowOrchestrationContext, build_agent_task from azure.durable_functions import DurableOrchestrationContext from ._orchestration import AzureFunctionsAgentExecutor @@ -63,11 +63,13 @@ def prepare_agent_task( orchestration_instance_id: str, context_messages: list[dict[str, Any]] | None = None, ) -> Any: - session_id = AgentSessionId(name=executor_id, key=orchestration_instance_id) - session = DurableAgentSession(durable_session_id=session_id) - az_executor = AzureFunctionsAgentExecutor(self._context) - agent = DurableAIAgent(az_executor, executor_id) - return agent.run(message, session=session, context_messages=context_messages) + return build_agent_task( + AzureFunctionsAgentExecutor(self._context), + executor_id, + message, + orchestration_instance_id, + context_messages, + ) def prepare_activity_task(self, activity_name: str, input_json: str) -> Any: orchestration_context: Any = self._context @@ -109,3 +111,8 @@ def cancel_task(self, task: Any) -> None: def get_task_result(self, task: Any) -> Any: return getattr(task, "result", None) + + +# Ensure the adapter satisfies the protocol. Validated statically by the type checker, +# so a signature change on the protocol is caught here rather than at a distant call site. +_protocol_check: type[WorkflowOrchestrationContext] = AzureFunctionsWorkflowContext diff --git a/python/packages/durabletask/agent_framework_durabletask/__init__.py b/python/packages/durabletask/agent_framework_durabletask/__init__.py index fecc925..7e91d30 100644 --- a/python/packages/durabletask/agent_framework_durabletask/__init__.py +++ b/python/packages/durabletask/agent_framework_durabletask/__init__.py @@ -54,7 +54,7 @@ from ._models import AgentSessionId, DurableAgentSession, RunRequest from ._orchestration_context import DurableAIAgentOrchestrationContext from ._response_utils import ensure_response_format, load_agent_response -from ._shim import DurableAIAgent +from ._shim import DurableAIAgent, build_agent_task from ._worker import DurableAIAgentWorker from ._workflows.activity import execute_workflow_activity from ._workflows.client import DurableWorkflowClient @@ -169,6 +169,7 @@ def __dir__() -> list[str]: "WorkflowOrchestrationContext", "WorkflowRegistrationPlan", "__version__", + "build_agent_task", "collect_hosted_workflows", "deserialize_workflow_output", "ensure_response_format", diff --git a/python/packages/durabletask/agent_framework_durabletask/_executors.py b/python/packages/durabletask/agent_framework_durabletask/_executors.py index 1b97b08..90f3b54 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_executors.py +++ b/python/packages/durabletask/agent_framework_durabletask/_executors.py @@ -155,6 +155,14 @@ def generate_unique_id(self) -> str: """Generate a new Unique ID.""" return uuid.uuid4().hex + def _orchestration_id(self) -> str | None: + """Return the orchestration instance that issued this request. + + Overridden by executors that run inside an orchestration. Client-side executors + have no orchestration, so the default is ``None``. + """ + return None + def get_run_request( self, message: str, @@ -181,6 +189,7 @@ def get_run_request( correlation_id=correlation_id, options=opts, context_messages=context_messages, + orchestration_id=self._orchestration_id(), ) def _create_acceptance_response(self, correlation_id: str) -> AgentResponse: @@ -451,25 +460,8 @@ def generate_unique_id(self) -> str: """Create a new UUID that is safe for replay within an orchestration or operation.""" return self._context.new_uuid() - def get_run_request( - self, - message: str, - *, - options: dict[str, Any] | None = None, - context_messages: list[dict[str, Any]] | None = None, - ) -> RunRequest: - """Get the current run request from the orchestration context. - - Returns: - RunRequest: The current run request - """ - request = super().get_run_request( - message, - options=options, - context_messages=context_messages, - ) - request.orchestration_id = self._context.instance_id - return request + def _orchestration_id(self) -> str | None: + return self._context.instance_id def run_durable_agent( self, diff --git a/python/packages/durabletask/agent_framework_durabletask/_shim.py b/python/packages/durabletask/agent_framework_durabletask/_shim.py index 6340033..084163c 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_shim.py +++ b/python/packages/durabletask/agent_framework_durabletask/_shim.py @@ -16,13 +16,43 @@ from agent_framework._types import AgentRunInputs from ._executors import DurableAgentExecutor -from ._models import DurableAgentSession +from ._models import AgentSessionId, DurableAgentSession # TypeVar for the task type returned by executors # Covariant because TaskT only appears in return positions (output) TaskT = TypeVar("TaskT", covariant=True) +def build_agent_task( + executor: DurableAgentExecutor[Any], + executor_id: str, + message: str, + orchestration_instance_id: str, + context_messages: list[dict[str, Any]] | None = None, +) -> Any: + """Create the yieldable task that runs a workflow's agent node. + + Shared by every host adapter: the only host-specific part of dispatching an agent is + which :class:`DurableAgentExecutor` drives it, so the surrounding session/agent wiring + lives here rather than being repeated per host. + + Args: + executor: The host's executor, which knows how to reach the agent entity. + executor_id: The workflow-scoped agent identity to dispatch to. + message: The text message for this turn. + orchestration_instance_id: Used as the entity session key, keeping conversation + state isolated per workflow run. + context_messages: Optional upstream conversation delivered as prior context. + + Returns: + A yieldable task whose result is an ``AgentResponse``. + """ + session_id = AgentSessionId(name=executor_id, key=orchestration_instance_id) + session = DurableAgentSession(durable_session_id=session_id) + agent = DurableAIAgent(executor, executor_id) + return agent.run(message, session=session, context_messages=context_messages) + + class DurableAgentProvider(ABC, Generic[TaskT]): """Abstract provider for constructing durable agent proxies. diff --git a/python/packages/durabletask/agent_framework_durabletask/_workflows/dt_context.py b/python/packages/durabletask/agent_framework_durabletask/_workflows/dt_context.py index 4892b31..5ed23d0 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_workflows/dt_context.py +++ b/python/packages/durabletask/agent_framework_durabletask/_workflows/dt_context.py @@ -20,8 +20,7 @@ ) from .._executors import OrchestrationAgentExecutor -from .._models import AgentSessionId, DurableAgentSession -from .._shim import DurableAIAgent +from .._shim import build_agent_task from .context import WorkflowOrchestrationContext logger = logging.getLogger(__name__) @@ -64,10 +63,13 @@ def prepare_agent_task( orchestration_instance_id: str, context_messages: list[dict[str, Any]] | None = None, ) -> Any: - session_id = AgentSessionId(name=executor_id, key=orchestration_instance_id) - session = DurableAgentSession(durable_session_id=session_id) - agent = DurableAIAgent(self._executor, executor_id) - return agent.run(message, session=session, context_messages=context_messages) + return build_agent_task( + self._executor, + executor_id, + message, + orchestration_instance_id, + context_messages, + ) def prepare_activity_task(self, activity_name: str, input_json: str) -> Any: return cast(Any, self._context.call_activity(activity_name, input=input_json)) From c3bf4017c53bbdf822c00b9e94c12a4afed38974 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 19:44:05 -0500 Subject: [PATCH 08/32] chore: ignore local .env files The '!python/packages/**' negation earlier in the file un-ignored everything beneath it, so the integration test .env files holding endpoints and credentials were staged by a plain 'git add'. A trailing '**/.env' rule wins over that negation; .env.example templates stay tracked. --- .gitignore | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.gitignore b/.gitignore index a5f1ea2..a95ea58 100644 --- a/.gitignore +++ b/.gitignore @@ -445,3 +445,7 @@ FodyWeavers.xsd *.msix *.msm *.msp + +# Local environment files with credentials (templates use .env.example and stay tracked). +# Must come after the '!python/packages/**' negation above so it wins for test .env files. +**/.env From 72fab1c8af0f3f900a682ac73eeccf8b27999edb Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 19:44:14 -0500 Subject: [PATCH 09/32] fix: honor explicit store option and give history providers a stable session id Two ways an agent that works in core could silently lose its conversation under the durable runtime, both failing without an error: - Ownership of history was decided from the chat client's STORES_BY_DEFAULT alone. Core's rule is that an explicit 'store' in the agent's options wins, so an agent using the Responses API with store=False kept a plain in-memory provider that the durable runtime never persists. - The entity built its per-operation session without an id, so core generated a fresh one each turn. External history providers (Cosmos, Redis, file) key their storage on session.session_id and were therefore reading and writing a different key on every turn. --- .../agent_framework_durabletask/_entities.py | 11 +++-- .../_history_provider.py | 15 ++++++- .../tests/test_durable_history_autoswap.py | 23 ++++++++++ .../tests/test_durable_history_provider.py | 42 +++++++++++++++++++ 4 files changed, 85 insertions(+), 6 deletions(-) diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 7db7efc..9d8f421 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -317,16 +317,19 @@ def _create_session(self) -> Any: """Create the session for this operation. Conversation history lives in the agent's context providers (durable entity state, an - external store, or the model service), so a fresh session per operation is enough. Any - previously issued service conversation id is restored so service-backed agents continue - the same thread. + external store, or the model service), so a fresh session per operation is enough - but it + must carry the entity's **stable** session id. External history providers (Cosmos, Redis, + file) key their storage on ``session.session_id``; with a freshly generated id they would + read and write a different key every turn and never see prior history. Any previously + issued service conversation id is restored so service-backed agents continue the same + thread. """ create_session = getattr(self.agent, "create_session", None) if not callable(create_session): raise TypeError( f"Agent {type(self.agent).__name__} exposes context providers but does not support create_session()." ) - session: Any = create_session() + session: Any = create_session(session_id=self._state_provider.session_id) service_session_id = self.state.data.service_session_id if service_session_id and getattr(session, "service_session_id", None) is None: diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py index 56ddff3..5abc79e 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -15,7 +15,7 @@ import copy import logging -from collections.abc import Iterator, Sequence +from collections.abc import Iterator, Mapping, Sequence from contextvars import ContextVar, Token from dataclasses import dataclass from typing import TYPE_CHECKING, Any, cast @@ -317,7 +317,18 @@ def _prune(binding: DurableHistoryBinding, pruned: list[tuple[DurableAgentStateE def _service_stores_history(agent: Any) -> bool: - """Return whether the agent's client keeps conversation history server-side.""" + """Return whether the service keeps conversation history for this agent. + + Mirrors core's precedence: an explicit ``store`` in the agent's default options wins, and only + when it is unset does the client's ``STORES_BY_DEFAULT`` apply. Clients that store by default + (such as the Responses API) can therefore be put back in client-side mode with ``store=False``, + in which case durable history is what makes the conversation survive. + """ + default_options = getattr(agent, "default_options", None) + if isinstance(default_options, Mapping): + explicit_store = cast("Mapping[str, Any]", default_options).get("store") + if explicit_store is not None: + return bool(explicit_store) client = getattr(agent, "client", None) return bool(getattr(client, "STORES_BY_DEFAULT", False)) diff --git a/python/packages/durabletask/tests/test_durable_history_autoswap.py b/python/packages/durabletask/tests/test_durable_history_autoswap.py index 6b3cd47..21dffd2 100644 --- a/python/packages/durabletask/tests/test_durable_history_autoswap.py +++ b/python/packages/durabletask/tests/test_durable_history_autoswap.py @@ -117,6 +117,29 @@ def test_service_managed_history_is_left_alone(self) -> None: assert prepared is agent assert not _history_providers(prepared) + def test_store_false_overrides_a_service_storing_client(self) -> None: + """``store=False`` puts history back in the client's hands, so durable must back it. + + Mirrors core's precedence: an explicit ``store`` wins over ``STORES_BY_DEFAULT``. Without + this, an agent using the Responses API with ``store=False`` would keep a plain in-memory + provider that the durable runtime never persists, silently losing the conversation. + """ + agent = Agent(client=_ServiceStoringClient(), name="a", default_options={"store": False}) + + prepared = ensure_durable_history(agent) + + providers = _history_providers(prepared) + assert len(providers) == 1 + assert isinstance(providers[0], DurableHistoryProvider) + + def test_store_true_keeps_history_with_the_service(self) -> None: + agent = Agent(client=_StubClient(), name="a", default_options={"store": True}) + + prepared = ensure_durable_history(agent) + + assert prepared is agent + assert not _history_providers(prepared) + def test_existing_durable_provider_is_untouched(self) -> None: """Explicit configuration (for example to enable pruning) wins.""" explicit = DurableHistoryProvider(prune_excluded=True) diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py index c74790f..1845b02 100644 --- a/python/packages/durabletask/tests/test_durable_history_provider.py +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -16,6 +16,7 @@ ChatResponseUpdate, CompactionProvider, Content, + HistoryProvider, InMemoryHistoryProvider, Message, ResponseStream, @@ -339,3 +340,44 @@ async def test_core_configured_agent_gets_durable_history_automatically(self) -> # History is served from durable state, so turn 2 sees turn 1. assert len(client.received_messages[1]) > len(client.received_messages[0]) assert len(entity.state.data.conversation_history) == 4 + + +class TestExternalHistoryProviders: + """Providers that own their own storage (Cosmos, Redis, file) keep working durably.""" + + async def test_external_provider_receives_the_entity_session_id(self) -> None: + """Their storage is keyed by session id, so it must be the entity's stable id. + + The entity builds a fresh session per operation. If that session carried a generated id, + an external provider would read and write a different key every turn and never see prior + history - broken continuity with no error to show for it. + """ + seen: list[str | None] = [] + + class _RecordingExternalProvider(HistoryProvider): + def __init__(self) -> None: + super().__init__(source_id="external") + + async def get_messages(self, session_id: str | None, **kwargs: Any) -> list[Message]: + seen.append(session_id) + return [] + + async def save_messages(self, session_id: str | None, messages: Any, **kwargs: Any) -> None: + seen.append(session_id) + + agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[_RecordingExternalProvider()]) + entity = _make_entity(agent, _InMemoryStateProvider(session_id="stable-session")) + + await _run_turns(entity, ["first", "second"]) + + assert seen, "the external provider should have taken part in the run" + assert set(seen) == {"stable-session"} + + async def test_external_provider_is_not_replaced(self) -> None: + """The user chose their own storage; durable must not swap it out.""" + external = HistoryProvider(source_id="external") + agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[external]) + + entity = _make_entity(agent, _InMemoryStateProvider()) + + assert entity.agent.context_providers[0] is external From af5798aae4966703e7ffcaac1ecad619d03121a1 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 19:44:22 -0500 Subject: [PATCH 10/32] docs: record history-ownership rules and entity lifetime concerns in ADR 0032 Documents the two rules the fixes above depend on (store precedence over STORES_BY_DEFAULT, and stable session ids for external providers), and restores the entity lifetime/TTL section. TTL is the natural sibling of the retention setting this ADR introduces - the retention rationale already refers to it - and the .NET/Python parity gap it describes belongs in this repository. --- .../0032-durable-thread-compaction.md | 41 ++++++++++++++++++- 1 file changed, 40 insertions(+), 1 deletion(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index 1d83c89..b801948 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -332,7 +332,7 @@ used, so the caller's agent still behaves normally in-process. | Nothing | Inject a durable history provider, using the `source_id` core's auto-injected provider would have - so default-wired compaction still resolves. No compaction by default (same as core). | | `InMemoryHistoryProvider` (± compaction) | Replace with the durable provider, **preserving `source_id` and `skip_excluded`** so any attached `CompactionProvider` keeps working untouched. | | Cosmos / Redis / file / custom provider | **Leave alone.** The user chose where their conversation lives; durable still supplies execution durability. | -| Service-managed history | **Leave alone.** The model service owns the conversation. | +| Service-managed history | **Leave alone.** The model service owns the conversation. Decided by core's precedence: explicit `store` first, then the client's `STORES_BY_DEFAULT`. | | Agent without the core context pipeline | **Leave alone.** Falls back to replaying persisted history. | Preserving `source_id` is the load-bearing detail: `CompactionProvider` locates history through @@ -363,6 +363,11 @@ re-sent history the service already had. ignored because no session was ever created. Store-side compaction still no-ops for them (core interface gap 1 below); only the in-run filter applies. +That session must also carry the entity's **stable** session id rather than a generated one. +External providers key their storage on `session.session_id`, so a per-operation id would make them +read and write a different key every turn - the conversation would silently restart each time with +nothing to indicate a problem. + ### Service-managed conversations When the model service stores the conversation, it identifies the thread with an id. The entity @@ -370,6 +375,13 @@ creates a fresh session per operation, so that id is **persisted in durable stat the next turn**; without it the service would start a new thread every turn. The durable history provider additionally no-ops (neither loading nor flushing) for service-managed sessions. +Whether the service owns history is decided with **core's precedence, not the client class alone**: +an explicit `store` in the agent's options wins, and only when it is unset does the client's +`STORES_BY_DEFAULT` apply. This matters because clients that store by default (such as the Responses +API) are routinely put back into client-side mode with `store=False`. Consulting only +`STORES_BY_DEFAULT` would leave such an agent with a plain in-memory provider that the durable +runtime never persists - silently losing the conversation between turns. + ### Retention is a deployment policy, not agent configuration Compaction annotates; it does not delete. Physically deleting excluded messages bounds durable @@ -378,6 +390,33 @@ with a per-agent override) rather than on the agent. This keeps the agent defini same agent runs in-memory, where a retention policy would be meaningless - and places the setting next to its natural sibling, entity lifetime/TTL. +## Related Concern: Entity Lifetime (TTL) and Cleanup + +Compaction bounds the *size* of a conversation; entity **lifetime** - when the persisted state is +deleted - is a separate axis. It is out of scope for the decision above, but is recorded here +because it is the natural sibling of the retention setting introduced by this ADR, and because it +has a notable cross-language parity gap in this repository. + +- **.NET agents:** `DurableAgentsOptions.DefaultTimeToLive` (default 14 days) provides a global TTL, + with a per-agent override via `AddAIAgent(agent, ttl)`. Idle entities self-delete via an + `ExpirationTimeUtc` + `CheckAndDeleteIfExpired` self-signal. +- **.NET workflows:** workflow agent executors are auto-registered *without* a TTL + (`DurableWorkflowOptions` calls `AddAIAgent(agent)`) and inherit the global default. There is **no + workflow-scoped TTL option**, and each agent-node invocation spawns a fresh, single-use entity that + then lingers for the full default (14 days) - far longer than needed for throwaway per-node state. +- **Python (agents *and* workflows):** there is **no TTL/cleanup mechanism at all** - no global + default, no per-agent option, no `expirationTimeUtc` in the state schema, and no deletion. Entities + persist indefinitely until manually deleted. This is a **.NET/Python parity gap**. + +Follow-ups (tracked separately from the compaction decision): + +1. **Port the TTL mechanism to Python** - a global default TTL, per-agent override, an + `expirationTimeUtc` state field (for cross-language schema parity), and idle-based self-deletion. +2. **Expose a configurable global TTL consistently** across both languages, for agents and workflows. +3. **Give workflow-spawned agent entities a sensible lifetime** - a short workflow-scoped default TTL, + or deterministic cleanup when the workflow completes, instead of the 14-day agent default (with an + idle-TTL backstop for workflows that pause or never reach a terminal state). + ## More Information - Builds on [ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md) (context compaction strategy), From 3ddb5898b95862a220de597eea7916359134d772 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 19:44:41 -0500 Subject: [PATCH 11/32] test: add samples and integration coverage for durable history and compaction Three samples, each showing an agent configured the ordinary core way running durably with no changes: compaction on the standalone worker (13) and on Azure Functions (14), and a user-owned external history store (14, Redis). The Redis sample defines its own small provider rather than depending on agent-framework-redis, whose only release is a beta that no longer imports against current core. Integration coverage asserts against real storage: compaction annotations and message ids survive entity serialization, an external provider keeps the whole conversation under one key, and a downstream workflow agent can reference the upstream conversation. Existing continuity tests were strengthened to assert recall rather than a bare 200. --- .../integration_tests/test_01_single_agent.py | 14 +- .../test_14_conversation_compaction.py | 83 ++++++++++ .../test_01_dt_single_agent.py | 20 ++- .../integration_tests/test_08_dt_workflow.py | 25 +++ .../test_13_dt_conversation_compaction.py | 125 +++++++++++++++ .../test_14_dt_external_history_redis.py | 132 ++++++++++++++++ .../13_conversation_compaction/.env.example | 5 + .../13_conversation_compaction/README.md | 85 ++++++++++ .../13_conversation_compaction/client.py | 102 ++++++++++++ .../requirements.txt | 13 ++ .../13_conversation_compaction/sample.py | 49 ++++++ .../13_conversation_compaction/worker.py | 147 ++++++++++++++++++ .../14_external_history_redis/.env.example | 8 + .../14_external_history_redis/README.md | 75 +++++++++ .../14_external_history_redis/client.py | 88 +++++++++++ .../redis_history_provider.py | 93 +++++++++++ .../requirements.txt | 13 ++ .../14_external_history_redis/sample.py | 49 ++++++ .../14_external_history_redis/worker.py | 130 ++++++++++++++++ python/samples/README.md | 5 + .../14_conversation_compaction/README.md | 77 +++++++++ .../14_conversation_compaction/demo.http | 63 ++++++++ .../function_app.py | 81 ++++++++++ .../14_conversation_compaction/host.json | 12 ++ .../local.settings.json.template | 11 ++ .../requirements.txt | 17 ++ 26 files changed, 1509 insertions(+), 13 deletions(-) create mode 100644 python/packages/azurefunctions/tests/integration_tests/test_14_conversation_compaction.py create mode 100644 python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py create mode 100644 python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py create mode 100644 python/samples/13_conversation_compaction/.env.example create mode 100644 python/samples/13_conversation_compaction/README.md create mode 100644 python/samples/13_conversation_compaction/client.py create mode 100644 python/samples/13_conversation_compaction/requirements.txt create mode 100644 python/samples/13_conversation_compaction/sample.py create mode 100644 python/samples/13_conversation_compaction/worker.py create mode 100644 python/samples/14_external_history_redis/.env.example create mode 100644 python/samples/14_external_history_redis/README.md create mode 100644 python/samples/14_external_history_redis/client.py create mode 100644 python/samples/14_external_history_redis/redis_history_provider.py create mode 100644 python/samples/14_external_history_redis/requirements.txt create mode 100644 python/samples/14_external_history_redis/sample.py create mode 100644 python/samples/14_external_history_redis/worker.py create mode 100644 python/samples/azure_functions/14_conversation_compaction/README.md create mode 100644 python/samples/azure_functions/14_conversation_compaction/demo.http create mode 100644 python/samples/azure_functions/14_conversation_compaction/function_app.py create mode 100644 python/samples/azure_functions/14_conversation_compaction/host.json create mode 100644 python/samples/azure_functions/14_conversation_compaction/local.settings.json.template create mode 100644 python/samples/azure_functions/14_conversation_compaction/requirements.txt diff --git a/python/packages/azurefunctions/tests/integration_tests/test_01_single_agent.py b/python/packages/azurefunctions/tests/integration_tests/test_01_single_agent.py index ff0e425..940189d 100644 --- a/python/packages/azurefunctions/tests/integration_tests/test_01_single_agent.py +++ b/python/packages/azurefunctions/tests/integration_tests/test_01_single_agent.py @@ -93,13 +93,13 @@ def test_legacy_thread_id_in_query_still_accepted(self) -> None: assert response.headers.get("x-ms-thread-id") is None def test_conversation_continuity(self) -> None: - """Test conversation context is maintained across requests.""" + """History must accumulate *and* reach the model on later turns.""" session_id = "test-continuity" - # First message + # First message establishes a fact that exists nowhere else. response1 = self.helper.post_json( f"{self.base_url}/run", - {"message": "Tell me a short joke about weather in Seattle.", "session_id": session_id}, + {"message": "My favorite animal is the axolotl. Tell me a short joke about it.", "session_id": session_id}, ) assert response1.status_code in [200, 202] @@ -107,13 +107,17 @@ def test_conversation_continuity(self) -> None: data1 = response1.json() assert data1["message_count"] == 2 # Initial + reply - # Second message in same session + # Second message in same session; only answerable from persisted history. response2 = self.helper.post_json( - f"{self.base_url}/run", {"message": "What about San Francisco?", "session_id": session_id} + f"{self.base_url}/run", + {"message": "What is my favorite animal? Reply with just the animal name.", "session_id": session_id}, ) assert response2.status_code == 200 data2 = response2.json() assert data2["message_count"] == 4 + assert "axolotl" in str(data2["response"]).lower(), ( + f"Agent lost conversation context across turns. Got: {data2['response']!r}" + ) else: # In async mode, we can't easily test message count # Just verify we can make multiple calls diff --git a/python/packages/azurefunctions/tests/integration_tests/test_14_conversation_compaction.py b/python/packages/azurefunctions/tests/integration_tests/test_14_conversation_compaction.py new file mode 100644 index 0000000..4d07c6e --- /dev/null +++ b/python/packages/azurefunctions/tests/integration_tests/test_14_conversation_compaction.py @@ -0,0 +1,83 @@ +# Copyright (c) Microsoft. All rights reserved. +""" +Integration Tests for the Conversation Compaction Sample + +Verifies that an agent configured the ordinary core way - an in-memory history provider plus a +compaction provider - runs durably under the Azure Functions host with no durable-specific +configuration, mirroring the standalone durabletask coverage. + +The function app is automatically started by the test fixture. + +Prerequisites: +- Azure OpenAI credentials configured (see packages/azurefunctions/tests/integration_tests/.env.example) +- Azurite or Azure Storage account configured + +Usage: + uv run pytest packages/azurefunctions/tests/integration_tests/test_14_conversation_compaction.py -v +""" + +import uuid + +import pytest + +# Matches function_app.py: only the most recent groups stay in the model's context. +KEEP_LAST_GROUPS = 4 + +# Module-level markers - applied to all tests in this file +pytestmark = [ + pytest.mark.flaky, + pytest.mark.integration, + pytest.mark.sample("14_conversation_compaction"), + pytest.mark.usefixtures("function_app_for_test"), +] + + +class TestSampleConversationCompaction: + """Tests for 14_conversation_compaction sample.""" + + @pytest.fixture(autouse=True) + def _setup(self, base_url: str, sample_helper) -> None: + """Provide agent-specific base URL and helper for the tests.""" + self.base_url = f"{base_url}/api/agents/Historian" + self.helper = sample_helper + + def _run(self, message: str, session_id: str) -> dict: + """Send one turn to the agent and return the parsed response. + + Args: + message: The user message for this turn. + session_id: The session id tying the turns into one conversation. + + Returns: + The parsed JSON response body. + """ + response = self.helper.post_json(f"{self.base_url}/run", {"message": message, "session_id": session_id}) + assert response.status_code in [200, 202] + return response.json() + + def test_health_check(self, base_url: str, sample_helper) -> None: + """Test health check endpoint.""" + response = sample_helper.get(f"{base_url}/api/health") + assert response.status_code == 200 + assert response.json()["status"] == "healthy" + + def test_recent_context_survives_compaction(self) -> None: + """A fact inside the retained window is still answerable after the window fills.""" + session_id = f"compaction-recent-{uuid.uuid4().hex[:8]}" + + for index in range(KEEP_LAST_GROUPS): + self._run(f"Name animal number {index + 1}.", session_id) + + self._run("My project codename is BLUEHERON.", session_id) + answer = self._run("What is my project codename? Reply with just the codename.", session_id) + + assert "blueheron" in str(answer["response"]).lower() + + def test_conversation_continues_across_turns(self) -> None: + """Durable history reaches the model, so the agent recalls an earlier turn.""" + session_id = f"compaction-continuity-{uuid.uuid4().hex[:8]}" + + self._run("My favorite animal is the axolotl.", session_id) + answer = self._run("What is my favorite animal? Reply with just the animal name.", session_id) + + assert "axolotl" in str(answer["response"]).lower() diff --git a/python/packages/durabletask/tests/integration_tests/test_01_dt_single_agent.py b/python/packages/durabletask/tests/integration_tests/test_01_dt_single_agent.py index 0bb5f4b..a546c2f 100644 --- a/python/packages/durabletask/tests/integration_tests/test_01_dt_single_agent.py +++ b/python/packages/durabletask/tests/integration_tests/test_01_dt_single_agent.py @@ -62,21 +62,25 @@ def test_single_interaction(self): assert len(response.text) > 0 def test_conversation_continuity(self): - """Test that conversation context is maintained across turns.""" + """Prior turns must reach the model, not just be recorded. + + The second turn is only answerable from persisted history, so this fails if durable + history is not actually being loaded and delivered to the agent. + """ agent = self.agent_client.get_agent("Joker") session = agent.create_session() - # First turn: Ask for a joke about a specific topic - response1 = agent.run("Tell me a joke about cats.", session=session) + # First turn establishes a fact that exists nowhere else. + response1 = agent.run("My favorite animal is the axolotl. Tell me a joke about it.", session=session) assert response1 is not None assert len(response1.text) > 0 - # Second turn: Ask a follow-up that requires context - response2 = agent.run("Can you make it funnier?", session=session) + # Second turn can only be answered from the conversation history. + response2 = agent.run("What is my favorite animal? Reply with just the animal name.", session=session) assert response2 is not None - assert len(response2.text) > 0 - - # The agent should understand "it" refers to the previous joke + assert "axolotl" in response2.text.lower(), ( + f"Agent lost conversation context across turns. Got: {response2.text!r}" + ) def test_multiple_sessions(self): """Test that different sessions maintain separate contexts.""" diff --git a/python/packages/durabletask/tests/integration_tests/test_08_dt_workflow.py b/python/packages/durabletask/tests/integration_tests/test_08_dt_workflow.py index 2aa9a9d..2d0be5d 100644 --- a/python/packages/durabletask/tests/integration_tests/test_08_dt_workflow.py +++ b/python/packages/durabletask/tests/integration_tests/test_08_dt_workflow.py @@ -69,6 +69,31 @@ def test_legitimate_email_drafts_response(self) -> None: assert output is not None assert "Email sent" in str(output) + def test_downstream_agent_receives_upstream_conversation(self) -> None: + """The email agent can only reference the original email if upstream context reached it. + + The edge into the email agent carries the spam agent's structured verdict, not the email. + A purchase order number is used as the marker because a spam verdict explains *why* a + message is legitimate and would not repeat an arbitrary code, whereas a drafted reply to + the email naturally does. + """ + instance_id = self.dts_client.schedule_new_orchestration( + orchestrator=workflow_orchestrator_name(WORKFLOW_NAME), + input=( + "Hi team, please confirm receipt of purchase order PRJ-4417 for the new lab " + "hardware, and let me know the expected delivery date." + ), + ) + + metadata, output = self.orch_helper.wait_for_orchestration_with_output( + instance_id=instance_id, + timeout=180.0, + ) + + assert metadata.runtime_status == OrchestrationStatus.COMPLETED + assert output is not None + assert "PRJ-4417" in str(output), f"drafted reply did not reference the original email: {output}" + def test_spam_email_handled(self) -> None: """A spam email routes to the non-agent spam handler.""" instance_id = self.dts_client.schedule_new_orchestration( diff --git a/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py new file mode 100644 index 0000000..60ab300 --- /dev/null +++ b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py @@ -0,0 +1,125 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Integration tests for durable conversation compaction. + +Covers the behavior an agent gets by simply being registered with the durable runtime: + +- history is persisted in the agent's durable entity and reaches the model on later turns, +- the configured compaction strategy runs and its annotations are persisted, so compaction + state survives entity state serialization rather than being recomputed each turn, +- the full conversation record is retained in storage even though the model sees less. +""" + +from typing import Any, Protocol + +import pytest +from durabletask.entities import EntityInstanceId + +from agent_framework_durabletask import DurableAgentState, DurableAIAgentClient + +# Matches worker.py: only the most recent groups stay in the model's context. +KEEP_LAST_GROUPS = 4 + + +class AgentClientFactoryProtocol(Protocol): + """Protocol for the agent client factory fixture.""" + + @classmethod + def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ... + + +pytestmark = [ + pytest.mark.flaky, + pytest.mark.integration, + pytest.mark.sample("13_conversation_compaction"), + pytest.mark.integration_test, + pytest.mark.requires_foundry, + pytest.mark.requires_dts, +] + + +class TestConversationCompaction: + """Compaction runs durably without any durable-specific agent configuration.""" + + @pytest.fixture(autouse=True) + def setup(self, agent_client_factory: type[AgentClientFactoryProtocol]) -> None: + """Setup test fixtures.""" + self.dts_client, self.agent_client = agent_client_factory.create() + + def _read_state(self, session_id: Any) -> DurableAgentState: + """Load the agent entity's persisted state straight from the scheduler.""" + entity_id = EntityInstanceId(entity=session_id.entity_name, key=session_id.key) + metadata = self.dts_client.get_entity(entity_id) + assert metadata is not None, f"no durable state found for {entity_id}" + + raw = metadata.get_state() + # The scheduler returns the entity payload as serialized JSON. + if isinstance(raw, str): + return DurableAgentState.from_json(raw) + assert isinstance(raw, dict), f"unexpected entity state payload: {type(raw)}" + return DurableAgentState.from_dict(raw) + + def test_agent_registration(self) -> None: + """The compacting agent is registered like any other agent.""" + agent = self.agent_client.get_agent("Historian") + assert agent is not None + assert agent.name == "Historian" + + def test_recent_context_survives_compaction(self) -> None: + """A fact inside the retained window is still answerable after several turns.""" + agent = self.agent_client.get_agent("Historian") + session = agent.create_session() + + for filler in ("Name a color.", "Name a country.", "Name a fruit."): + assert agent.run(filler, session=session) is not None + + agent.run("My project codename is BLUEHERON.", session=session) + answer = agent.run("What is my project codename? Reply with just the codename.", session=session) + + assert "blueheron" in answer.text.lower(), ( + f"Recent context was lost despite being inside the retained window. Got: {answer.text!r}" + ) + + def test_compaction_annotations_are_persisted(self) -> None: + """Compaction state must survive durable state serialization. + + This is what stops compaction from being recomputed on every turn, and it only works + because message-level metadata and ids are persisted with the conversation. + """ + agent = self.agent_client.get_agent("Historian") + session = agent.create_session() + + # Run enough turns that the sliding window must exclude earlier ones. + for index in range(KEEP_LAST_GROUPS + 3): + assert agent.run(f"Name animal number {index + 1}.", session=session) is not None + + state = self._read_state(session.durable_session_id) + + stored = [message for entry in state.data.conversation_history for message in entry.messages] + assert stored, "expected the conversation to be persisted" + + # Compaction excluded older messages, and that annotation round-tripped through storage. + annotated = [m for m in stored if m.extension_data] + assert annotated, "expected compaction annotations to be persisted in durable state" + + excluded = [m for m in annotated if (m.extension_data or {}).get("_excluded")] + assert excluded, "expected the sliding window to exclude older messages" + + # Reconciling compaction results across turns relies on stable ids, so every message + # the provider has processed must carry one. (The newest turn is annotated on the + # following load, so it is not required to have an id yet.) + assert all(m.message_id for m in annotated), "annotated messages must carry stable message ids" + + def test_full_record_is_retained(self) -> None: + """Compaction bounds what the model sees; it does not delete the record by default.""" + agent = self.agent_client.get_agent("Historian") + session = agent.create_session() + + turns = KEEP_LAST_GROUPS + 3 + for index in range(turns): + assert agent.run(f"Name city number {index + 1}.", session=session) is not None + + state = self._read_state(session.durable_session_id) + + # One request entry and one response entry per turn: nothing was pruned. + assert len(state.data.conversation_history) == turns * 2 diff --git a/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py b/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py new file mode 100644 index 0000000..19805a3 --- /dev/null +++ b/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py @@ -0,0 +1,132 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Integration tests for agents whose history lives in an external store. + +A user who deliberately configured their own history provider (Redis here, but Cosmos DB or a +file behaves the same) must get the same behavior under the durable runtime as in core: + +- the provider is not swapped out for durable-backed history, +- it participates in the run and its stored history reaches the model on later turns, +- it is handed the entity's stable session id, so its keys line up across turns. + +The last point is the load-bearing one: the entity builds a fresh session per operation, and if +that session carried a generated id an externally keyed store would silently start over every turn. +""" + +import os +from typing import Any, Protocol + +import pytest +import redis.asyncio as aioredis + +from agent_framework_durabletask import DurableAgentState, DurableAIAgentClient + + +class AgentClientFactoryProtocol(Protocol): + """Protocol for the agent client factory fixture.""" + + @classmethod + def create(cls, max_poll_retries: int = 90) -> tuple[Any, DurableAIAgentClient]: ... + + +pytestmark = [ + pytest.mark.flaky, + pytest.mark.integration, + pytest.mark.sample("14_external_history_redis"), + pytest.mark.integration_test, + pytest.mark.requires_foundry, + pytest.mark.requires_dts, + pytest.mark.requires_redis, +] + +# Matches redis_history_provider.py in the sample. +KEY_PREFIX = "durable_sample:history" + + +class TestExternalHistoryProvider: + """An external history provider works durably with no durable-specific configuration.""" + + @pytest.fixture(autouse=True) + def setup(self, agent_client_factory: type[AgentClientFactoryProtocol]) -> None: + """Setup test fixtures.""" + self.dts_client, self.agent_client = agent_client_factory.create() + self.redis_url = os.environ.get("REDIS_CONNECTION_STRING", "redis://localhost:6379") + + async def _history_entries(self, session_id: Any) -> list[str]: + """Read the raw history entries the sample's provider wrote for a session. + + Args: + session_id: The durable session id used for the conversation. + + Returns: + The serialized messages stored in Redis, oldest first. + """ + client = aioredis.from_url(self.redis_url, decode_responses=True) + try: + return await client.lrange(f"{KEY_PREFIX}:{session_id.key}", 0, -1) + finally: + await client.aclose() + + def test_agent_registration(self) -> None: + """The externally backed agent is registered like any other agent.""" + agent = self.agent_client.get_agent("Archivist") + assert agent is not None + assert agent.name == "Archivist" + + def test_history_from_the_external_store_reaches_the_model(self) -> None: + """Nothing else could supply the earlier turn, so recall proves the provider ran.""" + agent = self.agent_client.get_agent("Archivist") + session = agent.create_session() + + assert agent.run("My library card number is 4417.", session=session) is not None + answer = agent.run("What is my library card number? Reply with just the number.", session=session) + + assert answer is not None + assert "4417" in answer.text + + async def test_provider_is_keyed_by_the_stable_session_id(self) -> None: + """All turns must land under one key; a per-operation id would scatter them.""" + agent = self.agent_client.get_agent("Archivist") + session = agent.create_session() + + assert agent.run("Remember that my favorite number is 12.", session=session) is not None + assert agent.run("Remember that my favorite color is teal.", session=session) is not None + + entries = await self._history_entries(session.durable_session_id) + + # Two turns, each storing its input and the model's reply, all under the entity's own id. + assert len(entries) >= 4, f"expected the whole conversation under one key, found {len(entries)}" + assert any("12" in entry for entry in entries) + assert any("teal" in entry for entry in entries) + + def test_durable_state_still_records_the_conversation(self) -> None: + """Durable state remains the audit record even when history lives elsewhere.""" + agent = self.agent_client.get_agent("Archivist") + session = agent.create_session() + + assert agent.run("Note that the archive opens at nine.", session=session) is not None + + state = self._read_state(session.durable_session_id) + assert state.data.conversation_history, "expected the entity to record the conversation" + + def _read_state(self, session_id: Any) -> DurableAgentState: + """Load the agent entity's persisted state straight from the scheduler. + + Args: + session_id: The durable session id used for the conversation. + + Returns: + The deserialized durable agent state. + """ + from durabletask.entities import EntityInstanceId + + entity_id = EntityInstanceId(entity=session_id.entity_name, key=session_id.key) + metadata = self.dts_client.get_entity(entity_id) + assert metadata is not None, f"no durable state found for {entity_id}" + + raw = metadata.get_state() + # The scheduler returns the entity payload as serialized JSON. + if isinstance(raw, str): + return DurableAgentState.from_json(raw) + assert isinstance(raw, dict), f"unexpected entity state payload: {type(raw)}" + return DurableAgentState.from_dict(raw) diff --git a/python/samples/13_conversation_compaction/.env.example b/python/samples/13_conversation_compaction/.env.example new file mode 100644 index 0000000..b4ba5f8 --- /dev/null +++ b/python/samples/13_conversation_compaction/.env.example @@ -0,0 +1,5 @@ +# Azure OpenAI resource endpoint, e.g. https://your-resource.openai.azure.com/ +AZURE_OPENAI_ENDPOINT= + +# Model deployment name in your Azure OpenAI resource +AZURE_OPENAI_MODEL= diff --git a/python/samples/13_conversation_compaction/README.md b/python/samples/13_conversation_compaction/README.md new file mode 100644 index 0000000..5f2671b --- /dev/null +++ b/python/samples/13_conversation_compaction/README.md @@ -0,0 +1,85 @@ +# Conversation Compaction with Durable Agents + +Shows an agent whose conversation history is **persisted durably** and **compacted as it grows**, +using the same configuration you would write for in-process Agent Framework. + +## What this demonstrates + +The agent is built with a plain `InMemoryHistoryProvider` and a `CompactionProvider`: + +```python +history = InMemoryHistoryProvider(skip_excluded=True) +compaction = CompactionProvider( + after_strategy=SlidingWindowStrategy(keep_last_groups=4), + history_source_id=history.source_id, +) +agent = Agent( + client=..., + name="Historian", + default_options={"store": False}, + context_providers=[history, compaction], +) +``` + +Registering that agent with the durable runtime changes nothing about how you configure it, but: + +- **History becomes durable.** The runtime swaps the in-memory provider for a durable-backed one, + preserving its `source_id` so the compaction provider stays wired to it. Conversation state lives + in the agent's durable entity and survives worker restarts. +- **Compaction state is persisted.** Annotations produced by the strategy are stored alongside the + messages, so compaction is not recomputed from scratch on every turn. +- **Context stays bounded.** Only the messages the strategy keeps are sent to the model, so a long + conversation does not grow the per-turn context without limit. + +The full conversation remains in durable storage; compaction bounds what the *model* sees. To also +bound what is *stored*, opt in at registration with `add_agent(agent, prune_history=True)` — that is +lossy and therefore off by default. + +### Client-side vs service-managed history + +Compaction only applies to history the **client** owns. When a chat client keeps the conversation on +the service (Foundry threads, or the Responses API with `store=True`), the service owns the model's +context, the durable entity keeps the transcript purely as a record, and the durable history provider +stays out of the way. This sample sets `store=False` so history is client-side and compaction has +something to compact. + +## Running the sample + +1. Start the Durable Task Scheduler emulator: + + ```bash + docker run -d --name dts-emulator -p 8080:8080 -p 8082:8082 mcr.microsoft.com/dts/dts-emulator:latest + ``` + +2. Copy `.env.example` to `.env` and set `AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_MODEL`. + +3. Sign in for `AzureCliCredential`: + + ```bash + az login + ``` + +4. Install dependencies and start the worker: + + ```bash + pip install -r requirements.txt + python worker.py + ``` + +5. In another terminal, run the client: + + ```bash + python client.py + ``` + +## What to look for + +The client runs a multi-turn conversation and then asks the agent to recall a fact from a **recent** +turn, which it answers correctly. + +The trade-off is the point of the sample: a sliding window keeps context bounded by *dropping* older +turns from what the model sees, so facts from long-past turns are genuinely no longer available to +the model. Those messages are **not deleted** — they remain in durable storage, marked as excluded, +so the conversation record stays complete and auditable. Choose a strategy accordingly: use +summarization if old details must survive in the model's context, and a sliding window when only +recent context matters. diff --git a/python/samples/13_conversation_compaction/client.py b/python/samples/13_conversation_compaction/client.py new file mode 100644 index 0000000..2203104 --- /dev/null +++ b/python/samples/13_conversation_compaction/client.py @@ -0,0 +1,102 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Client that exercises a durable agent whose history is compacted as it grows. + +Runs a multi-turn conversation against the ``Historian`` agent hosted by ``worker.py`` and +shows that the conversation keeps working while the model's context stays bounded. +""" + +import logging +import os + +from agent_framework_durabletask import DurableAIAgentClient +from azure.identity import AzureCliCredential +from dotenv import load_dotenv +from durabletask.azuremanaged.client import DurableTaskSchedulerClient + +load_dotenv() + +logging.basicConfig(level=logging.WARNING) +logger = logging.getLogger(__name__) + +# Turns that fill the conversation before recall is tested. +FILLER_TURNS = [ + "Name a color.", + "Name a country.", + "Name a fruit.", + "Name a musical instrument.", +] + +CODENAME = "BLUEHERON" + + +def get_client( + taskhub: str | None = None, endpoint: str | None = None, log_handler: logging.Handler | None = None +) -> DurableAIAgentClient: + """Create a configured DurableAIAgentClient. + + Args: + taskhub: Task hub name (defaults to TASKHUB env var or "default") + endpoint: Scheduler endpoint (defaults to ENDPOINT env var or "http://localhost:8080") + log_handler: Optional logging handler for client logging + + Returns: + Configured DurableAIAgentClient instance + """ + taskhub_name = taskhub or os.getenv("TASKHUB", "default") + endpoint_url = endpoint or os.getenv("ENDPOINT", "http://localhost:8080") + + credential = None if endpoint_url == "http://localhost:8080" else AzureCliCredential() + + dts_client = DurableTaskSchedulerClient( + host_address=endpoint_url, + secure_channel=endpoint_url != "http://localhost:8080", + taskhub=taskhub_name, + token_credential=credential, + log_handler=log_handler, + ) + + return DurableAIAgentClient(dts_client) + + +def run_client(agent_client: DurableAIAgentClient) -> None: + """Run a multi-turn conversation against the compacting agent. + + Args: + agent_client: The durable agent client to use. + """ + agent = agent_client.get_agent("Historian") + session = agent.create_session() + + print("Running a multi-turn conversation...\n") + + for turn in FILLER_TURNS: + response = agent.run(turn, session=session) + print(f"[user] {turn}") + print(f"[agent] {response.text}\n") + + fact = f"My project codename is {CODENAME}." + print(f"[user] {fact}") + print(f"[agent] {agent.run(fact, session=session).text}\n") + + question = "What is my project codename? Reply with just the codename." + answer = agent.run(question, session=session) + print(f"[user] {question}") + print(f"[agent] {answer.text}\n") + + if CODENAME.lower() in answer.text.lower(): + print("Recent context was retained while the conversation stayed compacted.") + else: + print("The codename fell outside the retained window.") + + +def main() -> None: + """Client entry point.""" + try: + run_client(get_client()) + except Exception as e: + logger.exception(f"Error during agent interaction: {e}") + + +if __name__ == "__main__": + main() diff --git a/python/samples/13_conversation_compaction/requirements.txt b/python/samples/13_conversation_compaction/requirements.txt new file mode 100644 index 0000000..0fd0008 --- /dev/null +++ b/python/samples/13_conversation_compaction/requirements.txt @@ -0,0 +1,13 @@ +# Agent Framework packages +# To use the deployed version, uncomment the lines below and comment out the local installation lines +# agent-framework-openai +# agent-framework-durabletask + +# Local installation (for development and testing) +# Each package must be listed explicitly because pip doesn't resolve uv workspace sources. +# Without explicit entries, pip would fetch transitive dependencies from PyPI instead of local source. +agent-framework-openai>=1.10.1,<2 # Azure OpenAI support from PyPI +-e ../../packages/durabletask # Local Durable Task package under development + +# Azure authentication +azure-identity \ No newline at end of file diff --git a/python/samples/13_conversation_compaction/sample.py b/python/samples/13_conversation_compaction/sample.py new file mode 100644 index 0000000..16463d7 --- /dev/null +++ b/python/samples/13_conversation_compaction/sample.py @@ -0,0 +1,49 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Conversation Compaction Sample - Durable Task Integration (Combined Worker + Client) + +Runs both the worker and client in a single process. The worker is started first to +register the compacting agent, then the client drives a multi-turn conversation. + +Prerequisites: +- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODEL +- Sign in with Azure CLI for AzureCliCredential authentication +- Durable Task Scheduler must be running (e.g., using Docker) + +To run this sample: + python sample.py +""" + +import logging + +from client import get_client, run_client # pyrefly: ignore[missing-import] +from dotenv import load_dotenv +from worker import get_worker, setup_worker # pyrefly: ignore[missing-import] + +# Configure logging (must be after imports to override their basicConfig) +logging.basicConfig(level=logging.INFO, force=True) +logger = logging.getLogger(__name__) + + +def main(): + """Main entry point - runs both worker and client in single process.""" + silent_handler = logging.NullHandler() + + dts_worker = get_worker(log_handler=silent_handler) + with dts_worker: + setup_worker(dts_worker) + dts_worker.start() + logger.debug("Worker started and listening for requests...") + + agent_client = get_client(log_handler=silent_handler) + try: + run_client(agent_client) + except Exception as e: + logger.exception(f"Error during agent interaction: {e}") + + logger.debug("Sample completed. Worker shutting down...") + + +if __name__ == "__main__": + load_dotenv() + main() diff --git a/python/samples/13_conversation_compaction/worker.py b/python/samples/13_conversation_compaction/worker.py new file mode 100644 index 0000000..aff201f --- /dev/null +++ b/python/samples/13_conversation_compaction/worker.py @@ -0,0 +1,147 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Worker hosting an agent whose conversation history is compacted as it grows. + +The agent is configured exactly as it would be for in-process Agent Framework: an +``InMemoryHistoryProvider`` plus a ``CompactionProvider``. Registering it with the durable +runtime transparently swaps the history provider for a durable-backed one, so: + +- conversation history is persisted in the agent's durable entity and survives restarts, +- the compaction strategy still runs, and its annotations are persisted alongside the + messages, so compaction state is not recomputed on every turn, +- only the messages compaction keeps are sent to the model, bounding context growth. + +No durable-specific configuration is required on the agent itself. + +Note on service-managed conversations: compaction applies to history the *client* owns. When a +chat client keeps the conversation on the service (for example Foundry threads, or the Responses +API with ``store=True``), the service owns the model's context and the durable entity keeps the +full transcript purely as a record. This sample therefore uses ``store=False`` so history is +client-side and compaction has something to compact. + +Prerequisites: +- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODEL +- Sign in with Azure CLI for AzureCliCredential authentication +- Start a Durable Task Scheduler (e.g., using Docker) +""" + +import asyncio +import logging +import os + +from agent_framework import Agent, CompactionProvider, InMemoryHistoryProvider, SlidingWindowStrategy +from agent_framework.openai import OpenAIChatClient +from agent_framework_durabletask import DurableAIAgentWorker +from azure.identity import AzureCliCredential +from azure.identity.aio import AzureCliCredential as AsyncAzureCliCredential +from dotenv import load_dotenv +from durabletask.azuremanaged.worker import DurableTaskSchedulerWorker + +# Load environment variables from .env file +load_dotenv() + +# Configure logging +logging.basicConfig(level=logging.WARNING) +logger = logging.getLogger(__name__) + +# Keep only the most recent turns in the model's context. Deliberately small so the +# effect is easy to observe in a short sample conversation. +KEEP_LAST_GROUPS = 4 + + +def create_historian_agent() -> Agent: + """Create an agent that remembers facts while its context stays bounded. + + Returns: + Agent: The configured Historian agent. + """ + # A plain in-memory history provider: the durable runtime replaces it with a + # durable-backed provider at registration, preserving this ``source_id`` so the + # compaction provider below stays wired to it. + history = InMemoryHistoryProvider(skip_excluded=True) + + compaction = CompactionProvider( + after_strategy=SlidingWindowStrategy(keep_last_groups=KEEP_LAST_GROUPS), + history_source_id=history.source_id, + ) + + return Agent( + client=OpenAIChatClient( + azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], + model=os.environ["AZURE_OPENAI_MODEL"], + credential=AsyncAzureCliCredential(), + ), + name="Historian", + instructions=( + "You are a concise assistant. Answer in one short sentence. " + "When the user tells you a fact, remember it and repeat it exactly when asked." + ), + # Keep the conversation client-side so the history provider (and therefore compaction) + # owns the model's context. + default_options={"store": False}, + context_providers=[history, compaction], + ) + + +def get_worker( + taskhub: str | None = None, endpoint: str | None = None, log_handler: logging.Handler | None = None +) -> DurableTaskSchedulerWorker: + """Create a configured DurableTaskSchedulerWorker. + + Args: + taskhub: Task hub name (defaults to TASKHUB env var or "default") + endpoint: Scheduler endpoint (defaults to ENDPOINT env var or "http://localhost:8080") + log_handler: Optional logging handler for worker logging + + Returns: + Configured DurableTaskSchedulerWorker instance + """ + taskhub_name = taskhub or os.getenv("TASKHUB", "default") + endpoint_url = endpoint or os.getenv("ENDPOINT", "http://localhost:8080") + + credential = None if endpoint_url == "http://localhost:8080" else AzureCliCredential() + + return DurableTaskSchedulerWorker( + host_address=endpoint_url, + secure_channel=endpoint_url != "http://localhost:8080", + taskhub=taskhub_name, + token_credential=credential, + log_handler=log_handler, + ) + + +def setup_worker(worker: DurableTaskSchedulerWorker) -> DurableAIAgentWorker: + """Register the compacting agent with the durable worker. + + Args: + worker: The DurableTaskSchedulerWorker instance + + Returns: + DurableAIAgentWorker with agents registered + """ + agent_worker = DurableAIAgentWorker(worker) + + agent = create_historian_agent() + agent_worker.add_agent(agent) + + logger.debug(f"✓ Registered agent: {agent.name}") + return agent_worker + + +async def main(): + """Main entry point for the worker process.""" + worker = get_worker() + setup_worker(worker) + + logger.info("Worker is ready and listening for requests...") + + try: + worker.start() + while True: + await asyncio.sleep(1) + except KeyboardInterrupt: + logger.debug("Worker shutdown initiated") + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/python/samples/14_external_history_redis/.env.example b/python/samples/14_external_history_redis/.env.example new file mode 100644 index 0000000..58036ae --- /dev/null +++ b/python/samples/14_external_history_redis/.env.example @@ -0,0 +1,8 @@ +# Azure OpenAI resource endpoint, e.g. https://your-resource.openai.azure.com/ +AZURE_OPENAI_ENDPOINT= + +# Model deployment name in your Azure OpenAI resource +AZURE_OPENAI_MODEL= + +# Redis connection string used by the external history provider +REDIS_CONNECTION_STRING=redis://localhost:6379 diff --git a/python/samples/14_external_history_redis/README.md b/python/samples/14_external_history_redis/README.md new file mode 100644 index 0000000..2126e31 --- /dev/null +++ b/python/samples/14_external_history_redis/README.md @@ -0,0 +1,75 @@ +# External Conversation History (Redis) with Durable Agents + +Shows an agent whose conversation history lives in a **user-chosen external store** rather than in +durable entity state, using the same configuration you would write for in-process Agent Framework. + +## What this demonstrates + +The agent is built with an ordinary `HistoryProvider` that happens to be backed by Redis: + +```python +history = RedisHistoryProvider("redis://localhost:6379") +agent = Agent( + client=..., + name="Archivist", + default_options={"store": False}, + context_providers=[history], +) +``` + +Registering that agent with the durable runtime changes nothing about how you configure it: + +- **Your provider is left alone.** Unlike an `InMemoryHistoryProvider` — which is swapped for a + durable-backed one (see [13_conversation_compaction](../13_conversation_compaction)) — a provider + you chose deliberately is never substituted. You picked where the conversation lives. +- **It receives a stable session id.** The durable entity creates a fresh session per operation but + gives it the entity's own session id, so the provider reads and writes the same key every turn. + Without that, an externally keyed store would start a new conversation on each turn. +- **Execution is still durable.** Retries, restarts, and orchestration guarantees are unchanged, and + durable state still records the conversation for audit. + +`redis_history_provider.py` is deliberately small — roughly "read a list, append to a list" — to show +how little a bring-your-own-store provider needs. The same shape applies to Cosmos DB, a file, or any +other backend. + +## Running the sample + +1. Start the Durable Task Scheduler emulator and Redis: + + ```bash + docker run -d --name dts-emulator -p 8080:8080 -p 8082:8082 mcr.microsoft.com/dts/dts-emulator:latest + docker run -d --name redis -p 6379:6379 redis:latest + ``` + +2. Copy `.env.example` to `.env` and set `AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_MODEL`. + +3. Sign in for `AzureCliCredential`: + + ```bash + az login + ``` + +4. Install dependencies and start the worker: + + ```bash + pip install -r requirements.txt + python worker.py + ``` + +5. In another terminal, run the client: + + ```bash + python client.py + ``` + +## What to look for + +The client states a fact and then asks for it back in a later turn. The agent answers correctly, +which is only possible if Redis served the earlier turn back into the model's context — the durable +runtime itself never replays history for this agent. + +To see it directly, inspect the Redis key while the sample runs: + +```bash +docker exec -it redis redis-cli KEYS 'durable_sample:history:*' +``` diff --git a/python/samples/14_external_history_redis/client.py b/python/samples/14_external_history_redis/client.py new file mode 100644 index 0000000..f4b3071 --- /dev/null +++ b/python/samples/14_external_history_redis/client.py @@ -0,0 +1,88 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Client that exercises a durable agent whose history lives in Redis. + +Runs a multi-turn conversation against the ``Archivist`` agent hosted by ``worker.py`` and shows +that a user-chosen external store keeps the conversation going under the durable runtime. +""" + +import logging +import os + +from agent_framework_durabletask import DurableAIAgentClient +from azure.identity import AzureCliCredential +from dotenv import load_dotenv +from durabletask.azuremanaged.client import DurableTaskSchedulerClient + +load_dotenv() + +logging.basicConfig(level=logging.WARNING) +logger = logging.getLogger(__name__) + +FACT = "My library card number is 4417." + + +def get_client( + taskhub: str | None = None, endpoint: str | None = None, log_handler: logging.Handler | None = None +) -> DurableAIAgentClient: + """Create a configured DurableAIAgentClient. + + Args: + taskhub: Task hub name (defaults to TASKHUB env var or "default") + endpoint: Scheduler endpoint (defaults to ENDPOINT env var or "http://localhost:8080") + log_handler: Optional logging handler for client logging + + Returns: + Configured DurableAIAgentClient instance + """ + taskhub_name = taskhub or os.getenv("TASKHUB", "default") + endpoint_url = endpoint or os.getenv("ENDPOINT", "http://localhost:8080") + + credential = None if endpoint_url == "http://localhost:8080" else AzureCliCredential() + + dts_client = DurableTaskSchedulerClient( + host_address=endpoint_url, + secure_channel=endpoint_url != "http://localhost:8080", + taskhub=taskhub_name, + token_credential=credential, + log_handler=log_handler, + ) + + return DurableAIAgentClient(dts_client) + + +def run_client(agent_client: DurableAIAgentClient) -> None: + """Run a multi-turn conversation served from the external Redis store. + + Args: + agent_client: The durable agent client to use. + """ + agent = agent_client.get_agent("Archivist") + session = agent.create_session() + + print("Running a multi-turn conversation backed by Redis...\n") + + print(f"[user] {FACT}") + print(f"[agent] {agent.run(FACT, session=session).text}\n") + + question = "What is my library card number? Reply with just the number." + answer = agent.run(question, session=session) + print(f"[user] {question}") + print(f"[agent] {answer.text}\n") + + if "4417" in answer.text: + print("The agent recalled the fact, so Redis served the prior turn back to the model.") + else: + print("The agent did not recall the fact - check that Redis is reachable.") + + +def main() -> None: + """Client entry point.""" + try: + run_client(get_client()) + except Exception as e: + logger.exception(f"Error during agent interaction: {e}") + + +if __name__ == "__main__": + main() diff --git a/python/samples/14_external_history_redis/redis_history_provider.py b/python/samples/14_external_history_redis/redis_history_provider.py new file mode 100644 index 0000000..0241dcc --- /dev/null +++ b/python/samples/14_external_history_redis/redis_history_provider.py @@ -0,0 +1,93 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""A minimal Redis-backed history provider. + +This is an ordinary Agent Framework ``HistoryProvider`` - nothing about it is durable-specific. +It is included in the sample rather than imported from a package to keep the sample dependency +free and to show exactly how little a "bring your own store" provider needs: read the messages +for a session id, append new ones. + +The durable runtime leaves providers like this alone: the user chose where their conversation +lives, so durable supplies execution durability and stays out of the way of storage. +""" + +from collections.abc import Sequence +from typing import Any + +import redis.asyncio as aioredis +from agent_framework import HistoryProvider, Message + + +class RedisHistoryProvider(HistoryProvider): + """Stores conversation history in a Redis list, one entry per message. + + Messages are keyed by session id, so the same session id must be used on every turn for the + conversation to continue - which is exactly what the durable entity guarantees. + """ + + DEFAULT_SOURCE_ID = "redis_history" + + def __init__( + self, + redis_url: str, + *, + source_id: str = DEFAULT_SOURCE_ID, + key_prefix: str = "durable_sample:history", + ) -> None: + """Create a Redis-backed history provider. + + Args: + redis_url: Redis connection URL, for example ``redis://localhost:6379``. + source_id: Unique identifier for this provider instance. + key_prefix: Prefix for the Redis keys this provider owns. + """ + super().__init__(source_id) + self.key_prefix = key_prefix + self._client: aioredis.Redis = aioredis.from_url(redis_url, decode_responses=True) + + def _key(self, session_id: str | None) -> str: + """Build the Redis key holding the history for a session. + + Args: + session_id: The session ID to build a key for. + + Returns: + The Redis key for this session's history. + """ + return f"{self.key_prefix}:{session_id or 'default'}" + + async def get_messages( + self, session_id: str | None, *, state: dict[str, Any] | None = None, **kwargs: Any + ) -> list[Message]: + """Read this session's messages from Redis, oldest first. + + Args: + session_id: The session ID to retrieve messages for. + state: Unused; this provider keeps nothing in session state. + **kwargs: Additional arguments (unused). + + Returns: + The stored messages in chronological order. + """ + stored: list[str] = await self._client.lrange(self._key(session_id), 0, -1) + return [Message.from_json(entry) for entry in stored] + + async def save_messages( + self, + session_id: str | None, + messages: Sequence[Message], + *, + state: dict[str, Any] | None = None, + **kwargs: Any, + ) -> None: + """Append messages to this session's Redis list. + + Args: + session_id: The session ID to store messages for. + messages: The messages to persist. + state: Unused; this provider keeps nothing in session state. + **kwargs: Additional arguments (unused). + """ + if not messages: + return + await self._client.rpush(self._key(session_id), *[message.to_json() for message in messages]) diff --git a/python/samples/14_external_history_redis/requirements.txt b/python/samples/14_external_history_redis/requirements.txt new file mode 100644 index 0000000..21e7174 --- /dev/null +++ b/python/samples/14_external_history_redis/requirements.txt @@ -0,0 +1,13 @@ +# Agent Framework packages +# To use the deployed version, uncomment the lines below and comment out the local installation lines +# agent-framework-openai +# agent-framework-durabletask + +# Local installation (for development and testing) +# Each package must be listed explicitly because pip doesn't resolve uv workspace sources. +# Without explicit entries, pip would fetch transitive dependencies from PyPI instead of local source. +agent-framework-openai>=1.10.1,<2 # Azure OpenAI support from PyPI +-e ../../packages/durabletask # Local Durable Task package under development + +# External history store used by this sample +redis diff --git a/python/samples/14_external_history_redis/sample.py b/python/samples/14_external_history_redis/sample.py new file mode 100644 index 0000000..10c3739 --- /dev/null +++ b/python/samples/14_external_history_redis/sample.py @@ -0,0 +1,49 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""External History (Redis) Sample - Durable Task Integration (Combined Worker + Client) + +Runs both the worker and client in a single process. The worker is started first to register +the Redis-backed agent, then the client drives a multi-turn conversation. + +Prerequisites: +- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODEL +- Sign in with Azure CLI for AzureCliCredential authentication +- Durable Task Scheduler and Redis must be running (e.g., using Docker) + +To run this sample: + python sample.py +""" + +import logging + +from client import get_client, run_client # pyrefly: ignore[missing-import] +from dotenv import load_dotenv +from worker import get_worker, setup_worker # pyrefly: ignore[missing-import] + +# Configure logging (must be after imports to override their basicConfig) +logging.basicConfig(level=logging.INFO, force=True) +logger = logging.getLogger(__name__) + + +def main(): + """Main entry point - runs both worker and client in single process.""" + silent_handler = logging.NullHandler() + + dts_worker = get_worker(log_handler=silent_handler) + with dts_worker: + setup_worker(dts_worker) + dts_worker.start() + logger.debug("Worker started and listening for requests...") + + agent_client = get_client(log_handler=silent_handler) + try: + run_client(agent_client) + except Exception as e: + logger.exception(f"Error during agent interaction: {e}") + + logger.debug("Sample completed. Worker shutting down...") + + +if __name__ == "__main__": + load_dotenv() + main() diff --git a/python/samples/14_external_history_redis/worker.py b/python/samples/14_external_history_redis/worker.py new file mode 100644 index 0000000..b50c3dd --- /dev/null +++ b/python/samples/14_external_history_redis/worker.py @@ -0,0 +1,130 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Worker hosting an agent whose conversation history lives in Redis, not in durable state. + +The agent is configured exactly as it would be for in-process Agent Framework: a history +provider the user chose (here Redis) is passed as a context provider. Registering it with the +durable runtime requires no changes: + +- the runtime **leaves the provider alone** - the user picked where their conversation lives, +- it hands the provider the entity's **stable** session id on every turn, so history continues + across turns and across worker restarts, +- durable state still records the conversation for audit, and execution stays durable. + +Contrast with ``13_conversation_compaction``, where an in-memory provider is transparently +swapped for a durable-backed one. + +Prerequisites: +- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODEL +- Sign in with Azure CLI for AzureCliCredential authentication +- Start a Durable Task Scheduler and a Redis instance (e.g., using Docker) +""" + +import asyncio +import logging +import os + +from agent_framework import Agent +from agent_framework.openai import OpenAIChatClient +from agent_framework_durabletask import DurableAIAgentWorker +from azure.identity import AzureCliCredential +from azure.identity.aio import AzureCliCredential as AsyncAzureCliCredential +from dotenv import load_dotenv +from durabletask.azuremanaged.worker import DurableTaskSchedulerWorker +from redis_history_provider import RedisHistoryProvider # pyrefly: ignore[missing-import] + +# Load environment variables from .env file +load_dotenv() + +# Configure logging +logging.basicConfig(level=logging.WARNING) +logger = logging.getLogger(__name__) + + +def create_archivist_agent() -> Agent: + """Create an agent whose history is stored in Redis. + + Returns: + Agent: The configured Archivist agent. + """ + history = RedisHistoryProvider(os.getenv("REDIS_CONNECTION_STRING", "redis://localhost:6379")) + + return Agent( + client=OpenAIChatClient( + azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], + model=os.environ["AZURE_OPENAI_MODEL"], + credential=AsyncAzureCliCredential(), + ), + name="Archivist", + instructions=( + "You are a concise assistant. Answer in one short sentence. " + "When the user tells you a fact, remember it and repeat it exactly when asked." + ), + # Keep the conversation client-side so the history provider owns the model's context. + default_options={"store": False}, + context_providers=[history], + ) + + +def get_worker( + taskhub: str | None = None, endpoint: str | None = None, log_handler: logging.Handler | None = None +) -> DurableTaskSchedulerWorker: + """Create a configured DurableTaskSchedulerWorker. + + Args: + taskhub: Task hub name (defaults to TASKHUB env var or "default") + endpoint: Scheduler endpoint (defaults to ENDPOINT env var or "http://localhost:8080") + log_handler: Optional logging handler for worker logging + + Returns: + Configured DurableTaskSchedulerWorker instance + """ + taskhub_name = taskhub or os.getenv("TASKHUB", "default") + endpoint_url = endpoint or os.getenv("ENDPOINT", "http://localhost:8080") + + credential = None if endpoint_url == "http://localhost:8080" else AzureCliCredential() + + return DurableTaskSchedulerWorker( + host_address=endpoint_url, + secure_channel=endpoint_url != "http://localhost:8080", + taskhub=taskhub_name, + token_credential=credential, + log_handler=log_handler, + ) + + +def setup_worker(worker: DurableTaskSchedulerWorker) -> DurableAIAgentWorker: + """Register the Redis-backed agent with the durable worker. + + Args: + worker: The DurableTaskSchedulerWorker instance + + Returns: + DurableAIAgentWorker with agents registered + """ + agent_worker = DurableAIAgentWorker(worker) + + agent = create_archivist_agent() + agent_worker.add_agent(agent) + + logger.debug(f"✓ Registered agent: {agent.name}") + return agent_worker + + +async def main(): + """Main entry point for the worker process.""" + worker = get_worker() + setup_worker(worker) + + logger.info("Worker is ready and listening for requests...") + + try: + worker.start() + while True: + await asyncio.sleep(1) + except KeyboardInterrupt: + logger.debug("Worker shutdown initiated") + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/python/samples/README.md b/python/samples/README.md index 81a95ef..11277c0 100644 --- a/python/samples/README.md +++ b/python/samples/README.md @@ -70,6 +70,10 @@ az account show - **[11_subworkflow](11_subworkflow/)**: Compose workflows by embedding an inner `Workflow` as a node via `WorkflowExecutor`. On the durable host the inner workflow runs as its own child orchestration, and a single `configure_workflow` call registers both. - **[12_subworkflow_hitl](12_subworkflow_hitl/)**: A human-in-the-loop pause that lives **inside a sub-workflow**. The nested request surfaces to the client with a qualified request id (`{executor}~{ordinal}~{requestId}`) behind a single top-level addressing surface. +### Conversation History +- **[13_conversation_compaction](13_conversation_compaction/)**: Persist conversation history durably and compact it as it grows. An agent configured the ordinary core way (`InMemoryHistoryProvider` + `CompactionProvider`) gets durable-backed history automatically, with compaction annotations persisted alongside the messages. +- **[14_external_history_redis](14_external_history_redis/)**: Keep conversation history in a store you chose (Redis here) instead of durable state. The durable runtime leaves your provider alone and hands it the entity's stable session id, so it continues the conversation across turns and restarts. + ### Azure Functions Hosting These samples host workflows and agents on Azure Durable Functions (`func start`) instead of the worker-client model above. Each has its own setup steps in its README, and shared environment setup lives in [azure_functions/README.md](azure_functions/README.md). @@ -87,6 +91,7 @@ These samples host workflows and agents on Azure Durable Functions (`func start` - **[azure_functions/11_workflow_parallel](azure_functions/11_workflow_parallel/)**: Parallel execution of executors and agents in an Azure Durable Functions workflow. - **[azure_functions/12_workflow_hitl](azure_functions/12_workflow_hitl/)**: The workflow human-in-the-loop pattern on Azure Durable Functions, with the reviewer notified from inside the workflow via `WorkflowHitlContext`. - **[azure_functions/13_subworkflow_hitl](azure_functions/13_subworkflow_hitl/)**: A human-in-the-loop pause inside a sub-workflow on Azure Durable Functions, exposed through a single top-level respond surface. +- **[azure_functions/14_conversation_compaction](azure_functions/14_conversation_compaction/)**: Persist conversation history durably and compact it as it grows, on Azure Functions. The Functions counterpart to [13_conversation_compaction](13_conversation_compaction/). ## Running the Samples diff --git a/python/samples/azure_functions/14_conversation_compaction/README.md b/python/samples/azure_functions/14_conversation_compaction/README.md new file mode 100644 index 0000000..dc21d4e --- /dev/null +++ b/python/samples/azure_functions/14_conversation_compaction/README.md @@ -0,0 +1,77 @@ +# Conversation Compaction Sample (Python) + +This sample demonstrates hosting an agent whose conversation history is **persisted durably** and +**compacted as it grows**, using the same configuration you would write for in-process Agent +Framework. It is the Azure Functions counterpart to the standalone +[`13_conversation_compaction`](../../13_conversation_compaction) sample. + +## Key Concepts Demonstrated + +- Configuring compaction the ordinary core way — an `InMemoryHistoryProvider` plus a + `CompactionProvider` — with **no durable-specific configuration on the agent**. +- The durable runtime swapping the in-memory provider for a durable-backed one at registration, + preserving its `source_id` so the compaction provider stays wired to it. +- Compaction annotations being persisted alongside the messages, so compaction state is not + recomputed from scratch on every turn. +- Context growth being bounded: only the messages the strategy keeps are sent to the model. + +```python +history = InMemoryHistoryProvider(skip_excluded=True) +compaction = CompactionProvider( + after_strategy=SlidingWindowStrategy(keep_last_groups=4), + history_source_id=history.source_id, +) +agent = Agent( + client=..., + name="Historian", + default_options={"store": False}, + context_providers=[history, compaction], +) + +app = AgentFunctionApp(agents=[agent], enable_health_check=True) +``` + +The full conversation remains in durable storage; compaction bounds what the *model* sees. To also +bound what is *stored*, opt in at registration with `AgentFunctionApp(..., prune_history=True)` — +that is lossy and therefore off by default. + +### Client-side vs service-managed history + +Compaction only applies to history the **client** owns. When a chat client keeps the conversation on +the service (Foundry threads, or the Responses API with `store=True`), the service owns the model's +context, the durable entity keeps the transcript purely as a record, and the durable history provider +stays out of the way. This sample sets `store=False` so history is client-side and compaction has +something to compact. + +## Prerequisites + +Follow the common setup steps in `../README.md` to install tooling, configure Azure OpenAI +credentials, and install the Python dependencies for this sample. This sample uses +`AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_MODEL`. + +## Running the Sample + +Send several turns using the **same** session id so they form one conversation. `demo.http` contains +a ready-made sequence; the equivalent with `curl` is: + +```bash +curl -X POST http://localhost:7071/api/agents/Historian/run \ + -H "Content-Type: application/json" \ + -d '{"message": "My project codename is BLUEHERON.", "session_id": "compaction-demo-001"}' + +curl -X POST http://localhost:7071/api/agents/Historian/run \ + -H "Content-Type: application/json" \ + -d '{"message": "What is my project codename? Reply with just the codename.", "session_id": "compaction-demo-001"}' +``` + +## What to look for + +The agent answers correctly from a **recent** turn while older turns fall outside the retained +window. + +The trade-off is the point of the sample: a sliding window keeps context bounded by *dropping* older +turns from what the model sees, so facts from long-past turns are genuinely no longer available to +the model. Those messages are **not deleted** — they remain in durable storage, marked as excluded, +so the conversation record stays complete and auditable. Choose a strategy accordingly: use +summarization if old details must survive in the model's context, and a sliding window when only +recent context matters. diff --git a/python/samples/azure_functions/14_conversation_compaction/demo.http b/python/samples/azure_functions/14_conversation_compaction/demo.http new file mode 100644 index 0000000..e273795 --- /dev/null +++ b/python/samples/azure_functions/14_conversation_compaction/demo.http @@ -0,0 +1,63 @@ +### Conversation Compaction Sample Interactions +@baseUrl = http://localhost:7071 +@agentName = Historian +@agentRoute = {{baseUrl}}/api/agents/{{agentName}} +@healthRoute = {{baseUrl}}/api/health +@sessionId = compaction-demo-001 + +### Health Check +GET {{healthRoute}} + +### Turn 1 - filler +POST {{agentRoute}}/run +Content-Type: application/json + +{ + "message": "Name a color.", + "session_id": "{{sessionId}}" +} + +### Turn 2 - filler +POST {{agentRoute}}/run +Content-Type: application/json + +{ + "message": "Name a country.", + "session_id": "{{sessionId}}" +} + +### Turn 3 - filler +POST {{agentRoute}}/run +Content-Type: application/json + +{ + "message": "Name a fruit.", + "session_id": "{{sessionId}}" +} + +### Turn 4 - filler +POST {{agentRoute}}/run +Content-Type: application/json + +{ + "message": "Name a musical instrument.", + "session_id": "{{sessionId}}" +} + +### Turn 5 - state the fact to recall later +POST {{agentRoute}}/run +Content-Type: application/json + +{ + "message": "My project codename is BLUEHERON.", + "session_id": "{{sessionId}}" +} + +### Turn 6 - the fact is inside the retained window, so it is answered +POST {{agentRoute}}/run +Content-Type: application/json + +{ + "message": "What is my project codename? Reply with just the codename.", + "session_id": "{{sessionId}}" +} diff --git a/python/samples/azure_functions/14_conversation_compaction/function_app.py b/python/samples/azure_functions/14_conversation_compaction/function_app.py new file mode 100644 index 0000000..fdab6f3 --- /dev/null +++ b/python/samples/azure_functions/14_conversation_compaction/function_app.py @@ -0,0 +1,81 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Host an agent whose conversation history is compacted as it grows, inside Azure Functions. + +The agent is configured exactly as it would be for in-process Agent Framework: an +``InMemoryHistoryProvider`` plus a ``CompactionProvider``. Registering it with +``AgentFunctionApp`` transparently swaps the history provider for a durable-backed one, so +history is persisted in the agent's durable entity, the compaction strategy still runs, and its +annotations are persisted alongside the messages. Only the messages compaction keeps are sent to +the model, bounding context growth. + +This is the Azure Functions counterpart to the standalone ``13_conversation_compaction`` sample. + +Note on service-managed conversations: compaction applies to history the *client* owns. When a +chat client keeps the conversation on the service (for example Foundry threads, or the Responses +API with ``store=True``), the service owns the model's context and the durable entity keeps the +full transcript purely as a record. This sample therefore uses ``store=False`` so history is +client-side and compaction has something to compact. + +Prerequisites: set `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_MODEL`, and sign in +with Azure CLI before starting the Functions host.""" + +import os +from typing import Any + +from agent_framework import Agent, CompactionProvider, InMemoryHistoryProvider, SlidingWindowStrategy +from agent_framework.openai import OpenAIChatClient +from agent_framework_azurefunctions import AgentFunctionApp +from azure.identity.aio import AzureCliCredential +from dotenv import load_dotenv + +load_dotenv() + +# Keep only the most recent turns in the model's context. Deliberately small so the +# effect is easy to observe in a short sample conversation. +KEEP_LAST_GROUPS = 4 + + +# 1. Instantiate the agent the ordinary core way - no durable-specific configuration. +def _create_agent() -> Any: + """Create the Historian agent.""" + # A plain in-memory history provider: the durable runtime replaces it with a + # durable-backed provider at registration, preserving this ``source_id`` so the + # compaction provider below stays wired to it. + history = InMemoryHistoryProvider(skip_excluded=True) + + compaction = CompactionProvider( + after_strategy=SlidingWindowStrategy(keep_last_groups=KEEP_LAST_GROUPS), + history_source_id=history.source_id, + ) + + return Agent( + client=OpenAIChatClient( + azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], + model=os.environ["AZURE_OPENAI_MODEL"], + credential=AzureCliCredential(), + ), + name="Historian", + instructions=( + "You are a concise assistant. Answer in one short sentence. " + "When the user tells you a fact, remember it and repeat it exactly when asked." + ), + # Keep the conversation client-side so the history provider (and therefore compaction) + # owns the model's context. + default_options={"store": False}, + context_providers=[history, compaction], + ) + + +# 2. Register the agent with AgentFunctionApp so Azure Functions exposes the required triggers. +# Pass prune_history=True here to also delete compacted-out messages from durable storage; +# that is lossy, so the full record is kept by default. +app = AgentFunctionApp(agents=[_create_agent()], enable_health_check=True, max_poll_retries=50) + +""" +Expected behavior when posting several turns with the same `session_id`: + +- every turn is answered with the earlier turns in context, +- the model's context stops growing once the sliding window fills, +- the durable entity keeps the whole conversation, with compacted-out messages marked excluded. +""" diff --git a/python/samples/azure_functions/14_conversation_compaction/host.json b/python/samples/azure_functions/14_conversation_compaction/host.json new file mode 100644 index 0000000..9e7fd87 --- /dev/null +++ b/python/samples/azure_functions/14_conversation_compaction/host.json @@ -0,0 +1,12 @@ +{ + "version": "2.0", + "extensionBundle": { + "id": "Microsoft.Azure.Functions.ExtensionBundle", + "version": "[4.*, 5.0.0)" + }, + "extensions": { + "durableTask": { + "hubName": "%TASKHUB_NAME%" + } + } +} diff --git a/python/samples/azure_functions/14_conversation_compaction/local.settings.json.template b/python/samples/azure_functions/14_conversation_compaction/local.settings.json.template new file mode 100644 index 0000000..5b65dd2 --- /dev/null +++ b/python/samples/azure_functions/14_conversation_compaction/local.settings.json.template @@ -0,0 +1,11 @@ +{ + "IsEncrypted": false, + "Values": { + "FUNCTIONS_WORKER_RUNTIME": "python", + "AzureWebJobsStorage": "UseDevelopmentStorage=true", + "DURABLE_TASK_SCHEDULER_CONNECTION_STRING": "Endpoint=http://localhost:8080;TaskHub=default;Authentication=None", + "TASKHUB_NAME": "default", + "AZURE_OPENAI_ENDPOINT": "", + "AZURE_OPENAI_MODEL": "" + } +} diff --git a/python/samples/azure_functions/14_conversation_compaction/requirements.txt b/python/samples/azure_functions/14_conversation_compaction/requirements.txt new file mode 100644 index 0000000..48738ea --- /dev/null +++ b/python/samples/azure_functions/14_conversation_compaction/requirements.txt @@ -0,0 +1,17 @@ +# Agent Framework packages +# To use the deployed version, uncomment the lines below and comment out the local installation lines +# agent-framework-openai +# agent-framework-azurefunctions + +# Local installation (for development and testing) +# Each package must be listed explicitly because pip doesn't resolve uv workspace sources. +# Without explicit entries, pip would fetch transitive dependencies from PyPI instead of local source. +agent-framework-openai>=1.10.1,<2 # Azure OpenAI support from PyPI (pulls in core) +-e ../../../packages/durabletask # Durable Task support - dependency of azurefunctions +-e ../../../packages/azurefunctions # Azure Functions integration - the main package for this sample + +# Azure authentication +azure-identity + +# Local environment loading +python-dotenv From 29198bd096b3a900933a7b326f06b19c1043c02b Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 21:07:25 -0500 Subject: [PATCH 12/32] feat: persist the agent session so provider state survives across turns Core documents the per-provider 'state' dict handed to before_run/after_run as durable for the life of the session and persists it through AgentSession.to_dict(). The entity built a fresh session per operation, so everything providers kept there was discarded at the end of every turn: tool approval rules and queued approval requests, todo lists, background-task state, memory extraction state. Nothing failed - agents just silently started over. That is a poor fit for a runtime whose headline scenario is long-running human-in-the-loop, where an approval flow that spans turns cannot work if the pending requests are dropped between them. The entity now persists the whole serialized session instead of individual fields, which also removes the hand-rolled serviceSessionId state field and its capture/restore helpers - that id is already part of AgentSession.to_dict(). The durable history provider's own slice is excluded, since it is derived from conversationHistory and would otherwise duplicate the transcript. Restore applies the stored state onto a session built by the agent's own create_session(), preserving its session type. Known limitation, recorded in the ADR: core's state type registry is process-local and only pre-registers Message, so to_dict-based values come back as plain data rather than their original class. Core's own state is mostly plain data and its tool-approval accessor takes either form, so this is latent; the fix belongs in core. --- .../0032-durable-thread-compaction.md | 38 +++++- .../agent_framework_durabletask/_constants.py | 4 +- .../_durable_agent_state.py | 20 +-- .../agent_framework_durabletask/_entities.py | 63 ++++++--- .../tests/test_durable_history_autoswap.py | 2 +- .../tests/test_durable_history_provider.py | 124 +++++++++++++++++- schemas/durable-agent-entity-state.json | 12 ++ 7 files changed, 229 insertions(+), 34 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index b801948..7b8dfc2 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -368,12 +368,46 @@ External providers key their storage on `session.session_id`, so a per-operation read and write a different key every turn - the conversation would silently restart each time with nothing to indicate a problem. +### The session is persisted, not just its conversation id + +Core documents the per-provider `state` dict handed to `before_run`/`after_run` as durable for the +life of the session, and persists it through `AgentSession.to_dict()`. The entity builds a fresh +session per operation, so anything providers keep there was previously discarded at the end of every +turn: tool approval rules and **queued approval requests**, todo lists, background-task state, memory +extraction state. On .NET the same bag (`AgentSessionStateBag`) is a first-class part of the +`AIContextProvider` contract via `StateKeys`, so the gap is wider there. + +That is a poor fit for a durable runtime whose headline scenario is long-running human-in-the-loop: +an approval flow that spans turns cannot work if the pending requests are dropped between them. + +So the entity persists the **whole serialized session** rather than individual fields. Two +consequences: + +- The service-issued conversation id needs no bespoke field of its own - it is already part of + `AgentSession.to_dict()`. This replaces a hand-rolled `serviceSessionId` state field and its + capture/restore helpers with one general mechanism that matches core's own serialization contract. +- The durable history provider's own slice is **excluded** before persisting. It is derived from + `conversationHistory` on every turn, so storing it would duplicate the transcript and let the copy + drift from the record of truth. + +Restore applies the stored state onto a session created by the agent's own `create_session()`, so +the agent's session type is preserved. + +**Known limitation.** Core's state type registry is process-local and, for `to_dict`-based types, is +only populated by an explicit `register_state_type()` call - of which core makes exactly one, for +`Message`. A durable entity routinely deserializes in a process that never serialized the value, so +such types come back as plain dicts rather than their original class. Core's own state is mostly +plain JSON data (and its tool-approval accessor tolerates both forms), so this is latent rather than +breaking, but a provider that assumes it gets its class back will not. The fix belongs in core: +pre-register the state types it ships. + ### Service-managed conversations When the model service stores the conversation, it identifies the thread with an id. The entity creates a fresh session per operation, so that id is **persisted in durable state and restored on -the next turn**; without it the service would start a new thread every turn. The durable history -provider additionally no-ops (neither loading nor flushing) for service-managed sessions. +the next turn** (as part of the serialized session, above); without it the service would start a new +thread every turn. The durable history provider additionally no-ops (neither loading nor flushing) +for service-managed sessions. Whether the service owns history is decided with **core's precedence, not the client class alone**: an explicit `store` in the agent's options wins, and only when it is unset does the client's diff --git a/python/packages/durabletask/agent_framework_durabletask/_constants.py b/python/packages/durabletask/agent_framework_durabletask/_constants.py index 03398a4..445ca60 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_constants.py +++ b/python/packages/durabletask/agent_framework_durabletask/_constants.py @@ -134,8 +134,8 @@ class DurableStateFields: # Stable per-message identity (used for compaction reconciliation and idempotency) MESSAGE_ID: Final[str] = "messageId" - # Service-issued conversation id, for agents whose provider stores history server-side - SERVICE_SESSION_ID: Final[str] = "serviceSessionId" + # Serialized AgentSession: the provider state bag plus any service-issued conversation id + SESSION: Final[str] = "session" class ContentTypes: diff --git a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py index cd19973..dbecc32 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py +++ b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py @@ -326,31 +326,33 @@ class DurableAgentStateData: Attributes: conversation_history: Ordered list of conversation entries (requests and responses) - service_session_id: Conversation id issued by a model service that stores history - server-side. Persisted so later turns continue the same service thread. + session: Serialized ``AgentSession`` from the previous turn - the context provider state + bag plus any service-issued conversation id. Core treats session state as durable + across turns, so it is persisted here rather than discarded with the per-operation + session. extension_data: Optional dictionary for custom metadata (not part of core schema) """ conversation_history: list[DurableAgentStateEntry] - service_session_id: str | None + session: dict[str, Any] | None extension_data: dict[str, Any] | None def __init__( self, conversation_history: list[DurableAgentStateEntry] | None = None, extension_data: dict[str, Any] | None = None, - service_session_id: str | None = None, + session: dict[str, Any] | None = None, ) -> None: """Initialize the data container. Args: conversation_history: Initial conversation history (defaults to empty list) extension_data: Optional custom metadata - service_session_id: Optional service-issued conversation id + session: Optional serialized ``AgentSession`` from the previous turn """ self.conversation_history = conversation_history or [] self.extension_data = extension_data - self.service_session_id = service_session_id + self.session = session def to_dict(self) -> dict[str, Any]: result: dict[str, Any] = { @@ -358,8 +360,8 @@ def to_dict(self) -> dict[str, Any]: } if self.extension_data is not None: result[DurableStateFields.EXTENSION_DATA] = self.extension_data - if self.service_session_id is not None: - result[DurableStateFields.SERVICE_SESSION_ID] = self.service_session_id + if self.session is not None: + result[DurableStateFields.SESSION] = self.session return result @classmethod @@ -367,7 +369,7 @@ def from_dict(cls, data_dict: dict[str, Any]) -> DurableAgentStateData: return cls( conversation_history=_parse_history_entries(data_dict), extension_data=data_dict.get(DurableStateFields.EXTENSION_DATA), - service_session_id=data_dict.get(DurableStateFields.SERVICE_SESSION_ID), + session=data_dict.get(DurableStateFields.SESSION), ) diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 9d8f421..cfd2f1a 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -14,6 +14,7 @@ from agent_framework import ( AgentResponse, AgentResponseUpdate, + AgentSession, Content, Message, ResponseStream, @@ -40,6 +41,10 @@ logger = logging.getLogger("agent_framework.durabletask") +# Keys produced by core's ``AgentSession.to_dict()``. +_SESSION_ID_KEY = "session_id" +_SESSION_STATE_KEY = "state" + class AgentEntityStateProviderMixin: """Mixin implementing durable agent state caching + (de)serialization + persistence. @@ -233,7 +238,7 @@ async def run( state_response = DurableAgentStateResponse.from_run_response(correlation_id, agent_run_response) self.state.data.conversation_history.append(state_response) - self._capture_service_session(session) + self._capture_session(session) self.persist_state() return agent_run_response @@ -268,18 +273,32 @@ def _has_context_pipeline(self) -> bool: """ return isinstance(getattr(self.agent, "context_providers", None), (list, tuple)) - def _capture_service_session(self, session: Any) -> None: - """Persist a service-issued conversation id so later turns continue the same thread. + def _capture_session(self, session: Any) -> None: + """Persist the session so provider state survives to the next turn. + + The entity creates a fresh session per operation, so anything the context providers keep + in the session state bag - tool approval rules and queued approval requests, todo lists, + memory extraction state - would otherwise be discarded at the end of every turn. Core + documents that state as durable for the life of the session, so agents that rely on it + must behave the same way here. The serialized session also carries the service-issued + conversation id, so service-backed agents continue the same thread. - Service-backed agents keep the conversation on the service side and identify it with an - id. The entity creates a fresh session per operation, so without persisting this the - service would start a new thread on every turn. + The durable history provider's own slice is dropped before persisting: it is derived from + ``conversation_history`` on every turn, so storing it would duplicate the transcript and + let the copy drift from the record of truth. """ if session is None: return - service_session_id = getattr(session, "service_session_id", None) - if isinstance(service_session_id, str) and service_session_id: - self.state.data.service_session_id = service_session_id + to_dict = getattr(session, "to_dict", None) + if not callable(to_dict): + return + + payload = cast("dict[str, Any]", to_dict()) + state = payload.get(_SESSION_STATE_KEY) + durable_history = self._find_durable_history_provider() + if isinstance(state, dict) and durable_history is not None: + cast("dict[str, Any]", state).pop(durable_history.source_id, None) + self.state.data.session = payload def _drop_already_stored(self, messages: list[DurableAgentStateMessage]) -> list[DurableAgentStateMessage]: """Filter out upstream context messages this entity has already recorded. @@ -314,15 +333,13 @@ def _find_durable_history_provider(self) -> DurableHistoryProvider | None: return None def _create_session(self) -> Any: - """Create the session for this operation. + """Create the session for this operation and restore what the last turn left on it. Conversation history lives in the agent's context providers (durable entity state, an external store, or the model service), so a fresh session per operation is enough - but it must carry the entity's **stable** session id. External history providers (Cosmos, Redis, file) key their storage on ``session.session_id``; with a freshly generated id they would - read and write a different key every turn and never see prior history. Any previously - issued service conversation id is restored so service-backed agents continue the same - thread. + read and write a different key every turn and never see prior history. """ create_session = getattr(self.agent, "create_session", None) if not callable(create_session): @@ -330,12 +347,24 @@ def _create_session(self) -> Any: f"Agent {type(self.agent).__name__} exposes context providers but does not support create_session()." ) session: Any = create_session(session_id=self._state_provider.session_id) - - service_session_id = self.state.data.service_session_id - if service_session_id and getattr(session, "service_session_id", None) is None: - session.service_session_id = service_session_id + self._restore_session(session) return session + def _restore_session(self, session: Any) -> None: + """Apply the previous turn's session state onto a freshly created session. + + The agent's own ``create_session`` is used so its session type is preserved; only the + state bag and the service conversation id are carried over. + """ + stored = self.state.data.session + if not stored or _SESSION_ID_KEY not in stored: + return + + restored = AgentSession.from_dict(dict(stored)) + session.state.update(restored.state) + if getattr(session, "service_session_id", None) is None: + session.service_session_id = restored.service_session_id + @staticmethod def _to_replayable_message(message: DurableAgentStateMessage) -> Message | None: """Convert persisted history into a message safe to replay into chat clients.""" diff --git a/python/packages/durabletask/tests/test_durable_history_autoswap.py b/python/packages/durabletask/tests/test_durable_history_autoswap.py index 21dffd2..67eb29c 100644 --- a/python/packages/durabletask/tests/test_durable_history_autoswap.py +++ b/python/packages/durabletask/tests/test_durable_history_autoswap.py @@ -294,4 +294,4 @@ async def run( assert seen_ids[0] is None # first turn has no thread yet assert seen_ids[1] == "svc-thread-1" # second turn continues the same thread - assert provider._get_state_dict()["data"]["serviceSessionId"] == "svc-thread-1" + assert provider._get_state_dict()["data"]["session"]["service_session_id"] == "svc-thread-1" diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py index 1845b02..a6e3a57 100644 --- a/python/packages/durabletask/tests/test_durable_history_provider.py +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -10,12 +10,15 @@ from collections.abc import AsyncIterable, Awaitable, Sequence from typing import Any +import pytest from agent_framework import ( Agent, + AgentSession, ChatResponse, ChatResponseUpdate, CompactionProvider, Content, + ContextProvider, HistoryProvider, InMemoryHistoryProvider, Message, @@ -167,7 +170,7 @@ class TestDurableHistoryProvider: """Durable entity state is the single store behind core's HistoryProvider.""" async def test_history_is_stored_once(self) -> None: - """No side-car session blob: messages live only in conversation history.""" + """Messages live only in conversation history, never duplicated into the session blob.""" client = RecordingChatClient() provider = _InMemoryStateProvider() entity = _make_entity(_build_agent(client), provider) @@ -175,8 +178,11 @@ async def test_history_is_stored_once(self) -> None: await _run_turns(entity, ["first", "second"]) persisted = provider._get_state_dict()["data"] - assert "sessionState" not in persisted - assert list(persisted.keys()) == ["conversationHistory"] + assert "conversationHistory" in persisted + # The session is persisted for provider state, but the history provider's slice - the + # only place messages would appear - is excluded from it. + session_state = persisted["session"]["state"] + assert not any("messages" in slice_ for slice_ in session_state.values() if isinstance(slice_, dict)) assert len(entity.state.data.conversation_history) == 4 async def test_provider_supplies_history_across_turns(self) -> None: @@ -381,3 +387,115 @@ async def test_external_provider_is_not_replaced(self) -> None: entity = _make_entity(agent, _InMemoryStateProvider()) assert entity.agent.context_providers[0] is external + + +class TestSessionStatePersistence: + """Provider state kept in the session bag survives across turns. + + Core documents the per-provider ``state`` dict as durable for the life of the session and + persists it through ``AgentSession.to_dict()``. The entity builds a fresh session per + operation, so it has to carry that state forward - otherwise providers silently start from + scratch every turn (tool approval rules and queued approval requests, todo lists, memory + extraction state). + """ + + async def test_provider_state_survives_across_turns(self) -> None: + seen: list[dict[str, Any]] = [] + + class _CountingProvider(ContextProvider): + def __init__(self) -> None: + super().__init__("counter") + + async def before_run(self, *, agent: Any, session: Any, context: Any, state: dict[str, Any]) -> None: + seen.append(dict(state)) + + async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict[str, Any]) -> None: + state["runs"] = state.get("runs", 0) + 1 + + agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[_CountingProvider()]) + entity = _make_entity(agent, _InMemoryStateProvider()) + + await _run_turns(entity, ["first", "second", "third"]) + + assert seen[0] == {} # nothing stored yet on the first turn + assert seen[1] == {"runs": 1} + assert seen[2] == {"runs": 2} + + async def test_state_is_persisted_as_plain_data(self) -> None: + """Values go through core's serialization, so entity state stays JSON-safe.""" + + class _StoringProvider(ContextProvider): + def __init__(self) -> None: + super().__init__("storer") + + async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict[str, Any]) -> None: + state.setdefault("note", Message(role="user", contents=["remember me"])) + + provider = _InMemoryStateProvider() + agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[_StoringProvider()]) + await _run_turns(_make_entity(agent, provider), ["first"]) + + session_payload = provider._get_state_dict()["data"]["session"] + assert isinstance(session_payload["state"]["storer"]["note"], dict) + # ...and comes back as a Message, because core pre-registers that type. + restored = AgentSession.from_dict(dict(session_payload)) + assert isinstance(restored.state["storer"]["note"], Message) + + async def test_service_conversation_id_rides_along(self) -> None: + """It is part of the serialized session, so it needs no field of its own.""" + provider = _InMemoryStateProvider() + agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[InMemoryHistoryProvider()]) + entity = _make_entity(agent, provider) + + await _run_turns(entity, ["first"]) + assert "service_session_id" in provider._get_state_dict()["data"]["session"] + + async def test_tool_approval_state_survives_a_turn(self) -> None: + """The motivating case: standing approvals must outlive the turn that granted them. + + Also pins the known limitation - core only pre-registers ``Message`` in its state type + registry, and that registry is populated per process, so a ``to_dict``-based value comes + back as plain data rather than its original class. The data survives, which is what the + approval middleware needs (its accessor takes either form), but the type does not. + """ + # The harness is experimental; skip rather than fail if it moves. + tool_approval = pytest.importorskip("agent_framework._harness._tool_approval") + ToolApprovalRule = tool_approval.ToolApprovalRule + ToolApprovalState = tool_approval.ToolApprovalState + + seen: list[Any] = [] + approval_key = "_tool_approval" + + class _ApprovalCarryingProvider(ContextProvider): + def __init__(self) -> None: + super().__init__("approvals") + + async def before_run(self, *, agent: Any, session: Any, context: Any, state: dict[str, Any]) -> None: + seen.append(session.state.get(approval_key)) + + async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict[str, Any]) -> None: + session.state.setdefault( + approval_key, + ToolApprovalState(rules=[ToolApprovalRule("delete_file")]), + ) + + agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[_ApprovalCarryingProvider()]) + await _run_turns(_make_entity(agent, _InMemoryStateProvider()), ["first", "second"]) + + assert seen[0] is None # nothing granted yet + restored = seen[1] + assert restored is not None, "the approval granted on turn 1 was lost" + rules = restored["rules"] if isinstance(restored, dict) else restored.rules + assert rules[0]["tool_name"] == "delete_file" + + async def test_durable_history_slice_is_not_persisted(self) -> None: + """That slice is derived from conversation_history; storing it would duplicate it.""" + provider = _InMemoryStateProvider() + agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[InMemoryHistoryProvider()]) + entity = _make_entity(agent, provider) + + await _run_turns(entity, ["first", "second"]) + + durable_history = next(p for p in entity.agent.context_providers if isinstance(p, DurableHistoryProvider)) + session_state = provider._get_state_dict()["data"]["session"]["state"] + assert durable_history.source_id not in session_state diff --git a/schemas/durable-agent-entity-state.json b/schemas/durable-agent-entity-state.json index 53ac064..1f5e081 100644 --- a/schemas/durable-agent-entity-state.json +++ b/schemas/durable-agent-entity-state.json @@ -200,6 +200,18 @@ "type": "array", "description": "Ordered list of conversation entries.", "items": { "$ref": "#/$defs/conversationEntry" } + }, + "session": { + "type": "object", + "description": "Serialized agent session carried between turns: the per-provider state bag and any service-issued conversation id. The agent's own history provider slice is excluded, since conversationHistory is the record of truth.", + "properties": { + "session_id": { "type": "string" }, + "service_session_id": { "type": ["string", "null"] }, + "state": { + "type": "object", + "description": "Provider state keyed by context provider source id." + } + } } } } From 7e7a8219941ba6d016e930e568e09fc366410598 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 21:43:29 -0500 Subject: [PATCH 13/32] fix: restore session state values as their own types after a cold start Core deserializes session state through a type registry it seeds with exactly one entry (Message); anything else must be registered explicitly, and the registry is process-local. to_dict-based types are never auto-registered - only Pydantic models are, and only as a side effect of serializing. A durable entity routinely restores in a process that never serialized the value, so provider state came back as plain dicts instead of its own classes. Before restoring, the entity now registers the serializable types already loaded in the process. Nothing is imported from persisted data, so this cannot load code the application has not already loaded itself, and that is sufficient in practice: whoever put a value in the state bag had to import its class to construct it. The walk covers SerializationMixin subclasses and costs tens of microseconds. Pydantic values in state remain uncovered (they are keyed by class name and walking every BaseModel subclass would be broad and collision-prone). Core seeding the registry with the types it ships would make this unnecessary - register_state_type() is already public and documents cold-start restore as its motivating case. --- .../0032-durable-thread-compaction.md | 26 +++++++--- .../agent_framework_durabletask/_entities.py | 50 +++++++++++++++++++ .../tests/test_durable_history_provider.py | 12 ++--- 3 files changed, 74 insertions(+), 14 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index 7b8dfc2..c8a296a 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -393,13 +393,25 @@ consequences: Restore applies the stored state onto a session created by the agent's own `create_session()`, so the agent's session type is preserved. -**Known limitation.** Core's state type registry is process-local and, for `to_dict`-based types, is -only populated by an explicit `register_state_type()` call - of which core makes exactly one, for -`Message`. A durable entity routinely deserializes in a process that never serialized the value, so -such types come back as plain dicts rather than their original class. Core's own state is mostly -plain JSON data (and its tool-approval accessor tolerates both forms), so this is latent rather than -breaking, but a provider that assumes it gets its class back will not. The fix belongs in core: -pre-register the state types it ships. +**Restoring values as their own types.** Core deserializes state through a type registry that it +seeds with exactly one entry (`Message`); anything else must be registered explicitly, and the +registry is process-local. `to_dict`-based types are never auto-registered - only Pydantic models +are, and only as a side effect of serializing. A durable entity routinely restores in a process that +never serialized the value, so state would come back as plain dicts instead of its own classes. + +Before restoring, the entity therefore registers the serializable types **already loaded in the +process**. Nothing is imported from persisted data, so this cannot load code the application has not +already loaded itself - and that is sufficient in practice, because whoever put a value in the state +bag had to import its class to construct it. The walk is over `SerializationMixin` subclasses and +costs tens of microseconds. + +Residual gaps, both better fixed in core: + +- Pydantic values in state are keyed by `cls.__name__.lower()` and are not covered, since walking + every `BaseModel` subclass in the process would be broad and collision-prone. +- Core could seed the registry with the state types it ships, which would make this unnecessary. + `register_state_type()` is already public and its documentation names cold-start restore as the + motivating case; nothing calls it today. ### Service-managed conversations diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index cfd2f1a..427f574 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -19,6 +19,7 @@ Message, ResponseStream, SupportsAgentRun, + register_state_type, ) from durabletask.entities import DurableEntity @@ -45,6 +46,51 @@ _SESSION_ID_KEY = "session_id" _SESSION_STATE_KEY = "state" +try: + # Root of core's serializable state types. Not part of core's public surface, so a move must + # not break the entity: without it, restored provider state simply stays as plain dicts, + # which is core's own behavior. + from agent_framework._serialization import SerializationMixin + + _SerializableStateRoot: type | None = SerializationMixin +except ImportError: # pragma: no cover - depends on the installed core version + _SerializableStateRoot = None + +_registered_state_types: set[type] = set() + + +def _register_loaded_state_types() -> None: + """Let core restore session state values as their own classes after a cold start. + + Core deserializes session state through a type registry that it seeds with exactly one entry + (``Message``); anything else must be registered explicitly, and the registry is process-local. + A durable entity routinely restores state in a process that never serialized it, so without + this a provider's state comes back as a plain dict rather than its own class. + + Only classes already imported in this process are registered - nothing is imported from + persisted data - so this cannot load code the application has not already loaded itself. That + is enough in practice, because whoever put a value in the state bag had to import its class to + construct it. + """ + if _SerializableStateRoot is None: + return + + seen: set[type] = set() + pending: list[type] = [_SerializableStateRoot] + while pending: + for subclass in pending.pop().__subclasses__(): + if subclass in seen: + continue + seen.add(subclass) + pending.append(subclass) + if subclass in _registered_state_types: + continue + _registered_state_types.add(subclass) + try: + register_state_type(subclass) + except Exception: + logger.debug("Could not register session state type %s", subclass, exc_info=True) + class AgentEntityStateProviderMixin: """Mixin implementing durable agent state caching + (de)serialization + persistence. @@ -360,6 +406,10 @@ def _restore_session(self, session: Any) -> None: if not stored or _SESSION_ID_KEY not in stored: return + # Done here rather than at import: by now the agent and its providers are built, so the + # classes their state uses are loaded and can be resolved. + _register_loaded_state_types() + restored = AgentSession.from_dict(dict(stored)) session.state.update(restored.state) if getattr(session, "service_session_id", None) is None: diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py index a6e3a57..6530035 100644 --- a/python/packages/durabletask/tests/test_durable_history_provider.py +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -453,10 +453,9 @@ async def test_service_conversation_id_rides_along(self) -> None: async def test_tool_approval_state_survives_a_turn(self) -> None: """The motivating case: standing approvals must outlive the turn that granted them. - Also pins the known limitation - core only pre-registers ``Message`` in its state type - registry, and that registry is populated per process, so a ``to_dict``-based value comes - back as plain data rather than its original class. The data survives, which is what the - approval middleware needs (its accessor takes either form), but the type does not. + It also comes back as ``ToolApprovalState`` rather than a plain dict. Core seeds its state + type registry with only ``Message``, so the entity registers the serializable types loaded + in this process before restoring. """ # The harness is experimental; skip rather than fail if it moves. tool_approval = pytest.importorskip("agent_framework._harness._tool_approval") @@ -484,9 +483,8 @@ async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict assert seen[0] is None # nothing granted yet restored = seen[1] - assert restored is not None, "the approval granted on turn 1 was lost" - rules = restored["rules"] if isinstance(restored, dict) else restored.rules - assert rules[0]["tool_name"] == "delete_file" + assert isinstance(restored, ToolApprovalState), f"approval state came back as {type(restored).__name__}" + assert restored.rules[0].tool_name == "delete_file" async def test_durable_history_slice_is_not_persisted(self) -> None: """That slice is derived from conversation_history; storing it would duplicate it.""" From 0775b589d1c9339226c1fd1356e01a87fea1d016 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 22:34:15 -0500 Subject: [PATCH 14/32] docs: sharpen ADR 0032's account of the store-side compaction gap The gaps section claimed compaction 'bypasses the provider', which overstates it and would not survive review. Only one of CompactionProvider's two hooks is coupled to session state: before_strategy acts on the loaded invocation context and already works for every provider, so external stores do get in-run context bounding. What they do not get is the framework rewriting their store. Whether that is a defect depends on who owns the store - not rewriting a user's Cosmos container is defensible, but durable entity state is framework-owned, which is what makes it a real problem here rather than a reasonable omission. It is also unresolved rather than decided: ADR-0019 names three compaction points, scopes in Redis and Cosmos, and leaves the mechanism as an explicit open question that shipped unanswered. The languages then diverged - .NET put store reduction on the provider (IChatReducer, InMemory only; Cosmos has none), Python put it in CompactionProvider reaching into session state - and neither offers it to external providers. Also corrects the knock-on claims elsewhere in the ADR that both core hooks apply 'unchanged', since L2 in fact carries workaround code, and cross-references the two gaps recorded in other sections. --- .../0032-durable-thread-compaction.md | 132 ++++++++++++------ 1 file changed, 90 insertions(+), 42 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index c8a296a..06514c1 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -41,9 +41,14 @@ Core MAF already has a compaction system ([ADR-0019](https://github.com/microsof 1. **In-run filter** — a `CompactionProvider` (`AIContextProvider`) / `compaction_strategy` runs before each model call. It is **non-lossy**: it filters the projection sent to the model and stores incremental group state in the `AgentSession.StateBag`; the underlying store is untouched. -2. **Store reducer** — an `IChatReducer` on a `ChatHistoryProvider` (e.g. `InMemoryChatHistoryProvider`) - **lossily** rewrites the stored conversation. `strategy.AsChatReducer()` bridges any core strategy - into this hook, so it is the **same strategies** applied at the store instead of the model call. + This hook works with **any** history provider, since it acts on the messages already loaded into + the invocation context. +2. **Store reducer** — **lossily** rewrites the stored conversation, applying the same strategies at + the store instead of at the model call. Unlike the in-run filter, this hook is tied to a specific + storage mechanism in both languages: .NET exposes an `IChatReducer` on `InMemoryChatHistoryProvider` + only (bridged from any strategy by `strategy.AsChatReducer()`), and Python's + `CompactionProvider.after_strategy` reads the messages out of session state. Neither offers it to + a provider backed by anything else - see "Core Interface Gaps" below. The durable layer benefits from **neither** today, because `AgentEntity` **bypasses the `ChatHistoryProvider`**: it creates a fresh session per operation (so the StateBag — and any history @@ -95,10 +100,10 @@ bounded when the user opts into it?** automatically derive a lossy store reducer (`strategy.AsChatReducer()`) so durable storage is bounded even without an explicit reducer. - **Option 6 — Durable store as a `ChatHistoryProvider` (chosen).** Back the durable entity's - persisted conversation with a core `ChatHistoryProvider` implementation, so **both** core hooks - apply on the durable runtime unchanged: the in-run filter runs in the agent pipeline (L1), and a - user-configured `IChatReducer` bounds the store (L2, opt-in). The same seam makes external storage - backends (Cosmos, Valkey, blob) pluggable for capacity. + persisted conversation with a core `ChatHistoryProvider` implementation, so both core hooks apply + on the durable runtime from the user's unchanged configuration: the in-run filter runs in the + agent pipeline (L1), and a user-configured reducer/strategy bounds the store (L2, opt-in). The + same seam makes external storage backends (Cosmos, Valkey, blob) pluggable for capacity. ## Decision Outcome @@ -112,7 +117,7 @@ Compaction applies at **three layers**, mapped directly onto the core hooks: | Layer | Core mechanism reused | Lossy? | Role | | --- | --- | --- | --- | | **L1 — in-run filter** | `CompactionProvider` in the agent pipeline | No | Always-on. Bounds the **model input** (context window, token cost). Identical to core. | -| **L2 — store reducer** | `IChatReducer` on the durable `ChatHistoryProvider` | Yes | **Opt-in.** Bounds the **persisted store**, only when the user configures a reducer. Identical to core. | +| **L2 — store reducer** | the store-rewrite hook applied to the durable provider's store | Yes | **Opt-in.** Bounds the **persisted store**, only when the user configures a reducer/strategy. Same strategies as core, but the hook is bound to session state upstream, so this layer needs a workaround - see "Core Interface Gaps". | | **L3 — workflow hook** | the same strategy as the `AgentExecutor` `context_filter` | Yes | Bounds the inter-executor `full_conversation`. | **Two accumulation surfaces:** @@ -123,7 +128,7 @@ Compaction applies at **three layers**, mapped directly onto the core hooks: | **Inter-executor (workflow)** | `AgentExecutor.full_conversation`, checkpointed as envelopes | L3 | **Why Option 6 over bespoke entity compaction (Option 2).** Making the durable store a -`ChatHistoryProvider` means L2 is core's existing `IChatReducer` path — not new compaction code — +`ChatHistoryProvider` means L2 reuses core's strategies rather than introducing new compaction code, and the same abstraction is the seam for **external storage backends** (Cosmos/Valkey/blob) that relieve capacity. One abstraction delivers both the opt-in reducer and pluggable storage, all reused from core. @@ -154,14 +159,16 @@ conversation, the client holds no history to compact. - Good: **configuration parity** — the same core strategies/hooks apply on the durable runtime with no changes; the model input is bounded identically to core. -- Good: **no reinvention** — L2 is core's `IChatReducer` path; the `ChatHistoryProvider` seam also - makes external storage backends pluggable for capacity. +- Good: **no reinvention** — L2 reuses core's strategies rather than a durable-only compaction API; + the history-provider seam also makes external storage backends pluggable for capacity. - Good: **no silent data loss** — the durable record is only reduced when the user opts into a reducer; capacity limits surface explicitly. - Good: durable workflows inherit L1+L2; L3 reuses the existing `context_filter` seam. - Neutral: making the durable store a `ChatHistoryProvider` is a larger change to the entity than a bespoke compaction pass would be, and must preserve the existing `ConversationHistory` consumer contract (`AgentRunHandle` response polling, audit/replay, TTL). +- Bad: L2 carries workaround code, because upstream binds the store-rewrite hook to session state + rather than to the provider; that code can be deleted if the gap is closed upstream. - Bad: an opt-in LLM-based reducer runs inside the entity operation and re-runs on retry; mitigated by stable summary identity and (optionally) Option 3 to move heavy summarization off the request path. @@ -169,7 +176,7 @@ conversation, the client holds no history to compact. ### Validation - **Unit tests (both languages):** a core `CompactionProvider` on a durable agent bounds the model - input; a configured `IChatReducer` bounds the persisted store; with no reducer the store is not + input; a configured reducer/strategy bounds the persisted store; with no reducer the store is not silently truncated; atomic groups preserved; reducer idempotent across simulated entity retries; service-managed sessions skipped. - **Integration tests:** the same agent config produces equivalent compaction behavior in core and @@ -218,12 +225,14 @@ conversation, the client holds no history to compact. ### Option 6 — Durable store as a `ChatHistoryProvider` (chosen) -- Good, because **both** core hooks apply unchanged: L1 filter in the pipeline, L2 reducer on the - store — full configuration parity. +- Good, because the user's configuration carries over unchanged: L1 applies exactly as in core, and + L2 uses the same strategies rather than a durable-only API. - Good, because the same abstraction makes external storage backends (Cosmos/Valkey/blob) pluggable, relieving capacity without touching compaction. - Good, because it is core reuse rather than durable-specific compaction code. - Neutral, because L2 is opt-in — a store is only reduced when the user configures a reducer. +- Bad, because L2 does **not** come for free: upstream binds the store-rewrite hook to session state, + so the provider has to publish a working buffer and reconcile it itself (see "Core Interface Gaps"). - Bad, because it is a larger entity change and must preserve the `ConversationHistory` consumer contract (response polling, audit, TTL). @@ -231,12 +240,12 @@ conversation, the client holds no history to compact. - **Configuration parity (discovery over new API).** The durable runtime honors the compaction the user already configured on the agent — the `CompactionProvider` in the pipeline (L1) and any - `IChatReducer` on the history provider (L2). A durable-specific option exists at most as an - optional override, never as the required path. Moving core → durable entity → durable workflow - requires no reconfiguration. + store-side reducer/strategy attached to the history provider (L2). A durable-specific option + exists at most as an optional override, never as the required path. Moving core → durable entity → + durable workflow requires no reconfiguration. - **Two hooks, mapped.** In-run filter (`CompactionProvider`) → L1, non-lossy, bounds the model - input. Store reducer (`IChatReducer` on the durable `ChatHistoryProvider`) → L2, lossy, opt-in, - bounds the persisted store. Both accept the same `CompactionStrategy` (via `strategy.AsChatReducer()`). + input. Store reducer applied to the durable provider's store → L2, lossy, opt-in, bounds the + persisted store. Both accept the same `CompactionStrategy` (on .NET via `strategy.AsChatReducer()`). - **Reducer trigger.** Honor the configured `ReducerTriggerEvent`; `AfterMessageAdded` (compact-on-write, before checkpoint) is the natural durable default so the checkpoint is already bounded. `BeforeMessagesRetrieval` also works (reduce-on-load, then persist). @@ -258,31 +267,69 @@ conversation, the client holds no history to compact. ## Core Interface Gaps for Pluggable History Providers -Prototyping the Python `DurableHistoryProvider` surfaced three places where the current contracts -assume a *session-state-backed* history provider. They are recorded here because they affect **any** -external provider (Cosmos, Valkey, durable), not just this one. The prototype works around them; the -cleaner fix is upstream. - -1. **Compaction bypasses the provider.** `CompactionProvider.after_run` reads stored messages - directly from `session.state[history_source_id]["messages"]` rather than asking the provider. - A provider whose store is *not* session state therefore gets no post-run compaction - L2 silently - no-ops. *Workaround:* the provider publishes its loaded messages as a working buffer under that - key. *Upstream fix:* have compaction request messages from the history provider. - -2. **`save_messages()` is append-only.** It receives only the newly produced messages, so mutations - that compaction applies to *already stored* messages (setting `_excluded`, inserting a summary) - have no defined path back to the store. *Workaround (implemented):* the provider overrides - `after_run` and reconciles the working buffer itself **by `message_id`**, updating annotations on - known messages and inserting ones compaction added. This required persisting `messageId` in - durable state, which also gives summaries the **stable identity** the idempotency requirement - needs. *Upstream fix:* add an explicit replace/flush operation alongside append so every external - provider does not have to re-implement this reconciliation. +Prototyping the Python `DurableHistoryProvider` surfaced places where the current contracts assume a +*session-state-backed* history provider. They are recorded here because they affect **any** provider +whose store is not session state (Cosmos, Valkey, durable), not just this one. The prototype works +around them; the cleaner fix is upstream. + +1. **Store-side compaction is bound to session state rather than to the provider.** `CompactionProvider` + has two hooks and only one of them is coupled: + + - `before_strategy` runs on messages already in the invocation context, whichever provider loaded + them. Every provider gets this, so **in-run context bounding already works for external stores**. + - `after_strategy` is documented as operating on "the accumulated messages stored by a history + provider in session state", and "requires `history_source_id` to locate the messages in session + state". It reads `session.state[history_source_id]["messages"]` and mutates that list in place, + treating mutation as persistence - which only holds when the store *is* session state. + + So the missing capability is narrower than it first appears: an external provider can bound what + the model sees, but cannot have the framework rewrite its store. + + Whether that is a defect depends on **who owns the store**. For a user-owned store (Cosmos, Redis) + the framework arguably *should not* rewrite it implicitly. For a framework-owned store (in-memory, + and durable entity state) rewriting is squarely in scope. Durable is the first framework-owned + store that is not session state, which is what turns this from a defensible omission into a real + problem. + + It is also unresolved rather than decided. ADR-0019 names three compaction points (in-run, + pre-write, on existing storage), explicitly scopes in "local storage (e.g. `InMemoryHistoryProvider`, + Redis, Cosmos)", and then leaves the mechanism open: + + > Should pre-write and existing-storage compaction share one unified configuration/setup to reduce + > duplicate strategy wiring, and then either: each write overrides the full storage, or only new + > messages are compacted while a separate interface can be called to compact the existing storage? + + That question shipped unanswered, and the languages then diverged on where the hook lives: .NET + puts store reduction on the provider (`IChatReducer`) but only on `InMemoryChatHistoryProvider` + (`CosmosChatHistoryProvider` has none); Python puts it in `CompactionProvider` reaching into + session state. **Neither language offers it to external providers.** + + *Workaround:* the provider publishes its loaded messages as a working buffer under the expected + session-state key. *Upstream fix:* bind the store-rewrite hook to the provider abstraction instead + of to session state as a storage mechanism - .NET's shape generalizes, Python's does not. + +2. **`save_messages()` is append-only.** The other half of the same open question. It receives only + the newly produced messages, so mutations that compaction applies to *already stored* messages + (setting `_excluded`, inserting a summary) have no defined path back to the store. + *Workaround (implemented):* the provider overrides `after_run` and reconciles the working buffer + itself **by `message_id`**, updating annotations on known messages and inserting ones compaction + added. This required persisting `messageId` in durable state, which also gives summaries the + **stable identity** the idempotency requirement needs. *Upstream fix:* add an explicit + replace/flush operation alongside append so every external provider does not have to re-implement + this reconciliation. 3. **Message-level metadata was not persisted (durable schema).** `DurableAgentStateMessage.to_dict()` dropped `extension_data` while `from_dict()` read it - a write-lossy asymmetry that silently discarded compaction annotations on every state round-trip. Since annotations are what carry - compaction state, this had to be fixed for any of this to work. The Python side now serializes it; - **.NET and the shared state schema need the same treatment** for cross-language parity. + compaction state, this had to be fixed for any of this to work. This one is ours rather than + core's. The Python side now serializes it; **.NET and the shared state schema need the same + treatment** for cross-language parity, or compaction will appear to do nothing there for exactly + the same reason. + +Two further core gaps are recorded with the decisions they affect: the process-local **state type +registry** (see "The session is persisted, not just its conversation id") and the absence of a public +way to ask whether **the service owns history for a run** (see "Service-managed conversations"). Both +forced this layer to re-implement logic core already has. Consequence for ordering: core runs `before_run` forward and `after_run` in **reverse**. With `[history, compaction]`, compaction annotates the buffer *before* the history provider flushes it @@ -360,8 +407,9 @@ re-sent history the service already had. **Consequence:** passing a session is what re-engages the pipeline, so external history providers (Cosmos, Redis, file) now function under the durable runtime - previously they were silently -ignored because no session was ever created. Store-side compaction still no-ops for them (core -interface gap 1 below); only the in-run filter applies. +ignored because no session was ever created. They get the in-run filter like any other provider; +what they do not get is the framework rewriting their store, which no language offers today (core +interface gap 1 above). That session must also carry the entity's **stable** session id rather than a generated one. External providers key their storage on `session.session_id`, so a per-operation id would make them From cc74aff4c0409e71f8e94a429315ecb4557907e6 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 23:17:11 -0500 Subject: [PATCH 15/32] docs: complete the record of deferred core gaps in ADR 0032 These gaps are being followed up rather than fixed, so the ADR has to be the durable record. Three were under-captured: - The per-service-call cadence split was only ever discussed, never written down. Added as gap 4: history providers move to per-model-call while CompactionProvider stays per-run, so compaction annotates after the last flush. Latent (HarnessAgent only), but the symptom would be missing annotations rather than an error. - The .NET parity note said 'add extension data', which is misleading. .NET already has an ExtensionData property, but it is [JsonExtensionData] - the JSON overflow bucket, not a mapping of ChatMessage.AdditionalProperties. Annotations are lost at the conversion boundary, and MessageId does not exist at all. Anyone auditing for 'is extension data persisted?' would see the property and wrongly close the item. - Recorded that the store-precedence rule is re-derived here because core does not expose it, that drift would present as silent conversation loss, and that the only real net is the compaction sample rather than the unit tests. --- .../0032-durable-thread-compaction.md | 28 +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index 06514c1..128a8f8 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -322,9 +322,25 @@ around them; the cleaner fix is upstream. dropped `extension_data` while `from_dict()` read it - a write-lossy asymmetry that silently discarded compaction annotations on every state round-trip. Since annotations are what carry compaction state, this had to be fixed for any of this to work. This one is ours rather than - core's. The Python side now serializes it; **.NET and the shared state schema need the same - treatment** for cross-language parity, or compaction will appear to do nothing there for exactly - the same reason. + core's. The Python side now serializes it. + + **.NET needs the same treatment, and looks deceptively fine.** Its `DurableAgentStateMessage` + already has an `ExtensionData` property, but it is `[JsonExtensionData]` - System.Text.Json's + overflow bucket for *unmapped JSON properties*, not a mapping of `ChatMessage.AdditionalProperties` + where compaction annotations live. `FromChatMessage`/`ToChatMessage` copy neither + `AdditionalProperties` nor `MessageId` (which .NET does not have at all), so annotations are lost + at the **conversion** boundary rather than the JSON one. Anyone checking for "is extension data + persisted?" will see the property and wrongly conclude parity is done. + +4. **Provider cadence splits under per-service-call persistence.** With + `require_per_service_call_history_persistence=True`, the agent's once-per-run loop skips history + providers because the per-service-call middleware drives `before_run`/`after_run` itself - once per + **model call** instead of once per run. `CompactionProvider` is not a `HistoryProvider`, so it + stays on the once-per-run path. The pair is therefore split across two cadences, and compaction + annotates the buffer *after* the history provider last flushed it, so annotations would not reach + storage until the following flush. Only `HarnessAgent` sets this flag today, so this is latent + rather than live; it is recorded because the symptom would be missing annotations rather than an + error. Two further core gaps are recorded with the decisions they affect: the process-local **state type registry** (see "The session is persisted, not just its conversation id") and the absence of a public @@ -476,6 +492,12 @@ API) are routinely put back into client-side mode with `store=False`. Consulting `STORES_BY_DEFAULT` would leave such an agent with a plain in-memory provider that the durable runtime never persists - silently losing the conversation between turns. +Core resolves this rule inside `Agent._run` and does not expose the result, so this layer +**re-derives it** and can drift from core if the rule changes - with silent conversation loss as the +symptom, which is exactly the bug this rule was written to fix. The unit tests here only pin *our* +logic; the end-to-end net is the compaction sample, which runs `store=False` against a +store-by-default client and asserts recall. *Upstream fix:* expose the resolved decision. + ### Retention is a deployment policy, not agent configuration Compaction annotates; it does not delete. Physically deleting excluded messages bounds durable From 36b7fdad8b609acfac9915435f515fd4f5c87799 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 30 Jul 2026 23:42:56 -0500 Subject: [PATCH 16/32] docs: tighten ADR 0032 for coherence and length The ADR had grown into two documents in one hat: a forward-looking design decision in the present tense, followed by a retrospective implementation log, with no signal where one ended and the other began. Adds a short orientation note, marks the status accepted (Python implemented, .NET pending), and fixes the Context section's claim that the durable layer benefits from neither hook 'today' - no longer true. Also notes once that .NET's ChatHistoryProvider and Python's HistoryProvider are the same concept, since the decision sections use one name and the implementation sections the other. Deduplication: service-managed scope was stated four times and storage-capacity-is-separate five; each now has one home plus pointers. The per-option pros/cons lists restated Decision Outcome almost verbatim and are now one entry per option. Three Cross-Cutting bullets that repeated the drivers and the L1/L2 table are gone, as is the 'Why Option 6 over Option 2' paragraph now covered by the options summary. Validation was written as intent; it now separates what is actually covered in Python from what is still outstanding, so the .NET gap is visible rather than implied. 549 -> 504 lines, 5267 -> 4877 words, with no information removed. --- .../0032-durable-thread-compaction.md | 200 +++++++----------- 1 file changed, 78 insertions(+), 122 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index 128a8f8..7989fea 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -1,6 +1,6 @@ --- # These are optional elements. Feel free to remove any of them. -status: proposed +status: accepted contact: ahmedmuhsin date: 2026-07-27 deciders: ahmedmuhsin @@ -10,6 +10,13 @@ informed: # Thread Compaction for Durable Agents and Workflows +> **How to read this.** Everything through "Pros and Cons of the Options" is the design decision. +> Everything after it records how that decision was realized in Python and what the realization +> surfaced. **.NET is not implemented yet.** +> +> **Naming.** .NET's `ChatHistoryProvider` and Python's `HistoryProvider` are the same concept. The +> decision sections use the .NET name; the implementation sections use the Python one. + ## Context and Problem Statement Long-running **durable** agents and workflows accumulate conversation history in durable @@ -27,13 +34,12 @@ It helps to separate **three distinct pressures**, because they have different o | **Token cost / latency** — resending history each turn | tokens billed / round-trip | **Yes** — same mechanism | Compaction (in-run filter) | | **Storage capacity** — the cumulative persisted state | backend state-size limit | **No** — durable-only | Storage backend (built-in limit or external store) | -The first two are **per-operation** (what a single turn sends to the model) and are **identical in -core and durable** — the model's context window is the same regardless of runtime. The third is -**cumulative across all runs**: `ConversationHistory` is a single blob appended to every turn and -re-persisted whole, so it is bounded by the durable backend's state-size limit (backend-specific; -e.g. classic Azure Storage ~1 MB/entity), whereas a core process is bounded only by RAM and resets -on restart. **Storage capacity is an infrastructure concern, not a context-window concern** — it is -relieved by raising the limit or moving to an external store, not by trimming what the model sees. +The first two are per-operation and identical in both runtimes. The third is cumulative: +`ConversationHistory` is one blob appended to every turn and re-persisted whole, so it is bounded by +the backend's state-size limit (e.g. classic Azure Storage ~1 MB/entity), whereas a core process is +bounded only by RAM and resets on restart. **Storage capacity is an infrastructure concern, not a +context-window concern** - relieved by raising the limit or moving to an external store, not by +trimming what the model sees. Core MAF already has a compaction system ([ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md); .NET `Microsoft.Agents.AI.Compaction`; Python `agent_framework._compaction`) with **two hooks**: @@ -50,11 +56,10 @@ Core MAF already has a compaction system ([ADR-0019](https://github.com/microsof `CompactionProvider.after_strategy` reads the messages out of session state. Neither offers it to a provider backed by anything else - see "Core Interface Gaps" below. -The durable layer benefits from **neither** today, because `AgentEntity` **bypasses the -`ChatHistoryProvider`**: it creates a fresh session per operation (so the StateBag — and any history -provider store or reducer in it — is discarded) and feeds `ConversationHistory` directly as input -messages. So both the in-run filter's incremental state and the store reducer are thrown away each -turn. +The durable layer benefited from **neither**, because `AgentEntity` **bypassed the history +provider**: it created a fresh session per operation (so the StateBag - and any history provider +store or reducer in it - was discarded) and fed `ConversationHistory` directly as input messages. +Both the in-run filter's incremental state and the store reducer were thrown away every turn. The goal is **configuration parity**: a user's core compaction config must carry over to a durable entity or workflow **unchanged**, reusing the same strategies and hooks on the durable runtime, @@ -127,24 +132,15 @@ Compaction applies at **three layers**, mapped directly onto the core hooks: | **In-agent** | the agent's model input, and the persisted `AgentEntity` store | L1 (filter) + L2 (reducer, opt-in) | | **Inter-executor (workflow)** | `AgentExecutor.full_conversation`, checkpointed as envelopes | L3 | -**Why Option 6 over bespoke entity compaction (Option 2).** Making the durable store a -`ChatHistoryProvider` means L2 reuses core's strategies rather than introducing new compaction code, -and the same abstraction is the seam for **external storage backends** (Cosmos/Valkey/blob) that -relieve capacity. One abstraction delivers both the opt-in reducer and pluggable storage, all -reused from core. - -**Strict parity — no auto-derive (Option 5 rejected).** Durable honors exactly the hooks the user -configured. If only an in-run filter is configured, durable trims the model input just like core -and the store still grows — because the context window (which compaction addresses) is identical in -both runtimes, and storage capacity is a separate concern. Auto-deriving a lossy reducer would use a -context-window tool to solve a storage problem and **silently destroy the durable record**, breaking -both the "no data loss" driver and parity. Storage capacity is instead addressed by the backend: -the built-in store enforces a limit (surface a clear error/warning as it is approached), and an -external `ChatHistoryProvider` raises the ceiling for those who need unbounded durable records. - -**Ideal durable default:** keep the full record in a (possibly external) durable `ChatHistoryProvider` -and apply the L1 in-run filter to the model input — never lose the record, always bound what the -model sees. A lossy L2 reducer is a deliberate opt-in, not a durable surprise. +**Strict parity - no auto-derive (Option 5 rejected).** Durable honors exactly the hooks the user +configured. If only an in-run filter is configured, durable trims the model input just like core and +the store still grows - the context window is identical in both runtimes, and storage capacity is a +separate concern. Auto-deriving a lossy reducer would use a context-window tool to solve a storage +problem and **silently destroy the durable record**. Capacity is addressed by the backend instead: +the built-in store enforces a limit (surfacing a clear error as it is approached), and an external +provider raises the ceiling. The ideal durable default is therefore the full record in a (possibly +external) provider plus the L1 filter on the model input - never lose the record, always bound what +the model sees; a lossy L2 reducer stays a deliberate opt-in. **Why workflows largely come "for free."** Durable workflow agent execution (`DurableExecutorDispatcher.ExecuteAgentAsync`) runs an agent through the same @@ -152,107 +148,71 @@ model sees. A lossy L2 reducer is a deliberate opt-in, not a durable surprise. inherited by workflow agent executors**. The workflow's own `full_conversation` between executors does not pass through the agent, so it needs the separate **L3** hook. -**Service-managed storage** remains out of scope (mirrors ADR-0019): when the service owns the -conversation, the client holds no history to compact. +**Service-managed storage** is out of scope, mirroring ADR-0019: when the service owns the +conversation the client holds no history to compact. See "Service-managed conversations" for how the +runtime detects and handles it. ### Consequences -- Good: **configuration parity** — the same core strategies/hooks apply on the durable runtime with - no changes; the model input is bounded identically to core. -- Good: **no reinvention** — L2 reuses core's strategies rather than a durable-only compaction API; - the history-provider seam also makes external storage backends pluggable for capacity. -- Good: **no silent data loss** — the durable record is only reduced when the user opts into a - reducer; capacity limits surface explicitly. -- Good: durable workflows inherit L1+L2; L3 reuses the existing `context_filter` seam. -- Neutral: making the durable store a `ChatHistoryProvider` is a larger change to the entity than a - bespoke compaction pass would be, and must preserve the existing `ConversationHistory` consumer - contract (`AgentRunHandle` response polling, audit/replay, TTL). -- Bad: L2 carries workaround code, because upstream binds the store-rewrite hook to session state - rather than to the provider; that code can be deleted if the gap is closed upstream. -- Bad: an opt-in LLM-based reducer runs inside the entity operation and re-runs on retry; mitigated - by stable summary identity and (optionally) Option 3 to move heavy summarization off the request +- Good: **configuration parity** - the same core strategies and hooks apply on the durable runtime + with no changes; durable workflows inherit L1+L2, and L3 reuses the existing `context_filter` seam. +- Good: **no silent data loss** - the durable record is only reduced when the user opts into a + reducer; capacity limits surface explicitly rather than truncating. +- Neutral: a larger entity change than a bespoke compaction pass, and it must preserve the existing + `ConversationHistory` consumer contract (`AgentRunHandle` response polling, audit/replay, TTL). +- Bad: L2 carries workaround code because upstream binds the store-rewrite hook to session state; + deletable if that gap closes. +- Bad: an opt-in LLM-based reducer runs inside the entity operation and re-runs on retry - mitigated + by stable summary identity, and optionally by Option 3 to move heavy summarization off the request path. ### Validation -- **Unit tests (both languages):** a core `CompactionProvider` on a durable agent bounds the model - input; a configured reducer/strategy bounds the persisted store; with no reducer the store is not - silently truncated; atomic groups preserved; reducer idempotent across simulated entity retries; - service-managed sessions skipped. -- **Integration tests:** the same agent config produces equivalent compaction behavior in core and - durable; a fan-out/chained durable workflow keeps `full_conversation` bounded via L3; an external - `ChatHistoryProvider` stores history beyond the built-in limit. - -## Pros and Cons of the Options - -### Option 1 — In-run filter only - -- Good, because it is the existing core feature with (almost) no new code, and bounds the model - input within a run (including long tool loops). -- Good, because it applies to workflow agent executors too (shared agent path). -- Neutral, because it is non-lossy — by design it does not bound the persisted store. -- Bad, because the persisted `ConversationHistory` still grows and its incremental StateBag is - discarded each operation (recomputed every turn), so on its own it does not address storage. - -### Option 2 — Bespoke pre-write compaction in the agent entity - -- Good, because it directly bounds persisted state and can reuse the static `CompactAsync`. -- Neutral, because it requires a `DurableAgentStateMessage` ⇄ `ChatMessage` conversion. -- Bad, because it is **new durable-specific code** that duplicates what core's `IChatReducer` path - already does, and it does not give external-storage pluggability. +**Done (Python).** Unit tests cover the provider substitution rules, compaction annotations +surviving a state round-trip, summary insertion, pruning, service-managed skip, session-state +persistence, and workflow context projection. Integration tests run against a real scheduler and +assert that annotations and message ids survive entity serialization, that an external provider +keeps a whole conversation under one key, and that a downstream workflow agent can reference the +upstream conversation. -### Option 3 — On-storage maintenance compaction +**Outstanding.** The .NET realization and its schema parity (gap 3); an external history provider +storing history beyond the built-in state-size limit; idempotency of an LLM-based reducer across +simulated entity retries. -- Good, because it keeps expensive summarization off the request/response path and maps to the - "on existing storage" point from ADR-0019. -- Neutral, because it can layer on top of Option 6 later without rework. -- Bad, because it adds scheduling/trigger machinery and a window where state is temporarily - un-compacted; on its own it does not bound in-turn growth. - -### Option 4 — Workflow-level compaction hook - -- Good, because it bounds the inter-executor `full_conversation` that agent-level compaction never - sees, reusing the existing `context_filter` seam. -- Neutral, because it is only relevant to multi-agent workflows. -- Bad, because a naive filter could break atomic groups if it does not reuse the core grouping. - -### Option 5 — Auto-derive a durable store reducer - -- Good, because it would bound durable storage automatically even for in-run-filter-only configs. -- Bad, because it **conflates storage with context management** — using a lossy tool to solve a - capacity problem — and **silently truncates the durable record**, breaking parity and the - no-data-loss driver. Rejected. - -### Option 6 — Durable store as a `ChatHistoryProvider` (chosen) +## Pros and Cons of the Options -- Good, because the user's configuration carries over unchanged: L1 applies exactly as in core, and - L2 uses the same strategies rather than a durable-only API. -- Good, because the same abstraction makes external storage backends (Cosmos/Valkey/blob) pluggable, - relieving capacity without touching compaction. -- Good, because it is core reuse rather than durable-specific compaction code. -- Neutral, because L2 is opt-in — a store is only reduced when the user configures a reducer. -- Bad, because L2 does **not** come for free: upstream binds the store-rewrite hook to session state, - so the provider has to publish a working buffer and reconcile it itself (see "Core Interface Gaps"). -- Bad, because it is a larger entity change and must preserve the `ConversationHistory` consumer - contract (response polling, audit, TTL). +The full argument is in **Decision Outcome** above; this is the summary. + +- **Option 1 - In-run filter only.** Existing core feature, almost no new code, bounds the model + input including long tool loops, and applies to workflow agent executors too. But it is non-lossy + by design, so the persisted store keeps growing and the filter's incremental state is discarded + and recomputed every turn. +- **Option 2 - Bespoke pre-write compaction in the entity.** Directly bounds persisted state, but is + new durable-only code duplicating what core's store-reducer path already does, needs a + `DurableAgentStateMessage` ⇄ message conversion, and gives no external-storage pluggability. +- **Option 3 - On-storage maintenance compaction.** Keeps expensive summarization off the request + path and maps to ADR-0019's "on existing storage" point; can layer on top of Option 6 later + without rework. Adds scheduling machinery, leaves a window where state is un-compacted, and does + not bound in-turn growth. +- **Option 4 - Workflow-level hook.** Bounds the inter-executor `full_conversation` that agent-level + compaction never sees, reusing the existing `context_filter` seam. Only relevant to multi-agent + workflows, and must reuse core grouping or a naive filter breaks atomic groups. **Adopted + alongside Option 6 as L3.** +- **Option 5 - Auto-derive a store reducer.** Would bound durable storage automatically even for + filter-only configs, but conflates storage with context management and **silently truncates the + durable record**, breaking parity and the no-data-loss driver. **Rejected.** +- **Option 6 - Durable store as a history provider (chosen).** The user's configuration carries over + unchanged, and the same abstraction makes external backends pluggable, so one seam delivers both + the opt-in reducer and pluggable storage. Costs a larger entity change that must preserve the + `ConversationHistory` consumer contract (response polling, audit, TTL); and L2 does not come free - + upstream binds the store-rewrite hook to session state, so the provider publishes a working buffer + and reconciles it itself (see "Core Interface Gaps"). ## Cross-Cutting Design Details -- **Configuration parity (discovery over new API).** The durable runtime honors the compaction the - user already configured on the agent — the `CompactionProvider` in the pipeline (L1) and any - store-side reducer/strategy attached to the history provider (L2). A durable-specific option - exists at most as an optional override, never as the required path. Moving core → durable entity → - durable workflow requires no reconfiguration. -- **Two hooks, mapped.** In-run filter (`CompactionProvider`) → L1, non-lossy, bounds the model - input. Store reducer applied to the durable provider's store → L2, lossy, opt-in, bounds the - persisted store. Both accept the same `CompactionStrategy` (on .NET via `strategy.AsChatReducer()`). - **Reducer trigger.** Honor the configured `ReducerTriggerEvent`; `AfterMessageAdded` (compact-on-write, before checkpoint) is the natural durable default so the checkpoint is already bounded. `BeforeMessagesRetrieval` also works (reduce-on-load, then persist). -- **Storage capacity is separate.** The built-in entity store is bounded by the backend state-size - limit; approaching it should surface a clear error/warning, not silent truncation. An external - `ChatHistoryProvider` (Cosmos/Valkey/blob) raises the ceiling for unbounded durable records and - is enabled by the same Option 6 seam. - **Determinism & idempotency.** An opt-in lossy reducer runs inside the entity operation and re-runs on retry. Give any generated summary a **stable identity** (derived from the ids of the messages it replaces) so retries do not re-summarize or duplicate. Reduced content becomes @@ -262,8 +222,8 @@ conversation, the client holds no history to compact. pairings are preserved at every layer. - **Token counting.** Triggers must work without a live model call; use the estimator tokenizer (`CharacterEstimatorTokenizer` / equivalent) unless a real tokenizer is supplied. -- **Placement.** The durable `ChatHistoryProvider` backs `AgentEntity` (.NET) / `AgentEntity` in - `_entities.py` (Python). L3 lives in the `AgentExecutor` context handling in both languages. +- **Placement.** The durable history provider backs `AgentEntity` in both languages. L3 lives in the + `AgentExecutor` context handling. ## Core Interface Gaps for Pluggable History Providers @@ -374,10 +334,6 @@ Behavior difference that remains, by design: each agent node also keeps its **ow (keyed by workflow instance + executor), so per-agent memory survives restarts and is compacted independently - a superset of the in-process behavior rather than a strict match. -**Service-managed sessions** are a no-op at every layer: when a session carries a -`service_session_id` the model service owns the conversation, so the durable history provider -neither loads nor flushes. - ## Zero-Configuration Registration The parity goal is only met if a user can take an agent that **already works in core**, register it From 23f9aaa7054326efc48a652f4628a3d4ab85871a Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 31 Jul 2026 00:02:57 -0500 Subject: [PATCH 17/32] docs: drop em dashes and prose semicolons from this branch's content Punctuation pass over the material added on this branch: the ADR, the three new sample READMEs, and the docstrings, comments and log messages in the new provider, entity and sample code. Em dashes become commas, parentheses or sentence breaks, and semicolons joining independent clauses become separate sentences. Colons are kept only where they label something (Args, Returns, 'Chosen option', 'Workaround') rather than standing in for a conjunction. Pre-existing text is left alone, so the em dashes still in _models.py, _workflows/context.py, _workflows/orchestrator.py, tests/test_app.py and samples/README.md are untouched - none of those lines are from this branch, and rewriting them would add unrelated churn. Also fixes an indentation slip introduced while editing a comment in _history_provider.py. --- .../0032-durable-thread-compaction.md | 227 +++++++++--------- .../agent_framework_durabletask/_entities.py | 6 +- .../_history_provider.py | 16 +- .../13_conversation_compaction/README.md | 8 +- .../14_external_history_redis/README.md | 8 +- .../redis_history_provider.py | 4 +- .../14_conversation_compaction/README.md | 14 +- .../function_app.py | 4 +- 8 files changed, 145 insertions(+), 142 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index 7989fea..d20dddd 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -15,43 +15,43 @@ informed: > surfaced. **.NET is not implemented yet.** > > **Naming.** .NET's `ChatHistoryProvider` and Python's `HistoryProvider` are the same concept. The -> decision sections use the .NET name; the implementation sections use the Python one. +> decision sections use the .NET name, and the implementation sections use the Python one. ## Context and Problem Statement Long-running **durable** agents and workflows accumulate conversation history in durable storage and replay it on every turn. Durable agents persist a full `ConversationHistory` in -entity state (`AgentEntity` → `DurableAgentState`); durable workflows persist inter-executor -messages (`AgentExecutor.full_conversation`) as checkpointed envelopes. Unlike an in-memory agent -— whose history lives in process RAM (gigabytes) and disappears when the process recycles — this -history is **persisted, reloaded every turn, and permanent**. +entity state (`AgentEntity` → `DurableAgentState`). Durable workflows persist inter-executor +messages (`AgentExecutor.full_conversation`) as checkpointed envelopes. An in-memory agent keeps its +history in process RAM, where it disappears when the process recycles. This history is instead +**persisted, reloaded every turn, and permanent**. -It helps to separate **three distinct pressures**, because they have different owners: +It helps to separate **three distinct pressures**, because they have different owners. | Pressure | What bounds it | Same in core? | Owner | | --- | --- | --- | --- | -| **Context window** — the model's max input per call | the model | **Yes** — identical in core and durable | Compaction (in-run filter) | -| **Token cost / latency** — resending history each turn | tokens billed / round-trip | **Yes** — same mechanism | Compaction (in-run filter) | -| **Storage capacity** — the cumulative persisted state | backend state-size limit | **No** — durable-only | Storage backend (built-in limit or external store) | +| **Context window**, the model's max input per call | the model | **Yes**, identical in core and durable | Compaction (in-run filter) | +| **Token cost / latency**, resending history each turn | tokens billed / round-trip | **Yes**, same mechanism | Compaction (in-run filter) | +| **Storage capacity**, the cumulative persisted state | backend state-size limit | **No**, durable-only | Storage backend (built-in limit or external store) | -The first two are per-operation and identical in both runtimes. The third is cumulative: +The first two are per-operation and identical in both runtimes. The third is cumulative. `ConversationHistory` is one blob appended to every turn and re-persisted whole, so it is bounded by the backend's state-size limit (e.g. classic Azure Storage ~1 MB/entity), whereas a core process is bounded only by RAM and resets on restart. **Storage capacity is an infrastructure concern, not a -context-window concern** - relieved by raising the limit or moving to an external store, not by +context-window concern**, relieved by raising the limit or moving to an external store, not by trimming what the model sees. -Core MAF already has a compaction system ([ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md); -.NET `Microsoft.Agents.AI.Compaction`; Python `agent_framework._compaction`) with **two hooks**: +Core MAF already has a compaction system ([ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md), +.NET `Microsoft.Agents.AI.Compaction`, Python `agent_framework._compaction`) with **two hooks**. -1. **In-run filter** — a `CompactionProvider` (`AIContextProvider`) / `compaction_strategy` runs - before each model call. It is **non-lossy**: it filters the projection sent to the model and - stores incremental group state in the `AgentSession.StateBag`; the underlying store is untouched. - This hook works with **any** history provider, since it acts on the messages already loaded into - the invocation context. -2. **Store reducer** — **lossily** rewrites the stored conversation, applying the same strategies at +1. **In-run filter.** A `CompactionProvider` (`AIContextProvider`) / `compaction_strategy` runs + before each model call. It is **non-lossy**. It filters the projection sent to the model and + stores incremental group state in the `AgentSession.StateBag`, leaving the underlying store + untouched. This hook works with **any** history provider, since it acts on the messages already + loaded into the invocation context. +2. **Store reducer.** **Lossily** rewrites the stored conversation, applying the same strategies at the store instead of at the model call. Unlike the in-run filter, this hook is tied to a specific - storage mechanism in both languages: .NET exposes an `IChatReducer` on `InMemoryChatHistoryProvider` + storage mechanism in both languages. .NET exposes an `IChatReducer` on `InMemoryChatHistoryProvider` only (bridged from any strategy by `strategy.AsChatReducer()`), and Python's `CompactionProvider.after_strategy` reads the messages out of session state. Neither offers it to a provider backed by anything else - see "Core Interface Gaps" below. @@ -71,61 +71,61 @@ bounded when the user opts into it?** ## Decision Drivers -- **Configuration parity** — the same core compaction config (strategies, `CompactionProvider`, +- **Configuration parity.** The same core compaction config (strategies, `CompactionProvider`, `IChatReducer`) must apply unchanged when moving core → durable entity → durable workflow. No parallel durable-only API. -- **Reuse existing core hooks** — do not reinvent triggers/strategies/grouping; reuse the in-run +- **Reuse existing core hooks.** Do not reinvent triggers, strategies or grouping. Reuse the in-run filter and the store reducer. -- **Separate storage capacity from context management** — bound the model input with compaction - (parity with core); relieve persisted-storage capacity with infrastructure (backend limits / - external stores), not by silently trimming. -- **No silent data loss in the durable record** — a durable system of record must not quietly - truncate history; lossy reduction is explicit opt-in, and hard capacity limits should surface a - clear error/warning. -- **Determinism / idempotency** — durable entity operations can be retried; a lossy reducer +- **Separate storage capacity from context management.** Bound the model input with compaction + (parity with core), and relieve persisted-storage capacity with infrastructure (backend limits, + external stores) rather than by silently trimming. +- **No silent data loss in the durable record.** A durable system of record must not quietly + truncate history. Lossy reduction is explicit opt-in, and hard capacity limits should surface a + clear error or warning. +- **Determinism and idempotency.** Durable entity operations can be retried, so a lossy reducer (especially LLM summarization) must not corrupt or diverge persisted state across retries. -- **Message-list correctness** — preserve atomic groups (assistant tool-call + tool-result, and +- **Message-list correctness.** Preserve atomic groups (assistant tool-call plus tool-result, and reasoning pairings) so the model input stays valid. -- **Cover both surfaces** — durable agents **and** durable workflows, in **both** languages. -- **No-op for service-managed storage** — when the service owns the conversation (a - `ConversationId`/`service_session_id` is set), the client has no history to compact. +- **Cover both surfaces.** Durable agents **and** durable workflows, in **both** languages. +- **No-op for service-managed storage.** When the service owns the conversation (a + `ConversationId` or `service_session_id` is set), the client has no history to compact. ## Considered Options -- **Option 1 — In-run filter only.** Register the core `CompactionProvider` / `compaction_strategy` - on the inner agent; change nothing else in the durable layer. -- **Option 2 — Bespoke pre-write compaction in the agent entity.** Add durable-specific code that +- **Option 1, in-run filter only.** Register the core `CompactionProvider` / `compaction_strategy` + on the inner agent and change nothing else in the durable layer. +- **Option 2, bespoke pre-write compaction in the agent entity.** Add durable-specific code that compacts `ConversationHistory` inside the entity operation before checkpoint. -- **Option 3 — On-storage maintenance compaction.** Compact persisted history from a separate - entity signal/operation, decoupled from the request path. -- **Option 4 — Workflow-level compaction hook.** Apply a strategy at the `AgentExecutor` +- **Option 3, on-storage maintenance compaction.** Compact persisted history from a separate + entity signal or operation, decoupled from the request path. +- **Option 4, workflow-level compaction hook.** Apply a strategy at the `AgentExecutor` `context_mode` / `context_filter` boundary that governs the `full_conversation` chained between agent executors. -- **Option 5 — Auto-derive a durable store reducer.** When only an in-run filter is configured, +- **Option 5, auto-derive a durable store reducer.** When only an in-run filter is configured, automatically derive a lossy store reducer (`strategy.AsChatReducer()`) so durable storage is bounded even without an explicit reducer. -- **Option 6 — Durable store as a `ChatHistoryProvider` (chosen).** Back the durable entity's +- **Option 6, durable store as a `ChatHistoryProvider` (chosen).** Back the durable entity's persisted conversation with a core `ChatHistoryProvider` implementation, so both core hooks apply - on the durable runtime from the user's unchanged configuration: the in-run filter runs in the - agent pipeline (L1), and a user-configured reducer/strategy bounds the store (L2, opt-in). The + on the durable runtime from the user's unchanged configuration. The in-run filter runs in the + agent pipeline (L1), and a user-configured reducer or strategy bounds the store (L2, opt-in). The same seam makes external storage backends (Cosmos, Valkey, blob) pluggable for capacity. ## Decision Outcome -Chosen option: **Option 6 — express durable conversation storage as a core `ChatHistoryProvider`**, +Chosen option: **Option 6, express durable conversation storage as a core `ChatHistoryProvider`**, combined with the workflow hook (Option 4). This makes core's two compaction hooks apply on the durable runtime with **no config change**, and cleanly separates context management from storage capacity. -Compaction applies at **three layers**, mapped directly onto the core hooks: +Compaction applies at **three layers**, mapped directly onto the core hooks. | Layer | Core mechanism reused | Lossy? | Role | | --- | --- | --- | --- | -| **L1 — in-run filter** | `CompactionProvider` in the agent pipeline | No | Always-on. Bounds the **model input** (context window, token cost). Identical to core. | -| **L2 — store reducer** | the store-rewrite hook applied to the durable provider's store | Yes | **Opt-in.** Bounds the **persisted store**, only when the user configures a reducer/strategy. Same strategies as core, but the hook is bound to session state upstream, so this layer needs a workaround - see "Core Interface Gaps". | -| **L3 — workflow hook** | the same strategy as the `AgentExecutor` `context_filter` | Yes | Bounds the inter-executor `full_conversation`. | +| **L1, in-run filter** | `CompactionProvider` in the agent pipeline | No | Always-on. Bounds the **model input** (context window, token cost). Identical to core. | +| **L2, store reducer** | the store-rewrite hook applied to the durable provider's store | Yes | **Opt-in.** Bounds the **persisted store**, only when the user configures a reducer or strategy. Same strategies as core, but the hook is bound to session state upstream, so this layer needs a workaround (see "Core Interface Gaps"). | +| **L3, workflow hook** | the same strategy as the `AgentExecutor` `context_filter` | Yes | Bounds the inter-executor `full_conversation`. | -**Two accumulation surfaces:** +**Two accumulation surfaces.** | Surface | Where it accumulates | Covered by | | --- | --- | --- | @@ -139,8 +139,8 @@ separate concern. Auto-deriving a lossy reducer would use a context-window tool problem and **silently destroy the durable record**. Capacity is addressed by the backend instead: the built-in store enforces a limit (surfacing a clear error as it is approached), and an external provider raises the ceiling. The ideal durable default is therefore the full record in a (possibly -external) provider plus the L1 filter on the model input - never lose the record, always bound what -the model sees; a lossy L2 reducer stays a deliberate opt-in. +external) provider plus the L1 filter on the model input, never losing the record and always +bounding what the model sees. A lossy L2 reducer stays a deliberate opt-in. **Why workflows largely come "for free."** Durable workflow agent execution (`DurableExecutorDispatcher.ExecuteAgentAsync`) runs an agent through the same @@ -148,22 +148,23 @@ the model sees; a lossy L2 reducer stays a deliberate opt-in. inherited by workflow agent executors**. The workflow's own `full_conversation` between executors does not pass through the agent, so it needs the separate **L3** hook. -**Service-managed storage** is out of scope, mirroring ADR-0019: when the service owns the +**Service-managed storage** is out of scope, mirroring ADR-0019. When the service owns the conversation the client holds no history to compact. See "Service-managed conversations" for how the runtime detects and handles it. ### Consequences -- Good: **configuration parity** - the same core strategies and hooks apply on the durable runtime - with no changes; durable workflows inherit L1+L2, and L3 reuses the existing `context_filter` seam. -- Good: **no silent data loss** - the durable record is only reduced when the user opts into a - reducer; capacity limits surface explicitly rather than truncating. +- Good: **configuration parity**, since the same core strategies and hooks apply on the durable + runtime with no changes. Durable workflows inherit L1+L2, and L3 reuses the existing + `context_filter` seam. +- Good: **no silent data loss**, since the durable record is only reduced when the user opts into a + reducer. Capacity limits surface explicitly rather than truncating. - Neutral: a larger entity change than a bespoke compaction pass, and it must preserve the existing `ConversationHistory` consumer contract (`AgentRunHandle` response polling, audit/replay, TTL). -- Bad: L2 carries workaround code because upstream binds the store-rewrite hook to session state; - deletable if that gap closes. -- Bad: an opt-in LLM-based reducer runs inside the entity operation and re-runs on retry - mitigated - by stable summary identity, and optionally by Option 3 to move heavy summarization off the request +- Bad: L2 carries workaround code because upstream binds the store-rewrite hook to session state. + That code is deletable if the gap closes. +- Bad: an opt-in LLM-based reducer runs inside the entity operation and re-runs on retry, mitigated + by stable summary identity and optionally by Option 3 to move heavy summarization off the request path. ### Validation @@ -175,13 +176,15 @@ assert that annotations and message ids survive entity serialization, that an ex keeps a whole conversation under one key, and that a downstream workflow agent can reference the upstream conversation. -**Outstanding.** The .NET realization and its schema parity (gap 3); an external history provider -storing history beyond the built-in state-size limit; idempotency of an LLM-based reducer across -simulated entity retries. +**Outstanding.** Three things are not covered yet. + +- The .NET realization and its schema parity (gap 3). +- An external history provider storing history beyond the built-in state-size limit. +- Idempotency of an LLM-based reducer across simulated entity retries. ## Pros and Cons of the Options -The full argument is in **Decision Outcome** above; this is the summary. +The full argument is in **Decision Outcome** above. This is the summary. - **Option 1 - In-run filter only.** Existing core feature, almost no new code, bounds the model input including long tool loops, and applies to workflow agent executors too. But it is non-lossy @@ -191,7 +194,7 @@ The full argument is in **Decision Outcome** above; this is the summary. new durable-only code duplicating what core's store-reducer path already does, needs a `DurableAgentStateMessage` ⇄ message conversion, and gives no external-storage pluggability. - **Option 3 - On-storage maintenance compaction.** Keeps expensive summarization off the request - path and maps to ADR-0019's "on existing storage" point; can layer on top of Option 6 later + path and maps to ADR-0019's "on existing storage" point, and can layer on top of Option 6 later without rework. Adds scheduling machinery, leaves a window where state is un-compacted, and does not bound in-turn growth. - **Option 4 - Workflow-level hook.** Bounds the inter-executor `full_conversation` that agent-level @@ -204,13 +207,13 @@ The full argument is in **Decision Outcome** above; this is the summary. - **Option 6 - Durable store as a history provider (chosen).** The user's configuration carries over unchanged, and the same abstraction makes external backends pluggable, so one seam delivers both the opt-in reducer and pluggable storage. Costs a larger entity change that must preserve the - `ConversationHistory` consumer contract (response polling, audit, TTL); and L2 does not come free - - upstream binds the store-rewrite hook to session state, so the provider publishes a working buffer - and reconciles it itself (see "Core Interface Gaps"). + `ConversationHistory` consumer contract (response polling, audit, TTL). L2 also does not come free, + because upstream binds the store-rewrite hook to session state, so the provider publishes a working + buffer and reconciles it itself (see "Core Interface Gaps"). ## Cross-Cutting Design Details -- **Reducer trigger.** Honor the configured `ReducerTriggerEvent`; `AfterMessageAdded` +- **Reducer trigger.** Honor the configured `ReducerTriggerEvent`. `AfterMessageAdded` (compact-on-write, before checkpoint) is the natural durable default so the checkpoint is already bounded. `BeforeMessagesRetrieval` also works (reduce-on-load, then persist). - **Determinism & idempotency.** An opt-in lossy reducer runs inside the entity operation and @@ -220,7 +223,7 @@ The full argument is in **Decision Outcome** above; this is the summary. `ChatReducerCompactionStrategy` / `SummarizationCompactionStrategy`). - **Message-list correctness.** Reuse core grouping so atomic tool-call/result and reasoning pairings are preserved at every layer. -- **Token counting.** Triggers must work without a live model call; use the estimator tokenizer +- **Token counting.** Triggers must work without a live model call, so use the estimator tokenizer (`CharacterEstimatorTokenizer` / equivalent) unless a real tokenizer is supplied. - **Placement.** The durable history provider backs `AgentEntity` in both languages. L3 lives in the `AgentExecutor` context handling. @@ -230,10 +233,10 @@ The full argument is in **Decision Outcome** above; this is the summary. Prototyping the Python `DurableHistoryProvider` surfaced places where the current contracts assume a *session-state-backed* history provider. They are recorded here because they affect **any** provider whose store is not session state (Cosmos, Valkey, durable), not just this one. The prototype works -around them; the cleaner fix is upstream. +around them, but the cleaner fix is upstream. 1. **Store-side compaction is bound to session state rather than to the provider.** `CompactionProvider` - has two hooks and only one of them is coupled: + has two hooks and only one of them is coupled. - `before_strategy` runs on messages already in the invocation context, whichever provider loaded them. Every provider gets this, so **in-run context bounding already works for external stores**. @@ -259,14 +262,14 @@ around them; the cleaner fix is upstream. > duplicate strategy wiring, and then either: each write overrides the full storage, or only new > messages are compacted while a separate interface can be called to compact the existing storage? - That question shipped unanswered, and the languages then diverged on where the hook lives: .NET - puts store reduction on the provider (`IChatReducer`) but only on `InMemoryChatHistoryProvider` - (`CosmosChatHistoryProvider` has none); Python puts it in `CompactionProvider` reaching into + That question shipped unanswered, and the languages then diverged on where the hook lives. .NET + puts store reduction on the provider (`IChatReducer`) but only on `InMemoryChatHistoryProvider`, + and `CosmosChatHistoryProvider` has none. Python puts it in `CompactionProvider` reaching into session state. **Neither language offers it to external providers.** *Workaround:* the provider publishes its loaded messages as a working buffer under the expected session-state key. *Upstream fix:* bind the store-rewrite hook to the provider abstraction instead - of to session state as a storage mechanism - .NET's shape generalizes, Python's does not. + of to session state as a storage mechanism, since .NET's shape generalizes and Python's does not. 2. **`save_messages()` is append-only.** The other half of the same open question. It receives only the newly produced messages, so mutations that compaction applies to *already stored* messages @@ -279,13 +282,13 @@ around them; the cleaner fix is upstream. this reconciliation. 3. **Message-level metadata was not persisted (durable schema).** `DurableAgentStateMessage.to_dict()` - dropped `extension_data` while `from_dict()` read it - a write-lossy asymmetry that silently + dropped `extension_data` while `from_dict()` read it, a write-lossy asymmetry that silently discarded compaction annotations on every state round-trip. Since annotations are what carry compaction state, this had to be fixed for any of this to work. This one is ours rather than core's. The Python side now serializes it. **.NET needs the same treatment, and looks deceptively fine.** Its `DurableAgentStateMessage` - already has an `ExtensionData` property, but it is `[JsonExtensionData]` - System.Text.Json's + already has an `ExtensionData` property, but it is `[JsonExtensionData]`, System.Text.Json's overflow bucket for *unmapped JSON properties*, not a mapping of `ChatMessage.AdditionalProperties` where compaction annotations live. `FromChatMessage`/`ToChatMessage` copy neither `AdditionalProperties` nor `MessageId` (which .NET does not have at all), so annotations are lost @@ -294,12 +297,12 @@ around them; the cleaner fix is upstream. 4. **Provider cadence splits under per-service-call persistence.** With `require_per_service_call_history_persistence=True`, the agent's once-per-run loop skips history - providers because the per-service-call middleware drives `before_run`/`after_run` itself - once per + providers because the per-service-call middleware drives `before_run`/`after_run` itself, once per **model call** instead of once per run. `CompactionProvider` is not a `HistoryProvider`, so it stays on the once-per-run path. The pair is therefore split across two cadences, and compaction annotates the buffer *after* the history provider last flushed it, so annotations would not reach storage until the following flush. Only `HarnessAgent` sets this flag today, so this is latent - rather than live; it is recorded because the symptom would be missing annotations rather than an + rather than live. It is recorded because the symptom would be missing annotations rather than an error. Two further core gaps are recorded with the decisions they affect: the process-local **state type @@ -348,16 +351,16 @@ used, so the caller's agent still behaves normally in-process. | User configured | Durable behavior | | --- | --- | -| Nothing | Inject a durable history provider, using the `source_id` core's auto-injected provider would have - so default-wired compaction still resolves. No compaction by default (same as core). | +| Nothing | Inject a durable history provider, using the `source_id` core's auto-injected provider would have, so default-wired compaction still resolves. No compaction by default (same as core). | | `InMemoryHistoryProvider` (± compaction) | Replace with the durable provider, **preserving `source_id` and `skip_excluded`** so any attached `CompactionProvider` keeps working untouched. | -| Cosmos / Redis / file / custom provider | **Leave alone.** The user chose where their conversation lives; durable still supplies execution durability. | -| Service-managed history | **Leave alone.** The model service owns the conversation. Decided by core's precedence: explicit `store` first, then the client's `STORES_BY_DEFAULT`. | +| Cosmos / Redis / file / custom provider | **Leave alone.** The user chose where their conversation lives, and durable still supplies execution durability. | +| Service-managed history | **Leave alone.** The model service owns the conversation. Decided by core's precedence, explicit `store` first and then the client's `STORES_BY_DEFAULT`. | | Agent without the core context pipeline | **Leave alone.** Falls back to replaying persisted history. | -Preserving `source_id` is the load-bearing detail: `CompactionProvider` locates history through +Preserving `source_id` is the load-bearing detail. `CompactionProvider` locates history through `history_source_id` (default `"in_memory"`), so a provider swapped in under the same id is invisible to the rest of the user's configuration. Because the injected provider is a `HistoryProvider` with -`load_messages=True`, core's own auto-injection sees a provider present and stands down - no +`load_messages=True`, core's own auto-injection sees a provider present and stands down, leaving no duplicate provider. An explicit `DurableHistoryProvider` remains supported as an advanced escape hatch, and takes @@ -365,22 +368,22 @@ precedence over anything the runtime would inject. ### When the entity manages history itself -Two distinct decisions drive the entity, and conflating them caused bugs: +Two distinct decisions drive the entity, and conflating them caused bugs. 1. **Who supplies conversation context?** If the agent exposes core's context-provider pipeline, - the providers do - so the entity passes a session and delivers **only the new messages**. This + the providers do, so the entity passes a session and delivers **only the new messages**. This holds whether history lives in durable state, an external store, or the model service. 2. **Should durable state be bound?** Only when a `DurableHistoryProvider` is present. -The entity therefore replays its own persisted history in exactly one case: an agent that does not +The entity therefore replays its own persisted history in exactly one case, an agent that does not expose the context pipeline at all (for example a fully custom agent). Routing external-store or -service-backed agents down that path was incorrect - it either bypassed their provider entirely or -re-sent history the service already had. +service-backed agents down that path was incorrect, because it either bypassed their provider +entirely or re-sent history the service already had. **Consequence:** passing a session is what re-engages the pipeline, so external history providers -(Cosmos, Redis, file) now function under the durable runtime - previously they were silently -ignored because no session was ever created. They get the in-run filter like any other provider; -what they do not get is the framework rewriting their store, which no language offers today (core +(Cosmos, Redis, file) now function under the durable runtime. Previously they were silently +ignored because no session was ever created. They get the in-run filter like any other provider. +What they do not get is the framework rewriting their store, which no language offers today (core interface gap 1 above). That session must also carry the entity's **stable** session id rather than a generated one. @@ -414,58 +417,58 @@ Restore applies the stored state onto a session created by the agent's own `crea the agent's session type is preserved. **Restoring values as their own types.** Core deserializes state through a type registry that it -seeds with exactly one entry (`Message`); anything else must be registered explicitly, and the -registry is process-local. `to_dict`-based types are never auto-registered - only Pydantic models -are, and only as a side effect of serializing. A durable entity routinely restores in a process that -never serialized the value, so state would come back as plain dicts instead of its own classes. +seeds with exactly one entry (`Message`). Anything else must be registered explicitly, and the +registry is process-local. `to_dict`-based types are never auto-registered, and only Pydantic models +are, and then only as a side effect of serializing. A durable entity routinely restores in a process +that never serialized the value, so state would come back as plain dicts instead of its own classes. Before restoring, the entity therefore registers the serializable types **already loaded in the process**. Nothing is imported from persisted data, so this cannot load code the application has not -already loaded itself - and that is sufficient in practice, because whoever put a value in the state +already loaded itself, and that is sufficient in practice, because whoever put a value in the state bag had to import its class to construct it. The walk is over `SerializationMixin` subclasses and costs tens of microseconds. -Residual gaps, both better fixed in core: +Residual gaps, both better fixed in core. - Pydantic values in state are keyed by `cls.__name__.lower()` and are not covered, since walking every `BaseModel` subclass in the process would be broad and collision-prone. - Core could seed the registry with the state types it ships, which would make this unnecessary. `register_state_type()` is already public and its documentation names cold-start restore as the - motivating case; nothing calls it today. + motivating case, yet nothing calls it today. ### Service-managed conversations When the model service stores the conversation, it identifies the thread with an id. The entity creates a fresh session per operation, so that id is **persisted in durable state and restored on -the next turn** (as part of the serialized session, above); without it the service would start a new +the next turn** (as part of the serialized session, above). Without it the service would start a new thread every turn. The durable history provider additionally no-ops (neither loading nor flushing) for service-managed sessions. -Whether the service owns history is decided with **core's precedence, not the client class alone**: -an explicit `store` in the agent's options wins, and only when it is unset does the client's +Whether the service owns history is decided with **core's precedence, not the client class alone**. +An explicit `store` in the agent's options wins, and only when it is unset does the client's `STORES_BY_DEFAULT` apply. This matters because clients that store by default (such as the Responses API) are routinely put back into client-side mode with `store=False`. Consulting only `STORES_BY_DEFAULT` would leave such an agent with a plain in-memory provider that the durable -runtime never persists - silently losing the conversation between turns. +runtime never persists, silently losing the conversation between turns. Core resolves this rule inside `Agent._run` and does not expose the result, so this layer -**re-derives it** and can drift from core if the rule changes - with silent conversation loss as the +**re-derives it** and can drift from core if the rule changes, with silent conversation loss as the symptom, which is exactly the bug this rule was written to fix. The unit tests here only pin *our* -logic; the end-to-end net is the compaction sample, which runs `store=False` against a +logic. The end-to-end net is the compaction sample, which runs `store=False` against a store-by-default client and asserts recall. *Upstream fix:* expose the resolved decision. ### Retention is a deployment policy, not agent configuration -Compaction annotates; it does not delete. Physically deleting excluded messages bounds durable +Compaction annotates, it does not delete. Physically deleting excluded messages bounds durable storage but is **lossy**, so it is opt-in via `prune_history` at **registration** (app-level default -with a per-agent override) rather than on the agent. This keeps the agent definition portable - the -same agent runs in-memory, where a retention policy would be meaningless - and places the setting -next to its natural sibling, entity lifetime/TTL. +with a per-agent override) rather than on the agent. This keeps the agent definition portable, since +the same agent runs in-memory where a retention policy would be meaningless, and it places the +setting next to its natural sibling, entity lifetime/TTL. ## Related Concern: Entity Lifetime (TTL) and Cleanup -Compaction bounds the *size* of a conversation; entity **lifetime** - when the persisted state is -deleted - is a separate axis. It is out of scope for the decision above, but is recorded here +Compaction bounds the *size* of a conversation. Entity **lifetime**, when the persisted state is +deleted, is a separate axis. It is out of scope for the decision above, but is recorded here because it is the natural sibling of the retention setting introduced by this ADR, and because it has a notable cross-language parity gap in this repository. diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 427f574..325b44f 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -63,7 +63,7 @@ def _register_loaded_state_types() -> None: """Let core restore session state values as their own classes after a cold start. Core deserializes session state through a type registry that it seeds with exactly one entry - (``Message``); anything else must be registered explicitly, and the registry is process-local. + (``Message``). Anything else must be registered explicitly, and the registry is process-local. A durable entity routinely restores state in a process that never serialized it, so without this a provider's state comes back as a plain dict rather than its own class. @@ -384,7 +384,7 @@ def _create_session(self) -> Any: Conversation history lives in the agent's context providers (durable entity state, an external store, or the model service), so a fresh session per operation is enough - but it must carry the entity's **stable** session id. External history providers (Cosmos, Redis, - file) key their storage on ``session.session_id``; with a freshly generated id they would + file) key their storage on ``session.session_id``, and with a freshly generated id they would read and write a different key every turn and never see prior history. """ create_session = getattr(self.agent, "create_session", None) @@ -399,7 +399,7 @@ def _create_session(self) -> Any: def _restore_session(self, session: Any) -> None: """Apply the previous turn's session state onto a freshly created session. - The agent's own ``create_session`` is used so its session type is preserved; only the + The agent's own ``create_session`` is used so its session type is preserved. Only the state bag and the service conversation id are carried over. """ stored = self.state.data.session diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py index 5abc79e..17e5df1 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -104,7 +104,7 @@ def __init__( source_id: Unique identifier for this provider instance. skip_excluded: Omit compaction-excluded messages from loaded context. prune_excluded: Physically delete excluded messages from durable storage - on flush. Lossy; disabled by default. + on flush. Lossy, so it is disabled by default. """ super().__init__( source_id=source_id or self.DEFAULT_SOURCE_ID, @@ -120,7 +120,7 @@ def _binding(self) -> DurableHistoryBinding | None: binding = current_durable_history_binding() if binding is None: logger.warning( - "[DurableHistoryProvider] No durable binding is active; the provider yields no history. " + "[DurableHistoryProvider] No durable binding is active, so the provider yields no history. " "This provider only works inside a durable agent entity operation." ) return binding @@ -210,7 +210,7 @@ async def before_run( ) -> None: """Load durable history into context, unless the service owns the conversation.""" if self._is_service_managed(session): - logger.debug("[DurableHistoryProvider] Session is service-managed; skipping durable history load.") + logger.debug("[DurableHistoryProvider] Session is service-managed, skipping durable history load.") return await super().before_run(agent=agent, session=session, context=context, state=state) @@ -348,9 +348,9 @@ def ensure_durable_history(agent: SupportsAgentRun, *, prune_history: bool = Fal * **In-memory history** - replaced by a :class:`DurableHistoryProvider` carrying the *same* ``source_id`` and ``skip_excluded``, so any compaction wired to it keeps working untouched. * **Any other history provider** (Cosmos, Redis, file, custom) - left alone. The user chose - where their conversation lives; durable still provides execution durability. + where their conversation lives, and durable still provides execution durability. * **Service-managed history** - left alone. The model service owns the conversation. - * **Agents without the core context pipeline** - left alone; the entity falls back to + * **Agents without the core context pipeline** - left alone, and the entity falls back to replaying its own persisted history. Args: @@ -370,7 +370,7 @@ def ensure_durable_history(agent: SupportsAgentRun, *, prune_history: bool = Fal if _service_stores_history(agent): logger.debug( - "[DurableHistoryProvider] Agent %s stores history service-side; leaving providers unchanged.", + "[DurableHistoryProvider] Agent %s stores history service-side, leaving providers unchanged.", getattr(agent, "name", type(agent).__name__), ) return agent @@ -399,7 +399,7 @@ def ensure_durable_history(agent: SupportsAgentRun, *, prune_history: bool = Fal ) updated = [replacement if p is existing else p for p in provider_list] else: - # A deliberate storage choice (external or custom); do not override it. + # A deliberate storage choice (external or custom), so do not override it. return agent try: @@ -407,7 +407,7 @@ def ensure_durable_history(agent: SupportsAgentRun, *, prune_history: bool = Fal clone.context_providers = updated # type: ignore[attr-defined] except Exception: logger.warning( - "[DurableHistoryProvider] Could not attach durable history to agent %s; " + "[DurableHistoryProvider] Could not attach durable history to agent %s, " "falling back to replaying persisted history.", getattr(agent, "name", type(agent).__name__), exc_info=True, diff --git a/python/samples/13_conversation_compaction/README.md b/python/samples/13_conversation_compaction/README.md index 5f2671b..d130c6b 100644 --- a/python/samples/13_conversation_compaction/README.md +++ b/python/samples/13_conversation_compaction/README.md @@ -31,9 +31,9 @@ Registering that agent with the durable runtime changes nothing about how you co - **Context stays bounded.** Only the messages the strategy keeps are sent to the model, so a long conversation does not grow the per-turn context without limit. -The full conversation remains in durable storage; compaction bounds what the *model* sees. To also -bound what is *stored*, opt in at registration with `add_agent(agent, prune_history=True)` — that is -lossy and therefore off by default. +The full conversation remains in durable storage, and compaction bounds what the *model* sees. To +also bound what is *stored*, opt in at registration with `add_agent(agent, prune_history=True)`, +which is lossy and therefore off by default. ### Client-side vs service-managed history @@ -79,7 +79,7 @@ turn, which it answers correctly. The trade-off is the point of the sample: a sliding window keeps context bounded by *dropping* older turns from what the model sees, so facts from long-past turns are genuinely no longer available to -the model. Those messages are **not deleted** — they remain in durable storage, marked as excluded, +the model. Those messages are **not deleted**. They remain in durable storage, marked as excluded, so the conversation record stays complete and auditable. Choose a strategy accordingly: use summarization if old details must survive in the model's context, and a sliding window when only recent context matters. diff --git a/python/samples/14_external_history_redis/README.md b/python/samples/14_external_history_redis/README.md index 2126e31..166889d 100644 --- a/python/samples/14_external_history_redis/README.md +++ b/python/samples/14_external_history_redis/README.md @@ -19,8 +19,8 @@ agent = Agent( Registering that agent with the durable runtime changes nothing about how you configure it: -- **Your provider is left alone.** Unlike an `InMemoryHistoryProvider` — which is swapped for a - durable-backed one (see [13_conversation_compaction](../13_conversation_compaction)) — a provider +- **Your provider is left alone.** An `InMemoryHistoryProvider` is swapped for a durable-backed one + (see [13_conversation_compaction](../13_conversation_compaction)), but a provider you chose deliberately is never substituted. You picked where the conversation lives. - **It receives a stable session id.** The durable entity creates a fresh session per operation but gives it the entity's own session id, so the provider reads and writes the same key every turn. @@ -28,7 +28,7 @@ Registering that agent with the durable runtime changes nothing about how you co - **Execution is still durable.** Retries, restarts, and orchestration guarantees are unchanged, and durable state still records the conversation for audit. -`redis_history_provider.py` is deliberately small — roughly "read a list, append to a list" — to show +`redis_history_provider.py` is deliberately small, roughly "read a list, append to a list", to show how little a bring-your-own-store provider needs. The same shape applies to Cosmos DB, a file, or any other backend. @@ -65,7 +65,7 @@ other backend. ## What to look for The client states a fact and then asks for it back in a later turn. The agent answers correctly, -which is only possible if Redis served the earlier turn back into the model's context — the durable +which is only possible if Redis served the earlier turn back into the model's context. The durable runtime itself never replays history for this agent. To see it directly, inspect the Redis key while the sample runs: diff --git a/python/samples/14_external_history_redis/redis_history_provider.py b/python/samples/14_external_history_redis/redis_history_provider.py index 0241dcc..539af2b 100644 --- a/python/samples/14_external_history_redis/redis_history_provider.py +++ b/python/samples/14_external_history_redis/redis_history_provider.py @@ -63,7 +63,7 @@ async def get_messages( Args: session_id: The session ID to retrieve messages for. - state: Unused; this provider keeps nothing in session state. + state: Unused, since this provider keeps nothing in session state. **kwargs: Additional arguments (unused). Returns: @@ -85,7 +85,7 @@ async def save_messages( Args: session_id: The session ID to store messages for. messages: The messages to persist. - state: Unused; this provider keeps nothing in session state. + state: Unused, since this provider keeps nothing in session state. **kwargs: Additional arguments (unused). """ if not messages: diff --git a/python/samples/azure_functions/14_conversation_compaction/README.md b/python/samples/azure_functions/14_conversation_compaction/README.md index dc21d4e..858e475 100644 --- a/python/samples/azure_functions/14_conversation_compaction/README.md +++ b/python/samples/azure_functions/14_conversation_compaction/README.md @@ -7,8 +7,8 @@ Framework. It is the Azure Functions counterpart to the standalone ## Key Concepts Demonstrated -- Configuring compaction the ordinary core way — an `InMemoryHistoryProvider` plus a - `CompactionProvider` — with **no durable-specific configuration on the agent**. +- Configuring compaction the ordinary core way, an `InMemoryHistoryProvider` plus a + `CompactionProvider`, with **no durable-specific configuration on the agent**. - The durable runtime swapping the in-memory provider for a durable-backed one at registration, preserving its `source_id` so the compaction provider stays wired to it. - Compaction annotations being persisted alongside the messages, so compaction state is not @@ -31,9 +31,9 @@ agent = Agent( app = AgentFunctionApp(agents=[agent], enable_health_check=True) ``` -The full conversation remains in durable storage; compaction bounds what the *model* sees. To also -bound what is *stored*, opt in at registration with `AgentFunctionApp(..., prune_history=True)` — -that is lossy and therefore off by default. +The full conversation remains in durable storage, and compaction bounds what the *model* sees. To +also bound what is *stored*, opt in at registration with `AgentFunctionApp(..., prune_history=True)`, +which is lossy and therefore off by default. ### Client-side vs service-managed history @@ -52,7 +52,7 @@ credentials, and install the Python dependencies for this sample. This sample us ## Running the Sample Send several turns using the **same** session id so they form one conversation. `demo.http` contains -a ready-made sequence; the equivalent with `curl` is: +a ready-made sequence, and the equivalent with `curl` is: ```bash curl -X POST http://localhost:7071/api/agents/Historian/run \ @@ -71,7 +71,7 @@ window. The trade-off is the point of the sample: a sliding window keeps context bounded by *dropping* older turns from what the model sees, so facts from long-past turns are genuinely no longer available to -the model. Those messages are **not deleted** — they remain in durable storage, marked as excluded, +the model. Those messages are **not deleted**. They remain in durable storage, marked as excluded, so the conversation record stays complete and auditable. Choose a strategy accordingly: use summarization if old details must survive in the model's context, and a sliding window when only recent context matters. diff --git a/python/samples/azure_functions/14_conversation_compaction/function_app.py b/python/samples/azure_functions/14_conversation_compaction/function_app.py index fdab6f3..e8eadb9 100644 --- a/python/samples/azure_functions/14_conversation_compaction/function_app.py +++ b/python/samples/azure_functions/14_conversation_compaction/function_app.py @@ -68,8 +68,8 @@ def _create_agent() -> Any: # 2. Register the agent with AgentFunctionApp so Azure Functions exposes the required triggers. -# Pass prune_history=True here to also delete compacted-out messages from durable storage; -# that is lossy, so the full record is kept by default. +# Pass prune_history=True here to also delete compacted-out messages from durable storage. +# That is lossy, so the full record is kept by default. app = AgentFunctionApp(agents=[_create_agent()], enable_health_check=True, max_poll_retries=50) """ From c63cef7d28b8584f0733a71db511217639c3abb7 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 31 Jul 2026 00:07:03 -0500 Subject: [PATCH 18/32] docs: revert ADR 0032 to proposed and clear deciders/consulted The decision has not been accepted yet, so status goes back to proposed. Deciders and consulted are left blank rather than naming people who have not signed off. Also reverts one word in the orientation note: it said the design was 'realized' in Python, which implied a settled decision. It is a prototype, which is how the rest of the ADR already describes it. --- docs/decisions/0032-durable-thread-compaction.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index d20dddd..c23adc4 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -1,17 +1,17 @@ --- # These are optional elements. Feel free to remove any of them. -status: accepted +status: proposed contact: ahmedmuhsin date: 2026-07-27 -deciders: ahmedmuhsin -consulted: eavanvalkenburg +deciders: +consulted: informed: --- # Thread Compaction for Durable Agents and Workflows -> **How to read this.** Everything through "Pros and Cons of the Options" is the design decision. -> Everything after it records how that decision was realized in Python and what the realization +> **How to read this.** Everything through "Pros and Cons of the Options" is the proposed design. +> Everything after it records how that design was prototyped in Python and what the prototype > surfaced. **.NET is not implemented yet.** > > **Naming.** .NET's `ChatHistoryProvider` and Python's `HistoryProvider` are the same concept. The From b868498cfb181d0b67ace6670373df3501cd0923 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 31 Jul 2026 00:23:23 -0500 Subject: [PATCH 19/32] test: cover the persisted session in integration, and fix two tests that could not fail The session state bag was the one change on this branch verified only in-process. Its unit tests keep the session dict in memory, so they cannot show that the blob survives the entity's JSON encoding, that it carries the entity's own session id rather than a per-operation one, or that the durable history slice is really left out. A new test in test_13 reads the entity back from the scheduler and asserts all three against the real payload. Two pre-existing tests were passing regardless of behavior: - test_06 test_conditional_branching scheduled one spam email and asserted only that the orchestration COMPLETED, never which branch ran, so it would pass if the condition sent every email down the same path. It now asserts the branch-specific output and covers the legitimate branch too, which a stale comment implied was once intended. - test_07 test_hitl_orchestration_timeout wrapped the wait in 'except (RuntimeError, TimeoutError): pass'. Since the shared helper raises on FAILED, its assert was unreachable and the test passed on every outcome including a hung orchestration. It now waits on the client directly and asserts the run failed with an approval timeout rather than for some other reason. --- ..._multi_agent_orchestration_conditionals.py | 41 ++++++++++++++----- ...t_07_dt_single_agent_orchestration_hitl.py | 30 ++++++++------ .../test_13_dt_conversation_compaction.py | 28 +++++++++++++ 3 files changed, 75 insertions(+), 24 deletions(-) diff --git a/python/packages/durabletask/tests/integration_tests/test_06_dt_multi_agent_orchestration_conditionals.py b/python/packages/durabletask/tests/integration_tests/test_06_dt_multi_agent_orchestration_conditionals.py index d50748f..f64a980 100644 --- a/python/packages/durabletask/tests/integration_tests/test_06_dt_multi_agent_orchestration_conditionals.py +++ b/python/packages/durabletask/tests/integration_tests/test_06_dt_multi_agent_orchestration_conditionals.py @@ -63,23 +63,42 @@ def test_agents_registered(self): assert email_agent is not None assert email_agent.name == EMAIL_AGENT_NAME - def test_conditional_branching(self): - """Test that conditional branching works correctly.""" - # Test with obvious spam - spam_payload = { - "email_id": "spam-001", - "email_content": "Buy cheap medications online! No prescription needed! Limited time offer!", - } + def test_conditional_branching(self) -> None: + """Spam takes the spam-handler branch and legitimate mail takes the reply branch. + Asserting only that the orchestration completed would pass even if the condition sent + every email down the same branch, so each case checks the branch-specific output. + """ spam_instance_id = self.dts_client.schedule_new_orchestration( orchestrator="spam_detection_orchestration", - input=spam_payload, + input={ + "email_id": "spam-001", + "email_content": "Buy cheap medications online! No prescription needed! Limited time offer!", + }, ) - - # Both should complete successfully (different branches) - spam_metadata = self.orch_helper.wait_for_orchestration( + spam_metadata, spam_output = self.orch_helper.wait_for_orchestration_with_output( instance_id=spam_instance_id, timeout=300.0, ) assert spam_metadata.runtime_status == OrchestrationStatus.COMPLETED + # The spam handler returns "Email marked as spam: ..."; the other branch returns "Email sent: ...". + assert "marked as spam" in str(spam_output).lower(), f"spam took the wrong branch: {spam_output}" + + legit_instance_id = self.dts_client.schedule_new_orchestration( + orchestrator="spam_detection_orchestration", + input={ + "email_id": "legit-001", + "email_content": ( + "Hi team, please confirm receipt of purchase order PRJ-4417 for the new lab " + "hardware, and let me know the expected delivery date." + ), + }, + ) + legit_metadata, legit_output = self.orch_helper.wait_for_orchestration_with_output( + instance_id=legit_instance_id, + timeout=300.0, + ) + + assert legit_metadata.runtime_status == OrchestrationStatus.COMPLETED + assert "email sent" in str(legit_output).lower(), f"legitimate mail took the wrong branch: {legit_output}" diff --git a/python/packages/durabletask/tests/integration_tests/test_07_dt_single_agent_orchestration_hitl.py b/python/packages/durabletask/tests/integration_tests/test_07_dt_single_agent_orchestration_hitl.py index 8c90d07..49c9c25 100644 --- a/python/packages/durabletask/tests/integration_tests/test_07_dt_single_agent_orchestration_hitl.py +++ b/python/packages/durabletask/tests/integration_tests/test_07_dt_single_agent_orchestration_hitl.py @@ -147,7 +147,13 @@ def test_hitl_orchestration_with_rejection_and_feedback(self): assert metadata.runtime_status == OrchestrationStatus.COMPLETED def test_hitl_orchestration_timeout(self): - """Test HITL orchestration timeout behavior.""" + """With no approval sent, the orchestration fails on its own approval timeout. + + The shared helper raises when an orchestration reaches FAILED, so this waits on the client + directly. Catching and ignoring that exception (as this test used to) also swallowed a + TimeoutError from a hung orchestration, which left no outcome that could fail the test for + the right reason. + """ payload = { "topic": "Cloud computing fundamentals", "max_review_attempts": 1, @@ -160,15 +166,13 @@ def test_hitl_orchestration_timeout(self): input=payload, ) - # Don't send any approval - let it timeout - # The orchestration should fail due to timeout - try: - metadata = self.orch_helper.wait_for_orchestration( - instance_id=instance_id, - timeout=90.0, - ) - # If it completes, it should be failed status due to timeout - assert metadata.runtime_status == OrchestrationStatus.FAILED - except (RuntimeError, TimeoutError): - # Expected - orchestration should timeout and fail - pass + # Don't send any approval - let it hit its own approval timeout. + metadata = self.dts_client.wait_for_orchestration_completion(instance_id=instance_id, timeout=90) + + assert metadata is not None, "orchestration never reached a terminal state" + assert metadata.runtime_status == OrchestrationStatus.FAILED + + # Fail for the right reason: the sample raises TimeoutError("Human approval timed out ..."). + failure = metadata.failure_details + details = getattr(failure, "message", None) or str(failure) + assert "timed out" in details.lower(), f"expected an approval timeout, got: {details}" diff --git a/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py index 60ab300..46e6592 100644 --- a/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py +++ b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py @@ -65,6 +65,34 @@ def test_agent_registration(self) -> None: assert agent is not None assert agent.name == "Historian" + def test_session_is_persisted_and_scoped(self) -> None: + """The serialized session survives real entity storage with the right shape. + + Unit tests keep the session dict in memory, so they cannot show that the blob survives the + entity's JSON encoding, that it carries the entity's **own** session id, or that the durable + history provider's slice really is kept out of it. + """ + agent = self.agent_client.get_agent("Historian") + session = agent.create_session() + + assert agent.run("Name a color.", session=session) is not None + assert agent.run("Name a fruit.", session=session) is not None + + stored = self._read_state(session.durable_session_id).data.session + assert stored is not None, "the session was not persisted" + + # The entity's own id rather than a per-operation one. External history providers key + # their storage on this, so a generated id would restart their conversation every turn. + assert stored["session_id"] == session.durable_session_id.key + + slices = stored["state"] + # The compaction provider's own slice is carried across turns... + assert "compaction" in slices, f"expected provider state to be persisted, got {slices}" + # ...but the durable history provider's is not, since it is derived from + # conversationHistory and would otherwise duplicate the transcript. "in_memory" is the + # source_id the sample's provider keeps after the durable swap. + assert "in_memory" not in slices, f"durable history slice leaked into the session: {slices}" + def test_recent_context_survives_compaction(self) -> None: """A fact inside the retained window is still answerable after several turns.""" agent = self.agent_client.get_agent("Historian") From ebad2f0b4d2127f8f48bf093f756cf83674d7201 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 31 Jul 2026 07:50:34 -0500 Subject: [PATCH 20/32] fix: satisfy mypy on the tests, which CI runs and I did not CI type-checks tests with mypy in addition to ruff and pyright. I ran the other two locally but not mypy, so all four Python jobs failed on the first push. Most errors were stub clients and stub agents passed where the full client or agent protocol is expected. Rather than scattering per-call-site ignores, each affected test file now builds its agent through a small helper that relaxes the type once. That also removed some duplicated construction. Two were real rather than cosmetic. test_durable_history_provider instantiated the abstract HistoryProvider directly, which now uses a concrete stub, and test_durabletask_workflow_initial_input had a context stub whose prepare_agent_task predated the context_messages parameter this branch adds to the protocol. The remaining local mypy error is in integration_tests/conftest.py and comes from redis typing in my environment. CI does not report it, and the file is untouched here. --- .../test_13_dt_conversation_compaction.py | 1 + .../test_14_dt_external_history_redis.py | 3 +- .../tests/test_durable_history_autoswap.py | 40 ++++++++------ .../tests/test_durable_history_provider.py | 54 ++++++++++++++----- ...test_durabletask_workflow_initial_input.py | 8 ++- .../tests/test_workflow_context_parity.py | 21 +++++--- 6 files changed, 89 insertions(+), 38 deletions(-) diff --git a/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py index 46e6592..a26a0e8 100644 --- a/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py +++ b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py @@ -83,6 +83,7 @@ def test_session_is_persisted_and_scoped(self) -> None: # The entity's own id rather than a per-operation one. External history providers key # their storage on this, so a generated id would restart their conversation every turn. + assert session.durable_session_id is not None assert stored["session_id"] == session.durable_session_id.key slices = stored["state"] diff --git a/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py b/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py index 19805a3..2c6d695 100644 --- a/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py +++ b/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py @@ -63,7 +63,8 @@ async def _history_entries(self, session_id: Any) -> list[str]: """ client = aioredis.from_url(self.redis_url, decode_responses=True) try: - return await client.lrange(f"{KEY_PREFIX}:{session_id.key}", 0, -1) + entries: list[str] = await client.lrange(f"{KEY_PREFIX}:{session_id.key}", 0, -1) # type: ignore[misc] + return entries finally: await client.aclose() diff --git a/python/packages/durabletask/tests/test_durable_history_autoswap.py b/python/packages/durabletask/tests/test_durable_history_autoswap.py index 67eb29c..063a4cf 100644 --- a/python/packages/durabletask/tests/test_durable_history_autoswap.py +++ b/python/packages/durabletask/tests/test_durable_history_autoswap.py @@ -63,6 +63,16 @@ def _get_session_id_from_entity(self) -> str: return self._session_id +def _agent(client: Any = None, **kwargs: Any) -> Agent: + """Build an agent with a stub client. + + The stubs cover the parts of the client protocol these tests exercise but not its full generic + signature, so the type is relaxed here rather than at every call site. + """ + chat_client: Any = client if client is not None else _StubClient() + return Agent(client=chat_client, name="a", **kwargs) + + def _history_providers(agent: Any) -> list[Any]: return [p for p in agent.context_providers if isinstance(p, HistoryProvider)] @@ -71,7 +81,7 @@ class TestAutomaticDurableHistory: """The durable runtime substitutes durable-backed history where appropriate.""" def test_agent_without_providers_gets_durable_history(self) -> None: - agent = Agent(client=_StubClient(), name="a") + agent = _agent() prepared = ensure_durable_history(agent) @@ -83,11 +93,7 @@ def test_agent_without_providers_gets_durable_history(self) -> None: assert providers[0].source_id == InMemoryHistoryProvider.DEFAULT_SOURCE_ID def test_in_memory_history_is_replaced_preserving_source_id(self) -> None: - agent = Agent( - client=_StubClient(), - name="a", - context_providers=[InMemoryHistoryProvider(source_id="custom_slot", skip_excluded=True)], - ) + agent = _agent(context_providers=[InMemoryHistoryProvider(source_id="custom_slot", skip_excluded=True)]) prepared = ensure_durable_history(agent) @@ -102,7 +108,7 @@ def test_in_memory_history_is_replaced_preserving_source_id(self) -> None: def test_external_history_provider_is_left_alone(self) -> None: """The user deliberately chose their own storage; durable must not override it.""" external = _ExternalHistoryProvider() - agent = Agent(client=_StubClient(), name="a", context_providers=[external]) + agent = _agent(context_providers=[external]) prepared = ensure_durable_history(agent) @@ -110,7 +116,7 @@ def test_external_history_provider_is_left_alone(self) -> None: assert _history_providers(prepared) == [external] def test_service_managed_history_is_left_alone(self) -> None: - agent = Agent(client=_ServiceStoringClient(), name="a") + agent = _agent(_ServiceStoringClient()) prepared = ensure_durable_history(agent) @@ -124,7 +130,7 @@ def test_store_false_overrides_a_service_storing_client(self) -> None: this, an agent using the Responses API with ``store=False`` would keep a plain in-memory provider that the durable runtime never persists, silently losing the conversation. """ - agent = Agent(client=_ServiceStoringClient(), name="a", default_options={"store": False}) + agent = _agent(_ServiceStoringClient(), default_options={"store": False}) prepared = ensure_durable_history(agent) @@ -133,7 +139,7 @@ def test_store_false_overrides_a_service_storing_client(self) -> None: assert isinstance(providers[0], DurableHistoryProvider) def test_store_true_keeps_history_with_the_service(self) -> None: - agent = Agent(client=_StubClient(), name="a", default_options={"store": True}) + agent = _agent(default_options={"store": True}) prepared = ensure_durable_history(agent) @@ -143,7 +149,7 @@ def test_store_true_keeps_history_with_the_service(self) -> None: def test_existing_durable_provider_is_untouched(self) -> None: """Explicit configuration (for example to enable pruning) wins.""" explicit = DurableHistoryProvider(prune_excluded=True) - agent = Agent(client=_StubClient(), name="a", context_providers=[explicit]) + agent = _agent(context_providers=[explicit]) prepared = ensure_durable_history(agent) @@ -168,7 +174,7 @@ class TestUserAgentIsNotMutated: def test_original_agent_keeps_its_providers(self) -> None: original_provider = InMemoryHistoryProvider() - agent = Agent(client=_StubClient(), name="a", context_providers=[original_provider]) + agent = _agent(context_providers=[original_provider]) original_list = agent.context_providers prepared = ensure_durable_history(agent) @@ -178,7 +184,7 @@ def test_original_agent_keeps_its_providers(self) -> None: assert agent.context_providers == [original_provider] def test_entity_construction_does_not_mutate_the_agent(self) -> None: - agent = Agent(client=_StubClient(), name="a", context_providers=[InMemoryHistoryProvider()]) + agent = _agent(context_providers=[InMemoryHistoryProvider()]) entity = AgentEntity(agent, state_provider=_InMemoryStateProvider()) @@ -190,21 +196,21 @@ class TestPruneHistoryOptIn: """Pruning is a deployment-level retention policy, set at registration.""" def test_off_by_default(self) -> None: - agent = Agent(client=_StubClient(), name="a") + agent = _agent() prepared = ensure_durable_history(agent) assert _history_providers(prepared)[0].prune_excluded is False def test_enabled_via_registration(self) -> None: - agent = Agent(client=_StubClient(), name="a", context_providers=[InMemoryHistoryProvider()]) + agent = _agent(context_providers=[InMemoryHistoryProvider()]) prepared = ensure_durable_history(agent, prune_history=True) assert _history_providers(prepared)[0].prune_excluded is True def test_entity_forwards_the_flag(self) -> None: - agent = Agent(client=_StubClient(), name="a") + agent = _agent() entity = AgentEntity(agent, state_provider=_InMemoryStateProvider(), prune_history=True) @@ -213,7 +219,7 @@ def test_entity_forwards_the_flag(self) -> None: def test_explicit_provider_configuration_wins(self) -> None: """A hand-configured provider is never overridden by the registration flag.""" explicit = DurableHistoryProvider(prune_excluded=False) - agent = Agent(client=_StubClient(), name="a", context_providers=[explicit]) + agent = _agent(context_providers=[explicit]) prepared = ensure_durable_history(agent, prune_history=True) diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py index 6530035..888272f 100644 --- a/python/packages/durabletask/tests/test_durable_history_provider.py +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -134,6 +134,34 @@ async def _summarize_oldest(messages: list[Message]) -> bool: return True +def _agent(providers: list[Any], client: RecordingChatClient | None = None) -> Agent: + """Build an agent with the given context providers. + + The stub client covers the parts of the client protocol these tests exercise but not its full + generic signature, so the type is relaxed here rather than at every call site. + """ + chat_client: Any = client or RecordingChatClient() + return Agent(client=chat_client, name="assistant", context_providers=providers) + + +def _providers_of(entity: AgentEntity) -> list[Any]: + """Return the context providers on the entity's (possibly substituted) agent.""" + return list(getattr(entity.agent, "context_providers", [])) + + +class _StubExternalProvider(HistoryProvider): + """Stand-in for a provider the user configured deliberately (Cosmos, Redis, file).""" + + def __init__(self) -> None: + super().__init__(source_id="external") + + async def get_messages(self, session_id: str | None, **kwargs: Any) -> list[Message]: + return [] + + async def save_messages(self, session_id: str | None, messages: Any, **kwargs: Any) -> None: + return None + + def _build_agent( client: RecordingChatClient, *, @@ -150,7 +178,7 @@ def _build_agent( history_source_id=history.source_id, ) ) - return Agent(client=client, name="assistant", context_providers=providers) + return _agent(providers, client) def _make_entity(agent: Agent, provider: _InMemoryStateProvider) -> AgentEntity: @@ -334,13 +362,13 @@ async def test_service_managed_session_is_skipped(self) -> None: async def test_core_configured_agent_gets_durable_history_automatically(self) -> None: """An agent configured the ordinary core way runs durably with no changes.""" client = RecordingChatClient() - agent = Agent(client=client, name="assistant", context_providers=[InMemoryHistoryProvider()]) + agent = _agent([InMemoryHistoryProvider()], client) entity = _make_entity(agent, _InMemoryStateProvider()) await _run_turns(entity, ["first", "second"]) # The entity swapped in durable-backed history without the user asking. - assert any(isinstance(p, DurableHistoryProvider) for p in entity.agent.context_providers) + assert any(isinstance(p, DurableHistoryProvider) for p in _providers_of(entity)) # The caller's agent is untouched. assert any(isinstance(p, InMemoryHistoryProvider) for p in agent.context_providers) # History is served from durable state, so turn 2 sees turn 1. @@ -371,7 +399,7 @@ async def get_messages(self, session_id: str | None, **kwargs: Any) -> list[Mess async def save_messages(self, session_id: str | None, messages: Any, **kwargs: Any) -> None: seen.append(session_id) - agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[_RecordingExternalProvider()]) + agent = _agent([_RecordingExternalProvider()]) entity = _make_entity(agent, _InMemoryStateProvider(session_id="stable-session")) await _run_turns(entity, ["first", "second"]) @@ -381,12 +409,12 @@ async def save_messages(self, session_id: str | None, messages: Any, **kwargs: A async def test_external_provider_is_not_replaced(self) -> None: """The user chose their own storage; durable must not swap it out.""" - external = HistoryProvider(source_id="external") - agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[external]) + external = _StubExternalProvider() + agent = _agent([external]) entity = _make_entity(agent, _InMemoryStateProvider()) - assert entity.agent.context_providers[0] is external + assert _providers_of(entity)[0] is external class TestSessionStatePersistence: @@ -412,7 +440,7 @@ async def before_run(self, *, agent: Any, session: Any, context: Any, state: dic async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict[str, Any]) -> None: state["runs"] = state.get("runs", 0) + 1 - agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[_CountingProvider()]) + agent = _agent([_CountingProvider()]) entity = _make_entity(agent, _InMemoryStateProvider()) await _run_turns(entity, ["first", "second", "third"]) @@ -432,7 +460,7 @@ async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict state.setdefault("note", Message(role="user", contents=["remember me"])) provider = _InMemoryStateProvider() - agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[_StoringProvider()]) + agent = _agent([_StoringProvider()]) await _run_turns(_make_entity(agent, provider), ["first"]) session_payload = provider._get_state_dict()["data"]["session"] @@ -444,7 +472,7 @@ async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict async def test_service_conversation_id_rides_along(self) -> None: """It is part of the serialized session, so it needs no field of its own.""" provider = _InMemoryStateProvider() - agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[InMemoryHistoryProvider()]) + agent = _agent([InMemoryHistoryProvider()]) entity = _make_entity(agent, provider) await _run_turns(entity, ["first"]) @@ -478,7 +506,7 @@ async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict ToolApprovalState(rules=[ToolApprovalRule("delete_file")]), ) - agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[_ApprovalCarryingProvider()]) + agent = _agent([_ApprovalCarryingProvider()]) await _run_turns(_make_entity(agent, _InMemoryStateProvider()), ["first", "second"]) assert seen[0] is None # nothing granted yet @@ -489,11 +517,11 @@ async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict async def test_durable_history_slice_is_not_persisted(self) -> None: """That slice is derived from conversation_history; storing it would duplicate it.""" provider = _InMemoryStateProvider() - agent = Agent(client=RecordingChatClient(), name="assistant", context_providers=[InMemoryHistoryProvider()]) + agent = _agent([InMemoryHistoryProvider()]) entity = _make_entity(agent, provider) await _run_turns(entity, ["first", "second"]) - durable_history = next(p for p in entity.agent.context_providers if isinstance(p, DurableHistoryProvider)) + durable_history = next(p for p in _providers_of(entity) if isinstance(p, DurableHistoryProvider)) session_state = provider._get_state_dict()["data"]["session"]["state"] assert durable_history.source_id not in session_state diff --git a/python/packages/durabletask/tests/test_durabletask_workflow_initial_input.py b/python/packages/durabletask/tests/test_durabletask_workflow_initial_input.py index a948d64..293d2e5 100644 --- a/python/packages/durabletask/tests/test_durabletask_workflow_initial_input.py +++ b/python/packages/durabletask/tests/test_durabletask_workflow_initial_input.py @@ -35,7 +35,13 @@ def supports_event_streaming(self) -> bool: def current_utc_datetime(self) -> datetime: return datetime.now(timezone.utc) - def prepare_agent_task(self, executor_id: str, message: str, orchestration_instance_id: str) -> Any: + def prepare_agent_task( + self, + executor_id: str, + message: str, + orchestration_instance_id: str, + context_messages: list[dict[str, Any]] | None = None, + ) -> Any: raise AssertionError("This test workflow has no agent executors") def prepare_activity_task(self, activity_name: str, input_json: str) -> str: diff --git a/python/packages/durabletask/tests/test_workflow_context_parity.py b/python/packages/durabletask/tests/test_workflow_context_parity.py index 71893b0..771b173 100644 --- a/python/packages/durabletask/tests/test_workflow_context_parity.py +++ b/python/packages/durabletask/tests/test_workflow_context_parity.py @@ -69,11 +69,20 @@ def _upstream_response(*, texts: list[str], agent_text: str) -> AgentExecutorRes ) +def _stub_agent() -> Any: + """Return the stub agent typed loosely. + + It implements the parts of the agent protocol these tests exercise but not its full signature, + so the type is relaxed here rather than at every call site. + """ + return _StubAgent() + + class TestContextProjection: """The orchestrator projects upstream conversation per context_mode.""" def test_full_mode_forwards_entire_conversation(self) -> None: - executor = AgentExecutor(_StubAgent(), id="downstream") + executor = AgentExecutor(_stub_agent(), id="downstream") upstream = _upstream_response(texts=["first", "second"], agent_text="reply") projected = _build_context_messages(executor, upstream) @@ -82,7 +91,7 @@ def test_full_mode_forwards_entire_conversation(self) -> None: assert len(projected) == 3 def test_last_agent_mode_forwards_only_agent_messages(self) -> None: - executor = AgentExecutor(_StubAgent(), id="downstream", context_mode="last_agent") + executor = AgentExecutor(_stub_agent(), id="downstream", context_mode="last_agent") upstream = _upstream_response(texts=["first", "second"], agent_text="reply") projected = _build_context_messages(executor, upstream) @@ -92,7 +101,7 @@ def test_last_agent_mode_forwards_only_agent_messages(self) -> None: def test_custom_mode_uses_context_filter(self) -> None: executor = AgentExecutor( - _StubAgent(), + _stub_agent(), id="downstream", context_mode="custom", context_filter=lambda messages: messages[-2:], @@ -106,7 +115,7 @@ def test_custom_mode_uses_context_filter(self) -> None: def test_non_agent_input_has_no_upstream_context(self) -> None: """The first node receives raw input, so there is no conversation to forward.""" - executor = AgentExecutor(_StubAgent(), id="downstream") + executor = AgentExecutor(_stub_agent(), id="downstream") assert _build_context_messages(executor, "plain input") is None @@ -134,7 +143,7 @@ def test_context_messages_become_request_messages(self) -> None: def test_repeated_context_is_not_duplicated(self) -> None: """A node that runs twice in a cycle must not re-record the same conversation.""" provider = _InMemoryStateProvider() - entity = AgentEntity(_StubAgent(), state_provider=provider) + entity = AgentEntity(_stub_agent(), state_provider=provider) first = [Message(role="user", contents=["hello"], message_id="m0")] entity.state.data.conversation_history.append( @@ -153,7 +162,7 @@ def test_repeated_context_is_not_duplicated(self) -> None: def test_fully_duplicate_context_keeps_last_message(self) -> None: """The agent must always receive at least one input message.""" provider = _InMemoryStateProvider() - entity = AgentEntity(_StubAgent(), state_provider=provider) + entity = AgentEntity(_stub_agent(), state_provider=provider) messages = [Message(role="user", contents=["hello"], message_id="m0")] entity.state.data.conversation_history.append( From 56c29c09fd9538c2e899f16ac07f18d85d58394d Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 31 Jul 2026 07:54:05 -0500 Subject: [PATCH 21/32] fix: make the redis lrange result typing environment independent redis-py types lrange differently depending on version, so annotating the result as list[str] passed locally and failed on CI with list[bytes | str]. The helper now takes the result loosely and coerces each entry, which holds either way. --- .../integration_tests/test_14_dt_external_history_redis.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py b/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py index 2c6d695..f12b5cf 100644 --- a/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py +++ b/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py @@ -63,8 +63,10 @@ async def _history_entries(self, session_id: Any) -> list[str]: """ client = aioredis.from_url(self.redis_url, decode_responses=True) try: - entries: list[str] = await client.lrange(f"{KEY_PREFIX}:{session_id.key}", 0, -1) # type: ignore[misc] - return entries + # The client is configured with decode_responses, so entries come back as strings. + # Coerce anyway, since redis-py types lrange as bytes or str depending on version. + entries: Any = await client.lrange(f"{KEY_PREFIX}:{session_id.key}", 0, -1) # type: ignore[misc] + return [entry if isinstance(entry, str) else entry.decode() for entry in entries] finally: await client.aclose() From 002b9efa5e23706b9fe282450a0ba3283694ca5c Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 31 Jul 2026 08:08:34 -0500 Subject: [PATCH 22/32] fix: keep compaction reconciliation correct when messages shift or repeat Both issues come from review on PR 59 and both were real. Each corrupts durable state quietly rather than raising. flush() looked messages up by an index recorded before the run, then inserted compaction-generated summaries into the same entry. The insertion pushed every later message along by one, so the recorded index then pointed at the wrong message and its annotations were written there. Pruning had the same flaw and could delete the wrong message. Positions are now shifted alongside the insertion, and pruning removes by identity rather than index. This stayed hidden because entries normally hold a single message. A workflow node receives the upstream conversation as several messages in one request entry, which is where it bites. The new regression test builds that shape and fails without the fix. _drop_already_stored() kept the newest message when the whole upstream context was already recorded, so the agent still had an input, but it kept the id too. Two stored messages under one id collide in the position map, so only the later one was ever annotated and the earlier copy could never be excluded. The kept copy now drops its id and is assigned a fresh one on load. --- .../agent_framework_durabletask/_entities.py | 7 ++- .../_history_provider.py | 49 +++++++++++++------ .../tests/test_durable_history_provider.py | 48 ++++++++++++++++++ .../tests/test_workflow_context_parity.py | 26 +++++++++- 4 files changed, 113 insertions(+), 17 deletions(-) diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 325b44f..a30811a 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -365,7 +365,12 @@ def _drop_already_stored(self, messages: list[DurableAgentStateMessage]) -> list deduped = [m for m in messages if not m.message_id or m.message_id not in known_ids] if not deduped and messages: - return [messages[-1]] + # Keep the newest message so the agent still has an input, but drop the id it shares + # with the copy already in history. Two stored messages under one id collide in the + # compaction position map, so annotations and pruning would target the wrong one. + repeated = messages[-1] + repeated.message_id = None + return [repeated] return deduped def _find_durable_history_provider(self) -> DurableHistoryProvider | None: diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py index 17e5df1..be14635 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -248,7 +248,7 @@ def flush(self, state: dict[str, Any]) -> None: buffer = cast("list[Message]", raw_buffer) stored_by_id = cast("dict[str, tuple[DurableAgentStateEntry, int]]", raw_positions) - pruned: list[tuple[DurableAgentStateEntry, int]] = [] + pruned: list[tuple[DurableAgentStateEntry, DurableAgentStateMessage]] = [] # Messages that compaction added (summaries) are inserted right after the last # known message so ordering in durable state matches the compacted conversation. last_known: tuple[DurableAgentStateEntry, int] | None = None @@ -260,6 +260,9 @@ def flush(self, state: dict[str, Any]) -> None: if position is None: inserted = self._insert_new_message(binding, message, after=last_known) if inserted is not None: + # The insertion pushed everything after it in that entry along by one, so the + # recorded positions have to move too or later updates land on the wrong message. + self._shift_positions(stored_by_id, inserted) last_known = inserted continue @@ -268,13 +271,29 @@ def flush(self, state: dict[str, Any]) -> None: stored.extension_data = annotations last_known = position if self.prune_excluded and annotations and annotations.get(EXCLUDED_KEY): - pruned.append(position) + pruned.append((entry, stored)) if pruned: self._prune(binding, pruned) binding.state_provider.persist_state() + @staticmethod + def _shift_positions( + stored_by_id: dict[str, tuple[DurableAgentStateEntry, int]], + inserted: tuple[DurableAgentStateEntry, int], + ) -> None: + """Move recorded positions that an insertion pushed further along their entry. + + Args: + stored_by_id: Recorded ``message_id`` to position mapping, updated in place. + inserted: The entry and index the new message was inserted at. + """ + entry, index = inserted + for message_id, (stored_entry, stored_index) in list(stored_by_id.items()): + if stored_entry is entry and stored_index >= index: + stored_by_id[message_id] = (stored_entry, stored_index + 1) + @staticmethod def _insert_new_message( binding: DurableHistoryBinding, @@ -297,18 +316,20 @@ def _insert_new_message( return first, 0 @staticmethod - def _prune(binding: DurableHistoryBinding, pruned: list[tuple[DurableAgentStateEntry, int]]) -> None: - """Physically remove excluded messages (and any entries left empty).""" - by_entry: dict[int, list[int]] = {} - for entry, index in pruned: - by_entry.setdefault(id(entry), []).append(index) - - for entry, _ in pruned: - indexes = by_entry.pop(id(entry), None) - if indexes is None: - continue - for index in sorted(indexes, reverse=True): - del entry.messages[index] + def _prune( + binding: DurableHistoryBinding, + pruned: list[tuple[DurableAgentStateEntry, DurableAgentStateMessage]], + ) -> None: + """Physically remove excluded messages (and any entries left empty). + + Removal is by identity rather than index, since insertions earlier in this flush may have + moved messages within their entry. + """ + for entry, stored in pruned: + for index, candidate in enumerate(entry.messages): + if candidate is stored: + del entry.messages[index] + break history = binding.state_provider.state.data.conversation_history remaining = [entry for entry in history if entry.messages] diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py index 888272f..75a2c8d 100644 --- a/python/packages/durabletask/tests/test_durable_history_provider.py +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -328,6 +328,54 @@ async def test_summary_is_not_duplicated_across_turns(self) -> None: ids = [m.message_id for m in _stored_messages(entity) if m.message_id] assert len(ids) == len(set(ids)), f"duplicate message ids persisted: {ids}" + async def test_insertion_keeps_later_positions_valid(self) -> None: + """Inserting into an entry shifts its later messages, so recorded positions must follow. + + Entries normally hold a single message, which hides this. A workflow node receives the + upstream conversation as several messages in one request entry, so a summary inserted in + the middle of that entry invalidates the recorded index of everything after it, and the + annotation lands on the wrong stored message. + """ + + async def _insert_then_exclude_up3(messages: list[Message]) -> bool: + if any((m.additional_properties or {}).get("_marker") for m in messages): + return False + summary = Message( + role="assistant", + contents=["summary"], + message_id="summary_mid", + additional_properties={"_marker": True}, + ) + # Insert near the front, so messages later in the *same* durable entry shift. + messages.insert(1, summary) + for message in messages: + if message.message_id == "up-3": + message.additional_properties = dict(message.additional_properties or {}) | {"_excluded": True} + return True + + client = RecordingChatClient() + entity = _make_entity( + _build_agent(client, with_compaction=True, strategy=_insert_then_exclude_up3), + _InMemoryStateProvider(), + ) + + # An upstream conversation delivered as one multi-message request entry. + context = [ + Message(role="user", contents=["upstream one"], message_id="up-1").to_dict(), + Message(role="assistant", contents=["upstream two"], message_id="up-2").to_dict(), + Message(role="user", contents=["upstream three"], message_id="up-3").to_dict(), + ] + await entity.run({"message": "upstream three", "correlationId": "c0", "contextMessages": context}) + await entity.run({"message": "next", "correlationId": "c1"}) + + stored = {m.message_id: m for m in _stored_messages(entity) if m.message_id} + assert "up-3" in stored, f"expected the upstream messages to be persisted: {list(stored)}" + + # The annotation must land on up-3 itself, not on the neighbour that shifted when the + # summary was inserted earlier in the same entry. + assert (stored["up-3"].extension_data or {}).get("_excluded"), "annotation did not reach up-3" + assert not (stored["up-2"].extension_data or {}).get("_excluded"), "annotation shifted onto up-2" + async def test_service_managed_session_is_skipped(self) -> None: """When the model service owns the conversation, the provider must not participate.""" from types import SimpleNamespace diff --git a/python/packages/durabletask/tests/test_workflow_context_parity.py b/python/packages/durabletask/tests/test_workflow_context_parity.py index 771b173..28e20d9 100644 --- a/python/packages/durabletask/tests/test_workflow_context_parity.py +++ b/python/packages/durabletask/tests/test_workflow_context_parity.py @@ -160,7 +160,11 @@ def test_repeated_context_is_not_duplicated(self) -> None: assert [m.message_id for m in entry.messages] == ["m1"] def test_fully_duplicate_context_keeps_last_message(self) -> None: - """The agent must always receive at least one input message.""" + """The agent must always receive at least one input message. + + The kept copy loses its id, because storing two messages under one id would collide in the + compaction position map and send annotations or pruning to the wrong stored message. + """ provider = _InMemoryStateProvider() entity = AgentEntity(_stub_agent(), state_provider=provider) @@ -172,7 +176,25 @@ def test_fully_duplicate_context_keeps_last_message(self) -> None: entry = DurableAgentStateRequest.from_run_request(self._request(messages, "corr-1")) entry.messages = entity._drop_already_stored(entry.messages) - assert [m.message_id for m in entry.messages] == ["m0"] + assert len(entry.messages) == 1 + assert entry.messages[0].message_id is None + assert entry.messages[0].to_chat_message().text == "hello" + + def test_repeated_context_does_not_duplicate_message_ids(self) -> None: + """A cycle that re-delivers the whole upstream conversation must not collide ids.""" + provider = _InMemoryStateProvider() + entity = AgentEntity(_stub_agent(), state_provider=provider) + + messages = [Message(role="user", contents=["hello"], message_id="m0")] + for index in range(3): + entry = DurableAgentStateRequest.from_run_request(self._request(messages, f"corr-{index}")) + entry.messages = entity._drop_already_stored(entry.messages) + entity.state.data.conversation_history.append(entry) + + stored_ids = [ + m.message_id for entry in entity.state.data.conversation_history for m in entry.messages if m.message_id + ] + assert len(stored_ids) == len(set(stored_ids)), f"duplicate message ids persisted: {stored_ids}" class TestRunRequestRoundTrip: From 1f38a6b7181318cf47515f0a48a1808603f7ed20 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 31 Jul 2026 08:23:44 -0500 Subject: [PATCH 23/32] fix: derive synthesized message ids from persisted state, not object identity Third issue from review on PR 59, and also real, though not for the reason given. Messages stored without an id were given one built from id(entry). Within a single load and flush cycle that is consistent, and the id is written back into durable state, so a cold start before the first flush just regenerates a fresh consistent set rather than corrupting anything. The actual hazard is address reuse. A later run can allocate an entry at an address a previous run already used, producing an id that run persisted. Two stored messages then share a key in the position map, which is the same corruption the duplicate id fix addressed. The id now comes from the entry type, its correlation id or created_at, and the message index, all of which are persisted. The entry type is needed because a request and its response share a correlation id. The new test reloads the same state twice and fails when the id is taken from object identity. --- .../_history_provider.py | 22 ++++++++++++- .../tests/test_durable_history_provider.py | 32 +++++++++++++++++++ 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py index be14635..a95f5de 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -136,6 +136,26 @@ def _replayable_entries(self, binding: DurableHistoryBinding) -> Iterator[tuple[ for index in range(len(entry.messages)): yield entry, index + @staticmethod + def _synthetic_message_id(entry: DurableAgentStateEntry, index: int) -> str: + """Build an id for a stored message that arrived without one. + + The id comes from persisted fields, so a cold start or a retried flush regenerates the + same value. An id derived from object identity would not, and a recycled address could + collide with an id an earlier run already persisted. + + Args: + entry: History entry holding the message. + index: Position of the message within that entry. + + Returns: + An id unique within the conversation history. + """ + # A request and its response share a correlation id, so the entry type is what tells the + # two sides of an exchange apart. + scope = entry.correlation_id or entry.created_at.isoformat() + return f"durable_{entry.json_type.value}_{scope}_{index}" + @staticmethod def _to_message(stored: DurableAgentStateMessage) -> Message | None: """Convert a persisted message into one that is safe to replay to a chat client.""" @@ -169,7 +189,7 @@ async def get_messages( if not message.message_id: # Give every loaded message a stable identity so compaction results can be # reconciled back onto durable state on flush. - message.message_id = f"durable_{id(entry):x}_{index}" + message.message_id = self._synthetic_message_id(entry, index) stored.message_id = message.message_id loaded.append(message) id_map[message.message_id] = (entry, index) diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py index 75a2c8d..16972df 100644 --- a/python/packages/durabletask/tests/test_durable_history_provider.py +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -8,6 +8,7 @@ """ from collections.abc import AsyncIterable, Awaitable, Sequence +from copy import deepcopy from typing import Any import pytest @@ -376,6 +377,37 @@ async def _insert_then_exclude_up3(messages: list[Message]) -> bool: assert (stored["up-3"].extension_data or {}).get("_excluded"), "annotation did not reach up-3" assert not (stored["up-2"].extension_data or {}).get("_excluded"), "annotation shifted onto up-2" + async def test_generated_ids_survive_a_cold_start(self) -> None: + """Ids synthesized for messages stored without one must derive from persisted state. + + A cold start or a retried flush rebuilds the entry objects at fresh addresses, so an id + taken from object identity would differ every run, and a recycled address could even + collide with an id an earlier run already persisted. + """ + provider = _InMemoryStateProvider() + entity = _make_entity(_build_agent(RecordingChatClient()), provider) + await _run_turns(entity, ["first", "second"]) + + # History as a producer that does not stamp message ids would have written it. + raw = deepcopy(provider._get_state_dict()) + for entry in raw["data"]["conversationHistory"]: + for message in entry["messages"]: + message.pop("messageId", None) + + async def _synthesized_ids() -> list[str]: + restarted_provider = _InMemoryStateProvider() + restarted_provider._set_state_dict(deepcopy(raw)) + restarted = _make_entity(_build_agent(RecordingChatClient()), restarted_provider) + await restarted.run({"message": "third", "correlationId": "corr-restart"}) + return [m.message_id for m in _stored_messages(restarted) if (m.message_id or "").startswith("durable_")] + + first = await _synthesized_ids() + second = await _synthesized_ids() + + assert first, "expected ids to be synthesized for the messages that had none" + assert len(first) == len(set(first)), f"synthesized ids collided within one run: {first}" + assert first == second, f"synthesized ids changed across a cold start: {first} != {second}" + async def test_service_managed_session_is_skipped(self) -> None: """When the model service owns the conversation, the provider must not participate.""" from types import SimpleNamespace From 65507c9ecc9acc39e5a8885177376a7de01ff852 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 31 Jul 2026 12:23:55 -0500 Subject: [PATCH 24/32] fix: put the new samples on Foundry, which is what CI provisions The three samples added on this branch read AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODEL. CI only sets FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL, so every worker subprocess died with KeyError: 'AZURE_OPENAI_MODEL' and took both new integration test classes down with it. Local runs passed because my integration .env happens to carry both sets of variables. Every other sample in the repo uses the Foundry pair, so this was a convention break my environment hid. The samples now use FoundryChatClient, with the env templates, requirements, Functions settings template, and READMEs updated to match. default_options store=False still carries the point of the compaction samples, since Foundry stores conversations on the service by default too. Verified against the real service: test_13 is 5 for 5 and test_14 is 4 for 4, so compaction is genuinely operating on client-side history. --- .../13_conversation_compaction/.env.example | 8 ++++---- .../13_conversation_compaction/README.md | 4 ++-- .../requirements.txt | 4 ++-- .../13_conversation_compaction/sample.py | 2 +- .../13_conversation_compaction/worker.py | 18 +++++++++--------- .../14_external_history_redis/.env.example | 8 ++++---- .../14_external_history_redis/README.md | 2 +- .../14_external_history_redis/requirements.txt | 4 ++-- .../14_external_history_redis/sample.py | 2 +- .../14_external_history_redis/worker.py | 10 +++++----- .../14_conversation_compaction/README.md | 6 +++--- .../14_conversation_compaction/function_app.py | 18 +++++++++--------- .../local.settings.json.template | 4 ++-- .../requirements.txt | 4 ++-- 14 files changed, 47 insertions(+), 47 deletions(-) diff --git a/python/samples/13_conversation_compaction/.env.example b/python/samples/13_conversation_compaction/.env.example index b4ba5f8..30f5c34 100644 --- a/python/samples/13_conversation_compaction/.env.example +++ b/python/samples/13_conversation_compaction/.env.example @@ -1,5 +1,5 @@ -# Azure OpenAI resource endpoint, e.g. https://your-resource.openai.azure.com/ -AZURE_OPENAI_ENDPOINT= +# Azure AI Foundry project endpoint URL, e.g. https://your-project.services.ai.azure.com/api/projects/your-project +FOUNDRY_PROJECT_ENDPOINT= -# Model deployment name in your Azure OpenAI resource -AZURE_OPENAI_MODEL= +# Model deployment name in your Foundry project +FOUNDRY_MODEL= diff --git a/python/samples/13_conversation_compaction/README.md b/python/samples/13_conversation_compaction/README.md index d130c6b..e54e5c1 100644 --- a/python/samples/13_conversation_compaction/README.md +++ b/python/samples/13_conversation_compaction/README.md @@ -38,7 +38,7 @@ which is lossy and therefore off by default. ### Client-side vs service-managed history Compaction only applies to history the **client** owns. When a chat client keeps the conversation on -the service (Foundry threads, or the Responses API with `store=True`), the service owns the model's +the service (Foundry and the Responses API both do so by default), the service owns the model's context, the durable entity keeps the transcript purely as a record, and the durable history provider stays out of the way. This sample sets `store=False` so history is client-side and compaction has something to compact. @@ -51,7 +51,7 @@ something to compact. docker run -d --name dts-emulator -p 8080:8080 -p 8082:8082 mcr.microsoft.com/dts/dts-emulator:latest ``` -2. Copy `.env.example` to `.env` and set `AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_MODEL`. +2. Copy `.env.example` to `.env` and set `FOUNDRY_PROJECT_ENDPOINT` and `FOUNDRY_MODEL`. 3. Sign in for `AzureCliCredential`: diff --git a/python/samples/13_conversation_compaction/requirements.txt b/python/samples/13_conversation_compaction/requirements.txt index 0fd0008..ea73f71 100644 --- a/python/samples/13_conversation_compaction/requirements.txt +++ b/python/samples/13_conversation_compaction/requirements.txt @@ -1,12 +1,12 @@ # Agent Framework packages # To use the deployed version, uncomment the lines below and comment out the local installation lines -# agent-framework-openai +# agent-framework-foundry # agent-framework-durabletask # Local installation (for development and testing) # Each package must be listed explicitly because pip doesn't resolve uv workspace sources. # Without explicit entries, pip would fetch transitive dependencies from PyPI instead of local source. -agent-framework-openai>=1.10.1,<2 # Azure OpenAI support from PyPI +agent-framework-foundry>=1.10.1,<2 # Foundry support from PyPI -e ../../packages/durabletask # Local Durable Task package under development # Azure authentication diff --git a/python/samples/13_conversation_compaction/sample.py b/python/samples/13_conversation_compaction/sample.py index 16463d7..5e1ee06 100644 --- a/python/samples/13_conversation_compaction/sample.py +++ b/python/samples/13_conversation_compaction/sample.py @@ -6,7 +6,7 @@ register the compacting agent, then the client drives a multi-turn conversation. Prerequisites: -- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODEL +- Set FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL - Sign in with Azure CLI for AzureCliCredential authentication - Durable Task Scheduler must be running (e.g., using Docker) diff --git a/python/samples/13_conversation_compaction/worker.py b/python/samples/13_conversation_compaction/worker.py index aff201f..be6b523 100644 --- a/python/samples/13_conversation_compaction/worker.py +++ b/python/samples/13_conversation_compaction/worker.py @@ -14,13 +14,13 @@ No durable-specific configuration is required on the agent itself. Note on service-managed conversations: compaction applies to history the *client* owns. When a -chat client keeps the conversation on the service (for example Foundry threads, or the Responses -API with ``store=True``), the service owns the model's context and the durable entity keeps the -full transcript purely as a record. This sample therefore uses ``store=False`` so history is -client-side and compaction has something to compact. +chat client keeps the conversation on the service (Foundry and the Responses API both do so by +default), the service owns the model's context and the durable entity keeps the full transcript +purely as a record. This sample therefore sets ``store=False`` so history is client-side and +compaction has something to compact. Prerequisites: -- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODEL +- Set FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL - Sign in with Azure CLI for AzureCliCredential authentication - Start a Durable Task Scheduler (e.g., using Docker) """ @@ -30,7 +30,7 @@ import os from agent_framework import Agent, CompactionProvider, InMemoryHistoryProvider, SlidingWindowStrategy -from agent_framework.openai import OpenAIChatClient +from agent_framework.foundry import FoundryChatClient from agent_framework_durabletask import DurableAIAgentWorker from azure.identity import AzureCliCredential from azure.identity.aio import AzureCliCredential as AsyncAzureCliCredential @@ -66,9 +66,9 @@ def create_historian_agent() -> Agent: ) return Agent( - client=OpenAIChatClient( - azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], - model=os.environ["AZURE_OPENAI_MODEL"], + client=FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], credential=AsyncAzureCliCredential(), ), name="Historian", diff --git a/python/samples/14_external_history_redis/.env.example b/python/samples/14_external_history_redis/.env.example index 58036ae..b89a793 100644 --- a/python/samples/14_external_history_redis/.env.example +++ b/python/samples/14_external_history_redis/.env.example @@ -1,8 +1,8 @@ -# Azure OpenAI resource endpoint, e.g. https://your-resource.openai.azure.com/ -AZURE_OPENAI_ENDPOINT= +# Azure AI Foundry project endpoint URL, e.g. https://your-project.services.ai.azure.com/api/projects/your-project +FOUNDRY_PROJECT_ENDPOINT= -# Model deployment name in your Azure OpenAI resource -AZURE_OPENAI_MODEL= +# Model deployment name in your Foundry project +FOUNDRY_MODEL= # Redis connection string used by the external history provider REDIS_CONNECTION_STRING=redis://localhost:6379 diff --git a/python/samples/14_external_history_redis/README.md b/python/samples/14_external_history_redis/README.md index 166889d..75cb2f9 100644 --- a/python/samples/14_external_history_redis/README.md +++ b/python/samples/14_external_history_redis/README.md @@ -41,7 +41,7 @@ other backend. docker run -d --name redis -p 6379:6379 redis:latest ``` -2. Copy `.env.example` to `.env` and set `AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_MODEL`. +2. Copy `.env.example` to `.env` and set `FOUNDRY_PROJECT_ENDPOINT` and `FOUNDRY_MODEL`. 3. Sign in for `AzureCliCredential`: diff --git a/python/samples/14_external_history_redis/requirements.txt b/python/samples/14_external_history_redis/requirements.txt index 21e7174..ffb066a 100644 --- a/python/samples/14_external_history_redis/requirements.txt +++ b/python/samples/14_external_history_redis/requirements.txt @@ -1,12 +1,12 @@ # Agent Framework packages # To use the deployed version, uncomment the lines below and comment out the local installation lines -# agent-framework-openai +# agent-framework-foundry # agent-framework-durabletask # Local installation (for development and testing) # Each package must be listed explicitly because pip doesn't resolve uv workspace sources. # Without explicit entries, pip would fetch transitive dependencies from PyPI instead of local source. -agent-framework-openai>=1.10.1,<2 # Azure OpenAI support from PyPI +agent-framework-foundry>=1.10.1,<2 # Foundry support from PyPI -e ../../packages/durabletask # Local Durable Task package under development # External history store used by this sample diff --git a/python/samples/14_external_history_redis/sample.py b/python/samples/14_external_history_redis/sample.py index 10c3739..2a9e52f 100644 --- a/python/samples/14_external_history_redis/sample.py +++ b/python/samples/14_external_history_redis/sample.py @@ -6,7 +6,7 @@ the Redis-backed agent, then the client drives a multi-turn conversation. Prerequisites: -- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODEL +- Set FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL - Sign in with Azure CLI for AzureCliCredential authentication - Durable Task Scheduler and Redis must be running (e.g., using Docker) diff --git a/python/samples/14_external_history_redis/worker.py b/python/samples/14_external_history_redis/worker.py index b50c3dd..cfe0b5e 100644 --- a/python/samples/14_external_history_redis/worker.py +++ b/python/samples/14_external_history_redis/worker.py @@ -15,7 +15,7 @@ swapped for a durable-backed one. Prerequisites: -- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_MODEL +- Set FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL - Sign in with Azure CLI for AzureCliCredential authentication - Start a Durable Task Scheduler and a Redis instance (e.g., using Docker) """ @@ -25,7 +25,7 @@ import os from agent_framework import Agent -from agent_framework.openai import OpenAIChatClient +from agent_framework.foundry import FoundryChatClient from agent_framework_durabletask import DurableAIAgentWorker from azure.identity import AzureCliCredential from azure.identity.aio import AzureCliCredential as AsyncAzureCliCredential @@ -50,9 +50,9 @@ def create_archivist_agent() -> Agent: history = RedisHistoryProvider(os.getenv("REDIS_CONNECTION_STRING", "redis://localhost:6379")) return Agent( - client=OpenAIChatClient( - azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], - model=os.environ["AZURE_OPENAI_MODEL"], + client=FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], credential=AsyncAzureCliCredential(), ), name="Archivist", diff --git a/python/samples/azure_functions/14_conversation_compaction/README.md b/python/samples/azure_functions/14_conversation_compaction/README.md index 858e475..c3b1db7 100644 --- a/python/samples/azure_functions/14_conversation_compaction/README.md +++ b/python/samples/azure_functions/14_conversation_compaction/README.md @@ -38,16 +38,16 @@ which is lossy and therefore off by default. ### Client-side vs service-managed history Compaction only applies to history the **client** owns. When a chat client keeps the conversation on -the service (Foundry threads, or the Responses API with `store=True`), the service owns the model's +the service (Foundry and the Responses API both do so by default), the service owns the model's context, the durable entity keeps the transcript purely as a record, and the durable history provider stays out of the way. This sample sets `store=False` so history is client-side and compaction has something to compact. ## Prerequisites -Follow the common setup steps in `../README.md` to install tooling, configure Azure OpenAI +Follow the common setup steps in `../README.md` to install tooling, configure Foundry credentials, and install the Python dependencies for this sample. This sample uses -`AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_MODEL`. +`FOUNDRY_PROJECT_ENDPOINT` and `FOUNDRY_MODEL`. ## Running the Sample diff --git a/python/samples/azure_functions/14_conversation_compaction/function_app.py b/python/samples/azure_functions/14_conversation_compaction/function_app.py index e8eadb9..2a34af4 100644 --- a/python/samples/azure_functions/14_conversation_compaction/function_app.py +++ b/python/samples/azure_functions/14_conversation_compaction/function_app.py @@ -12,19 +12,19 @@ This is the Azure Functions counterpart to the standalone ``13_conversation_compaction`` sample. Note on service-managed conversations: compaction applies to history the *client* owns. When a -chat client keeps the conversation on the service (for example Foundry threads, or the Responses -API with ``store=True``), the service owns the model's context and the durable entity keeps the -full transcript purely as a record. This sample therefore uses ``store=False`` so history is -client-side and compaction has something to compact. +chat client keeps the conversation on the service (Foundry and the Responses API both do so by +default), the service owns the model's context and the durable entity keeps the full transcript +purely as a record. This sample therefore sets ``store=False`` so history is client-side and +compaction has something to compact. -Prerequisites: set `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_MODEL`, and sign in +Prerequisites: set `FOUNDRY_PROJECT_ENDPOINT`, `FOUNDRY_MODEL`, and sign in with Azure CLI before starting the Functions host.""" import os from typing import Any from agent_framework import Agent, CompactionProvider, InMemoryHistoryProvider, SlidingWindowStrategy -from agent_framework.openai import OpenAIChatClient +from agent_framework.foundry import FoundryChatClient from agent_framework_azurefunctions import AgentFunctionApp from azure.identity.aio import AzureCliCredential from dotenv import load_dotenv @@ -50,9 +50,9 @@ def _create_agent() -> Any: ) return Agent( - client=OpenAIChatClient( - azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], - model=os.environ["AZURE_OPENAI_MODEL"], + client=FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], credential=AzureCliCredential(), ), name="Historian", diff --git a/python/samples/azure_functions/14_conversation_compaction/local.settings.json.template b/python/samples/azure_functions/14_conversation_compaction/local.settings.json.template index 5b65dd2..1d8bc82 100644 --- a/python/samples/azure_functions/14_conversation_compaction/local.settings.json.template +++ b/python/samples/azure_functions/14_conversation_compaction/local.settings.json.template @@ -5,7 +5,7 @@ "AzureWebJobsStorage": "UseDevelopmentStorage=true", "DURABLE_TASK_SCHEDULER_CONNECTION_STRING": "Endpoint=http://localhost:8080;TaskHub=default;Authentication=None", "TASKHUB_NAME": "default", - "AZURE_OPENAI_ENDPOINT": "", - "AZURE_OPENAI_MODEL": "" + "FOUNDRY_PROJECT_ENDPOINT": "", + "FOUNDRY_MODEL": "" } } diff --git a/python/samples/azure_functions/14_conversation_compaction/requirements.txt b/python/samples/azure_functions/14_conversation_compaction/requirements.txt index 48738ea..07296cd 100644 --- a/python/samples/azure_functions/14_conversation_compaction/requirements.txt +++ b/python/samples/azure_functions/14_conversation_compaction/requirements.txt @@ -1,12 +1,12 @@ # Agent Framework packages # To use the deployed version, uncomment the lines below and comment out the local installation lines -# agent-framework-openai +# agent-framework-foundry # agent-framework-azurefunctions # Local installation (for development and testing) # Each package must be listed explicitly because pip doesn't resolve uv workspace sources. # Without explicit entries, pip would fetch transitive dependencies from PyPI instead of local source. -agent-framework-openai>=1.10.1,<2 # Azure OpenAI support from PyPI (pulls in core) +agent-framework-foundry>=1.10.1,<2 # Foundry support from PyPI (pulls in core) -e ../../../packages/durabletask # Durable Task support - dependency of azurefunctions -e ../../../packages/azurefunctions # Azure Functions integration - the main package for this sample From 8a6de7e31b1116feb1567f88dee2f877b3906c64 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 31 Jul 2026 17:23:47 -0500 Subject: [PATCH 25/32] fix: close four defects review found in the entity's context and session handling All four come from review on PR 59, all four were real, and all four are in code this branch added. Each was reproduced with a test that fails without the fix. The duplicate check never fired for messages the workflow itself built. build_agent_executor_response left message_id unset and core does not fill one in, so every forwarded message looked new and a node in a cycle re-recorded the whole conversation on each visit, growing state quadratically. The check was written to prevent exactly that. It now stamps an id derived from the message's position, which is fixed once the message joins the conversation and is rebuilt identically on replay. The existing tests missed this because they assigned ids by hand, which is the one case that already worked. Every agent node in a workflow run received the same core session id. The id was the entity key alone, and workflow entities share the orchestration instance id as their key while differing by entity name, so an external history provider keyed on it filed every node's conversation under one entry. That broke the external provider scenario this branch is meant to support. The core session id is now qualified with the entity name in the existing @name@key form. The plain session_id still flows to callbacks and logs, so streaming is untouched, and the new entity-name hook defaults to empty so older state providers keep working. Session state was assigned to durable state without checking it could be stored. Core neither raises nor warns on a value it cannot serialize, it passes the live object through, and the entity state provider serializes eagerly. The save therefore failed, and the error handler saved again with the same payload, so the second failure escaped and buried whatever the agent had actually returned. The payload is now validated first and the last good session is kept otherwise. The in-memory test provider now serializes on write like the real one, so the test reproduces that whole chain rather than only its first step. The durable history slice was removed after serializing rather than before, so the full transcript and its position index were serialized and then discarded on every turn. It is now removed first and restored afterwards. Two integration tests pinned the old bare-key shape. The Redis one now discovers the key instead of reconstructing it and asserts only one matches, which also proves the conversation is not scattered. Worth recording that the runtime lowercases entity names, so the persisted id reads @dafx-historian@. --- .../_entities.py | 3 + .../agent_framework_durabletask/_entities.py | 73 +++++++++++++-- .../_workflows/orchestrator.py | 15 +++- .../test_13_dt_conversation_compaction.py | 14 ++- .../test_14_dt_external_history_redis.py | 11 ++- .../tests/test_durable_history_provider.py | 57 ++++++++++++ .../tests/test_workflow_context_parity.py | 89 ++++++++++++++++++- 7 files changed, 248 insertions(+), 14 deletions(-) diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py index c69697e..7678c16 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py @@ -47,6 +47,9 @@ def _set_state_dict(self, state: dict[str, Any]) -> None: def _get_session_id_from_entity(self) -> str: return str(self._context.entity_key) + def _get_entity_name_from_entity(self) -> str: + return str(self._context.entity_name) + def create_agent_entity( agent: SupportsAgentRun, diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index a30811a..dceed0b 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -5,6 +5,7 @@ from __future__ import annotations import inspect +import json import logging import warnings from collections.abc import Sequence @@ -42,9 +43,8 @@ logger = logging.getLogger("agent_framework.durabletask") -# Keys produced by core's ``AgentSession.to_dict()``. +# Key produced by core's ``AgentSession.to_dict()``. _SESSION_ID_KEY = "session_id" -_SESSION_STATE_KEY = "state" try: # Root of core's serializable state types. Not part of core's public surface, so a move must @@ -122,10 +122,33 @@ def _get_session_id_from_entity(self) -> str: return cast(str, legacy_hook()) raise NotImplementedError + def _get_entity_name_from_entity(self) -> str: + """Return the entity name, when the host exposes one. + + Optional, so state providers written before this hook existed keep working. They fall + back to a core session id built from the key alone. + """ + return "" + @property def session_id(self) -> str: return self._get_session_id_from_entity() + @property + def core_session_id(self) -> str: + """Identity handed to core's ``create_session``, unique to this entity. + + ``session_id`` is only the entity key, which is not unique on its own. Every agent node + in one workflow run shares a key (the orchestration instance id) and is told apart by + entity name, so an external history provider keyed on the key alone would mix the + histories of different nodes. The name is included here to keep them separate. + + Uses the same ``@name@key`` form as :class:`AgentSessionId`, so the result parses back. + """ + name = self._get_entity_name_from_entity() + key = self.session_id + return f"@{name}@{key}" if name else key + @property def thread_id(self) -> str: """Deprecated alias for :attr:`session_id`.""" @@ -331,7 +354,15 @@ def _capture_session(self, session: Any) -> None: The durable history provider's own slice is dropped before persisting: it is derived from ``conversation_history`` on every turn, so storing it would duplicate the transcript and - let the copy drift from the record of truth. + let the copy drift from the record of truth. It is removed *before* serializing rather + than after, because that slice holds the working message buffer and its position index, + and serializing the whole transcript only to discard it is pure waste. + + Provider state is arbitrary, so the payload is checked before it replaces the last good + one. Core neither raises nor warns on a value it cannot serialize, it passes the live + object through, and the entity state provider serializes eagerly. An unusable payload + would therefore fail the save, and fail it again from the error handler, masking whatever + the agent actually returned. """ if session is None: return @@ -339,11 +370,31 @@ def _capture_session(self, session: Any) -> None: if not callable(to_dict): return - payload = cast("dict[str, Any]", to_dict()) - state = payload.get(_SESSION_STATE_KEY) durable_history = self._find_durable_history_provider() - if isinstance(state, dict) and durable_history is not None: - cast("dict[str, Any]", state).pop(durable_history.source_id, None) + session_state = getattr(session, "state", None) + transient: Any = None + has_transient = False + if durable_history is not None and isinstance(session_state, dict): + bag = cast("dict[str, Any]", session_state) + if durable_history.source_id in bag: + transient = bag.pop(durable_history.source_id) + has_transient = True + try: + payload = cast("dict[str, Any]", to_dict()) + finally: + if has_transient: + cast("dict[str, Any]", session_state)[durable_history.source_id] = transient # type: ignore[union-attr] + + try: + json.dumps(payload) + except (TypeError, ValueError) as exc: + logger.warning( + "[AgentEntity] Session state could not be serialized and was not persisted, so the " + "previous turn's state is kept. A context provider is holding a value that is not " + "JSON-compatible: %s", + exc, + ) + return self.state.data.session = payload def _drop_already_stored(self, messages: list[DurableAgentStateMessage]) -> list[DurableAgentStateMessage]: @@ -391,13 +442,16 @@ def _create_session(self) -> Any: must carry the entity's **stable** session id. External history providers (Cosmos, Redis, file) key their storage on ``session.session_id``, and with a freshly generated id they would read and write a different key every turn and never see prior history. + + The id is qualified with the entity name (see ``core_session_id``) because the key alone + collides across the agent nodes of one workflow run. """ create_session = getattr(self.agent, "create_session", None) if not callable(create_session): raise TypeError( f"Agent {type(self.agent).__name__} exposes context providers but does not support create_session()." ) - session: Any = create_session(session_id=self._state_provider.session_id) + session: Any = create_session(session_id=self._state_provider.core_session_id) self._restore_session(session) return session @@ -579,3 +633,6 @@ def _set_state_dict(self, state: dict[str, Any]) -> None: def _get_session_id_from_entity(self) -> str: return self.entity_context.entity_id.key + + def _get_entity_name_from_entity(self) -> str: + return self.entity_context.entity_id.entity diff --git a/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py b/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py index d5f4340..94ce56c 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py +++ b/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py @@ -83,6 +83,11 @@ SOURCE_ORCHESTRATOR = "__orchestrator__" SOURCE_HITL_RESPONSE = "__hitl_response__" +# Identifies the workflow's own input in the conversation forwarded between agent nodes. Agent +# entities use message ids to recognize context they have already recorded, so every message the +# workflow puts in that conversation needs one. +WORKFLOW_INPUT_MESSAGE_ID = "wf_input_0" + # A WorkflowExecutor node runs its inner workflow as a durable child orchestration. # The parent wraps the node's input in SUBWORKFLOW_INPUT_KEY (defined alongside the # trust-boundary sanitizer in serialization.py) so the child orchestrator can tell a @@ -231,7 +236,15 @@ def build_agent_executor_response( if isinstance(previous_message, AgentExecutorResponse) and previous_message.full_conversation: full_conversation.extend(previous_message.full_conversation) elif isinstance(previous_message, str): - full_conversation.append(Message(role="user", contents=[previous_message])) + full_conversation.append( + Message(role="user", contents=[previous_message], message_id=WORKFLOW_INPUT_MESSAGE_ID) + ) + # Core leaves message_id unset, and a node that runs more than once receives this + # conversation again every time. Without an id the entity cannot tell the repeat from new + # input, so it re-records the whole conversation on each visit and state grows without bound. + # The position is fixed once a message joins the conversation and the orchestrator rebuilds + # the same sequence on replay, so deriving the id from it is both unique and replay-safe. + assistant_message.message_id = f"wf_{executor_id}_{len(full_conversation)}" full_conversation.append(assistant_message) return AgentExecutorResponse( diff --git a/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py index a26a0e8..d9df72f 100644 --- a/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py +++ b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py @@ -81,10 +81,20 @@ def test_session_is_persisted_and_scoped(self) -> None: stored = self._read_state(session.durable_session_id).data.session assert stored is not None, "the session was not persisted" - # The entity's own id rather than a per-operation one. External history providers key + # The entity's own identity rather than a per-operation id. External history providers key # their storage on this, so a generated id would restart their conversation every turn. + # It carries the entity name as well as the key, because agent nodes in one workflow run + # share a key and would otherwise all resolve to the same conversation. assert session.durable_session_id is not None - assert stored["session_id"] == session.durable_session_id.key + key = session.durable_session_id.key + assert stored["session_id"].endswith(f"@{key}"), ( + f"expected the session id to end with the entity key {key}, got {stored['session_id']}" + ) + # The runtime lowercases entity names, so compare that way. + entity_name = session.durable_session_id.entity_name.lower() + assert entity_name in stored["session_id"].lower(), ( + f"expected the entity name in the session id, got {stored['session_id']}" + ) slices = stored["state"] # The compaction provider's own slice is carried across turns... diff --git a/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py b/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py index f12b5cf..7c2323e 100644 --- a/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py +++ b/python/packages/durabletask/tests/integration_tests/test_14_dt_external_history_redis.py @@ -55,6 +55,10 @@ def setup(self, agent_client_factory: type[AgentClientFactoryProtocol]) -> None: async def _history_entries(self, session_id: Any) -> list[str]: """Read the raw history entries the sample's provider wrote for a session. + The provider keys on the core session id, which qualifies the entity key with the entity + name so that agent nodes sharing a key in a workflow run stay separate. The exact name + casing is the runtime's, so the key is discovered rather than reconstructed. + Args: session_id: The durable session id used for the conversation. @@ -63,9 +67,14 @@ async def _history_entries(self, session_id: Any) -> list[str]: """ client = aioredis.from_url(self.redis_url, decode_responses=True) try: + matches: Any = await client.keys(f"{KEY_PREFIX}:*{session_id.key}") # type: ignore[misc] + keys = [k if isinstance(k, str) else k.decode() for k in matches] + assert len(keys) <= 1, f"the conversation was scattered across keys: {keys}" + if not keys: + return [] # The client is configured with decode_responses, so entries come back as strings. # Coerce anyway, since redis-py types lrange as bytes or str depending on version. - entries: Any = await client.lrange(f"{KEY_PREFIX}:{session_id.key}", 0, -1) # type: ignore[misc] + entries: Any = await client.lrange(keys[0], 0, -1) # type: ignore[misc] return [entry if isinstance(entry, str) else entry.decode() for entry in entries] finally: await client.aclose() diff --git a/python/packages/durabletask/tests/test_durable_history_provider.py b/python/packages/durabletask/tests/test_durable_history_provider.py index 16972df..f3a9bc0 100644 --- a/python/packages/durabletask/tests/test_durable_history_provider.py +++ b/python/packages/durabletask/tests/test_durable_history_provider.py @@ -7,6 +7,7 @@ plugs in unchanged. """ +import json from collections.abc import AsyncIterable, Awaitable, Sequence from copy import deepcopy from typing import Any @@ -86,6 +87,9 @@ def _get_state_dict(self) -> dict[str, Any]: return self._state_dict def _set_state_dict(self, state: dict[str, Any]) -> None: + # The durable SDK serializes entity state as it is set, so a value it cannot encode + # surfaces here rather than later. Mirrored so tests see the same failure the host does. + json.dumps(state) self._state_dict = state def _get_session_id_from_entity(self) -> str: @@ -605,3 +609,56 @@ async def test_durable_history_slice_is_not_persisted(self) -> None: durable_history = next(p for p in _providers_of(entity) if isinstance(p, DurableHistoryProvider)) session_state = provider._get_state_dict()["data"]["session"]["state"] assert durable_history.source_id not in session_state + + async def test_durable_history_slice_is_dropped_before_serializing(self) -> None: + """Not after. That slice holds the working buffer, so serializing it is wasted work. + + It also keeps a position index whose values reference durable state objects, so the less + of it that reaches core's serializer the better. + """ + serialized_keys: list[list[str]] = [] + + class _SpySession: + def __init__(self, state: dict[str, Any]) -> None: + self.state = state + self.service_session_id = None + + def to_dict(self) -> dict[str, Any]: + serialized_keys.append(sorted(self.state)) + return {"session_id": "spy", "state": dict(self.state)} + + entity = _make_entity(_build_agent(RecordingChatClient()), _InMemoryStateProvider()) + durable_history = next(p for p in _providers_of(entity) if isinstance(p, DurableHistoryProvider)) + session = _SpySession({durable_history.source_id: {"messages": ["transcript"]}, "other": {"keep": 1}}) + + entity._capture_session(session) + + assert serialized_keys == [["other"]], f"the durable slice was serialized: {serialized_keys}" + assert durable_history.source_id in session.state, "the caller's session was left modified" + + async def test_unserializable_provider_state_does_not_break_the_turn(self) -> None: + """Core passes a value it cannot serialize straight through, without raising or warning. + + Assigning that to entity state fails the save, and the error handler saves again with the + same payload, so the second failure escapes and masks whatever the agent returned. The + payload is checked first instead, keeping the last good session. + """ + + class _UnserializableProvider(ContextProvider): + def __init__(self) -> None: + super().__init__("unserializable") + + async def after_run(self, *, agent: Any, session: Any, context: Any, state: dict[str, Any]) -> None: + state["handle"] = object() + + provider = _InMemoryStateProvider() + agent = _agent([InMemoryHistoryProvider(), _UnserializableProvider()]) + entity = _make_entity(agent, provider) + + await _run_turns(entity, ["first"]) + + stored = provider._get_state_dict() + assert stored["data"].get("session") is None, "an unusable session payload was persisted" + # The turn still completed and the conversation was recorded. + assert len(entity.state.data.conversation_history) == 2 + json.dumps(stored) diff --git a/python/packages/durabletask/tests/test_workflow_context_parity.py b/python/packages/durabletask/tests/test_workflow_context_parity.py index 28e20d9..f7a48d8 100644 --- a/python/packages/durabletask/tests/test_workflow_context_parity.py +++ b/python/packages/durabletask/tests/test_workflow_context_parity.py @@ -23,7 +23,10 @@ DurableAgentStateRequest, RunRequest, ) -from agent_framework_durabletask._workflows.orchestrator import _build_context_messages +from agent_framework_durabletask._workflows.orchestrator import ( + _build_context_messages, + build_agent_executor_response, +) class _StubAgent: @@ -44,8 +47,9 @@ def create_session(self, **kwargs: Any) -> Any: class _InMemoryStateProvider(AgentEntityStateProviderMixin): - def __init__(self, *, session_id: str = "wf-session") -> None: + def __init__(self, *, session_id: str = "wf-session", entity_name: str = "") -> None: self._session_id = session_id + self._entity_name = entity_name self._state_dict: dict[str, Any] = {} def _get_state_dict(self) -> dict[str, Any]: @@ -57,6 +61,9 @@ def _set_state_dict(self, state: dict[str, Any]) -> None: def _get_session_id_from_entity(self) -> str: return self._session_id + def _get_entity_name_from_entity(self) -> str: + return self._entity_name + def _upstream_response(*, texts: list[str], agent_text: str) -> AgentExecutorResponse: conversation = [Message(role="user", contents=[t], message_id=f"m{i}") for i, t in enumerate(texts)] @@ -197,6 +204,84 @@ def test_repeated_context_does_not_duplicate_message_ids(self) -> None: assert len(stored_ids) == len(set(stored_ids)), f"duplicate message ids persisted: {stored_ids}" +class TestWorkflowConversationIdentity: + """Messages the workflow itself builds must carry ids, or a repeated node cannot spot them. + + Core leaves ``message_id`` unset, and the entity's duplicate check treats a message without one + as new. An unstamped conversation therefore defeats the check entirely, and a node in a cycle + re-records the whole conversation on every visit. + """ + + def _cycle_ids(self) -> list[str]: + conversation: Any = "start" + for node in ["A", "B", "A", "B"]: + conversation = build_agent_executor_response(node, f"{node} says", None, conversation) + return [m.message_id or "" for m in conversation.full_conversation] + + def test_every_built_message_carries_an_id(self) -> None: + response = build_agent_executor_response("writer", "drafted", None, "start") + + ids = [m.message_id for m in response.full_conversation] + assert all(ids), f"a message went out without an id: {ids}" + + def test_ids_stay_unique_around_a_cycle(self) -> None: + ids = self._cycle_ids() + + assert all(ids), f"a message went out without an id: {ids}" + assert len(ids) == len(set(ids)), f"ids collided around the cycle: {ids}" + + def test_ids_are_replay_stable(self) -> None: + """The orchestrator rebuilds this conversation on replay, so the ids must not move.""" + assert self._cycle_ids() == self._cycle_ids() + + def test_a_revisited_node_records_only_what_is_new(self) -> None: + provider = _InMemoryStateProvider() + entity = AgentEntity(_stub_agent(), state_provider=provider) + + def _deliver_to_a(context: list[Message], correlation_id: str) -> int: + request = RunRequest( + message=context[-1].text or "", + correlation_id=correlation_id, + context_messages=[m.to_dict() for m in context], + ) + entry = DurableAgentStateRequest.from_run_request(request) + entry.messages = entity._drop_already_stored(entry.messages) + entity.state.data.conversation_history.append(entry) + return len(entry.messages) + + conversation: Any = "start" + conversation = build_agent_executor_response("A", "a1", None, conversation) + conversation = build_agent_executor_response("B", "b1", None, conversation) + first = _deliver_to_a(list(conversation.full_conversation), "corr-1") + + conversation = build_agent_executor_response("A", "a2", None, conversation) + conversation = build_agent_executor_response("B", "b2", None, conversation) + second = _deliver_to_a(list(conversation.full_conversation), "corr-2") + + assert first == 3, f"expected the first delivery to be recorded whole, got {first}" + assert second == 2, f"expected only the two new messages, got {second} of 5 delivered" + + +class TestCoreSessionIdentity: + """The id handed to core must identify one entity, not one workflow run.""" + + def test_workflow_nodes_do_not_share_a_core_session_id(self) -> None: + """Nodes of one workflow share the entity key and differ only by entity name. + + An external history provider keys its storage on the core session id, so taking the key + alone would file every node's conversation under one entry. + """ + writer = _InMemoryStateProvider(session_id="run-1", entity_name="dafx-writer") + reviewer = _InMemoryStateProvider(session_id="run-1", entity_name="dafx-reviewer") + + assert writer.session_id == reviewer.session_id + assert writer.core_session_id != reviewer.core_session_id, f"both nodes resolved to {writer.core_session_id}" + + def test_core_session_id_falls_back_to_the_key(self) -> None: + """State providers predating the entity-name hook keep working.""" + assert _InMemoryStateProvider(session_id="solo").core_session_id == "solo" + + class TestRunRequestRoundTrip: """context_messages survives the entity wire format.""" From 423d846499156c3b75656e8300ea719c8b0b2ba7 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 13 Aug 2026 16:53:03 -0500 Subject: [PATCH 26/32] docs: rework ADR 0032 around capacity, and correct what review found wrong Review on PR 59 was right on several counts and the document said things that were not true. Corrections. Azure Storage has no hard state limit, it offloads anything over 45 KB to blob and pays for size in CPU and memory instead. The 1 MB cap belongs to the scheduler. ChatMessage.MessageId does exist in .NET, so it needs mapping rather than inventing, and .NET keeps exclusion state on CompactionMessageGroup rather than in AdditionalProperties, which carries the summary marker instead. The claim that the built-in store enforces a limit and surfaces a clear error as it is approached was aspirational, nothing measures state size today, and it is withdrawn. Framing. Calling entity state a system of record that auto-derived reduction would silently destroy overstated it. It is a state bag, deleting from it is legitimate, and the driver now says deletion should be a last resort, proportionate, and observable. The service-managed driver now says model provider, because the durable entity is service-managed too under the other reading. L3 is recorded as a weaker seam rather than parity, since core compaction is agent-level and a workflow node inherits L1 unchanged while the inter-executor conversation only has a plain callable. New material. Option 7 adds the scheduler's large payload extension as the first capacity answer, non-lossy and needing no code from this layer. A fourth core gap records why L2 cannot work in .NET yet, because CompactionProvider persists full ChatMessage copies into the session state bag, so a durable provider either stores the transcript twice, loses exclusions and summaries, or forces an index rebuild. Retention replaces prune_history with three modes and auto as the default. It sits at the entity rather than the history provider, so it also covers external providers, service-managed agents and agents with no context pipeline, which previously had no mitigation at all. Under auto it clears context exclusions on a detached view before asking core for a verdict, because the budget is computed over included messages and a user's own window would otherwise make an over-budget conversation look empty. It passes no strategies, since early stop would satisfy the budget immediately and delete everything the user had excluded. Deletion reuses the existing prune path. Also records that TTL is a sliding idle timer, so an active conversation never expires and TTL does not substitute for retention. --- .../0032-durable-thread-compaction.md | 289 ++++++++++++++---- 1 file changed, 228 insertions(+), 61 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index c23adc4..f4b3e5e 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -36,10 +36,16 @@ It helps to separate **three distinct pressures**, because they have different o The first two are per-operation and identical in both runtimes. The third is cumulative. `ConversationHistory` is one blob appended to every turn and re-persisted whole, so it is bounded by -the backend's state-size limit (e.g. classic Azure Storage ~1 MB/entity), whereas a core process is -bounded only by RAM and resets on restart. **Storage capacity is an infrastructure concern, not a -context-window concern**, relieved by raising the limit or moving to an external store, not by -trimming what the model sees. +what the backend will store, and the two backends fail differently. **Durable Task Scheduler caps a +message at 1 MB.** The Azure Storage backend has no hard cap, because it compresses anything over +45 KB into a `-largemessages` blob, but it pays for size in CPU, I/O and memory. So one +backend stops working at the limit and the other degrades toward it, while a core process is bounded +only by RAM and resets on restart. + +**Storage capacity is an infrastructure concern, not a context-window concern.** It is relieved +first by raising the ceiling (blob offload, an external store) and only then by deleting. The two +are kept separate throughout this document, because a tool for bounding what the model reads is not +a tool for bounding what the backend holds. Core MAF already has a compaction system ([ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md), .NET `Microsoft.Agents.AI.Compaction`, Python `agent_framework._compaction`) with **two hooks**. @@ -79,16 +85,22 @@ bounded when the user opts into it?** - **Separate storage capacity from context management.** Bound the model input with compaction (parity with core), and relieve persisted-storage capacity with infrastructure (backend limits, external stores) rather than by silently trimming. -- **No silent data loss in the durable record.** A durable system of record must not quietly - truncate history. Lossy reduction is explicit opt-in, and hard capacity limits should surface a - clear error or warning. +- **Deleting is a last resort, and never silent.** Entity state is a state bag, not an immutable + system of record, so deleting from it is legitimate. But deletion should happen only when capacity + demands it, should remove no more than capacity demands, and should always be observable. - **Determinism and idempotency.** Durable entity operations can be retried, so a lossy reducer (especially LLM summarization) must not corrupt or diverge persisted state across retries. - **Message-list correctness.** Preserve atomic groups (assistant tool-call plus tool-result, and reasoning pairings) so the model input stays valid. -- **Cover both surfaces.** Durable agents **and** durable workflows, in **both** languages. -- **No-op for service-managed storage.** When the service owns the conversation (a - `ConversationId` or `service_session_id` is set), the client has no history to compact. +- **Cover both surfaces.** Durable agents **and** durable workflows, in **both** languages. Core's + compaction system is agent-level, so a workflow agent node inherits it unchanged. The conversation + chained *between* nodes is governed by `AgentExecutor`'s `context_mode` / `context_filter` seam, + which is a plain callable rather than the compaction system. That difference is real and is called + out rather than papered over. +- **Defer when the model provider owns the conversation.** When the chat client keeps history on the + service (a `ConversationId` or `service_session_id` is set), the client holds nothing to compact. + "Service" here means the model provider. The durable entity is not the service in this sense, even + though it is also storage someone else manages. ## Considered Options @@ -109,6 +121,10 @@ bounded when the user opts into it?** on the durable runtime from the user's unchanged configuration. The in-run filter runs in the agent pipeline (L1), and a user-configured reducer or strategy bounds the store (L2, opt-in). The same seam makes external storage backends (Cosmos, Valkey, blob) pluggable for capacity. +- **Option 7, offload large payloads to blob storage.** Raise the ceiling instead of reducing the + content, using the Durable Task Scheduler [large payload + extension](https://learn.microsoft.com/azure/durable-task/scheduler/durable-task-scheduler-large-payloads). + Non-lossy, and the same technique the Azure Storage backend has always used internally. ## Decision Outcome @@ -117,55 +133,121 @@ combined with the workflow hook (Option 4). This makes core's two compaction hoo durable runtime with **no config change**, and cleanly separates context management from storage capacity. -Compaction applies at **three layers**, mapped directly onto the core hooks. +Compaction applies at **three layers**, mapped directly onto the core hooks. Retention, described +below, is a fourth and separate concern: it bounds storage and never touches the model input. | Layer | Core mechanism reused | Lossy? | Role | | --- | --- | --- | --- | | **L1, in-run filter** | `CompactionProvider` in the agent pipeline | No | Always-on. Bounds the **model input** (context window, token cost). Identical to core. | -| **L2, store reducer** | the store-rewrite hook applied to the durable provider's store | Yes | **Opt-in.** Bounds the **persisted store**, only when the user configures a reducer or strategy. Same strategies as core, but the hook is bound to session state upstream, so this layer needs a workaround (see "Core Interface Gaps"). | -| **L3, workflow hook** | the same strategy as the `AgentExecutor` `context_filter` | Yes | Bounds the inter-executor `full_conversation`. | +| **L2, store reducer** | the store-rewrite hook applied to the durable provider's store | Yes | **Opt-in** (`follow_compaction`). Bounds the **persisted store** from the user's own strategy. Python only today, since the hook is bound to session state upstream and .NET cannot persist its compaction state without duplicating the transcript (see "Core Interface Gaps"). | +| **L3, workflow hook** | the same strategy at the `AgentExecutor` `context_filter` seam | Yes | Bounds the inter-executor `full_conversation`. A plainer seam than L1 and L2. | **Two accumulation surfaces.** | Surface | Where it accumulates | Covered by | | --- | --- | --- | -| **In-agent** | the agent's model input, and the persisted `AgentEntity` store | L1 (filter) + L2 (reducer, opt-in) | +| **In-agent** | the agent's model input, and the persisted `AgentEntity` store | L1 for the model input, retention for the store, L2 when opted into | | **Inter-executor (workflow)** | `AgentExecutor.full_conversation`, checkpointed as envelopes | L3 | -**Strict parity - no auto-derive (Option 5 rejected).** Durable honors exactly the hooks the user -configured. If only an in-run filter is configured, durable trims the model input just like core and -the store still grows - the context window is identical in both runtimes, and storage capacity is a -separate concern. Auto-deriving a lossy reducer would use a context-window tool to solve a storage -problem and **silently destroy the durable record**. Capacity is addressed by the backend instead: -the built-in store enforces a limit (surfacing a clear error as it is approached), and an external -provider raises the ceiling. The ideal durable default is therefore the full record in a (possibly -external) provider plus the L1 filter on the model input, never losing the record and always -bounding what the model sees. A lossy L2 reducer stays a deliberate opt-in. +**Strict parity for context, capacity handled separately.** Durable honors exactly the compaction +hooks the user configured, so the model input is identical in both runtimes. It does **not** infer a +storage policy from a context policy: an exclusion means "do not send this to the model", never +"this is safe to delete". Those are two of the three pressures above and conflating them would let a +token-cost decision quietly destroy records the user never agreed to lose. + +Capacity is therefore its own axis, with three answers applied in order. + +1. **Raise the ceiling first.** Blob offload (Option 7) or an external provider. Non-lossy. +2. **Honor an explicit retention choice.** `follow_compaction` is the user authorizing exclusion to + mean deletion (Option 5, in opt-in form). +3. **Evict as a last resort.** Under storage pressure, delete the minimum needed to stay alive. + +### Retention + +One setting, because a single question ("who deleted my message?") should have a single answer. + +| Mode | Behavior | +| --- | --- | +| `keep_all` | Never delete. The entity may reach the backend limit and fail. The honest choice when the complete record matters more than availability. | +| `auto` **(default)** | Delete only under storage pressure, and only down to the low watermark. | +| `follow_compaction` | Delete whatever compaction excluded, every turn. The previous `prune_history=True`. | + +**How `auto` works.** After the turn is recorded and before the state is persisted, the entity +serializes the state and measures it. Under the high watermark, nothing happens. Over it, the entity +builds a detached view of the stored messages **with context exclusions cleared**, hands it to core's +`TokenBudgetComposedStrategy` with no strategies of its own, and deletes whatever that marks. + +Each part earns its place. + +- **The entity triggers it, not the history provider.** `AgentEntity` appends to + `ConversationHistory` in every configuration, including external providers, service-managed agents + and agents with no context pipeline. A trigger inside the provider would protect only the + configurations that already have `follow_compaction` available, and miss the ones with no other + mitigation. +- **Exclusions are cleared on the detached view.** The strategy budgets over *included* messages, so + leaving a user's exclusions in place makes an over-budget conversation look empty and nothing is + evicted. Clearing them makes the budget reflect what is stored. The stored annotations are + untouched, so the user's context decisions survive. +- **No strategies are passed to the budget strategy.** With `early_stop`, a configured sliding window + would satisfy the budget immediately and everything it had excluded would be deleted, which is the + over-deletion this design exists to avoid. An empty strategy list goes straight to core's + deterministic oldest-group eviction, which preserves system messages and keeps tool-call groups + intact. +- **Deletion reuses the existing prune path**, which removes by identity and drops entries left + empty. No second deletion mechanism exists. +- **No summarization.** A model call on the request path re-runs on retry and can diverge. Eviction + is deterministic. + +**Values.** `max_state_bytes` defaults to `1_048_576`, the scheduler limit, and should be raised when +blob offload is configured. The high watermark is `0.85` and the low watermark `0.70`. The gap is +hysteresis: evicting to just under the trigger would evict again every subsequent turn. `0.85` rather +than `0.90` because the budget is approximate twice over, once in the byte-to-token estimate and once +because reasoning content is stripped from the candidate view. The byte budget converts to a token +budget using the ratio of content characters to serialized bytes measured on the spot, rather than a +guessed overhead constant. + +Measuring costs about 8 ms on a conversation at the 1 MB limit, against a turn dominated by a model +call, and `to_dict()` already runs on every persist regardless. + +**Why not simply reduce the store by default.** A default-on reducer only helps agents that already +configured compaction, because nothing else marks messages excludable, and those are the agents least +likely to hit the limit. It would leave every other configuration exactly as exposed as before while +changing behavior for users who were never at risk. + +**Why not rely on blob offload alone.** It raises the ceiling roughly tenfold and does not remove it. +It is preview, it needs a storage account, and its Functions support is currently .NET only. + +**Service-managed storage** is out of scope, mirroring ADR-0019. When the model provider owns the +conversation the client holds no history to compact. See "Service-managed conversations". **Why workflows largely come "for free."** Durable workflow agent execution (`DurableExecutorDispatcher.ExecuteAgentAsync`) runs an agent through the same -`DurableAIAgent → AgentEntity → inner agent` path as standalone durable agents, so **L1 and L2 are -inherited by workflow agent executors**. The workflow's own `full_conversation` between executors -does not pass through the agent, so it needs the separate **L3** hook. - -**Service-managed storage** is out of scope, mirroring ADR-0019. When the service owns the -conversation the client holds no history to compact. See "Service-managed conversations" for how the -runtime detects and handles it. +`DurableAIAgent → AgentEntity → inner agent` path as standalone durable agents, so **L1, L2 and +retention are inherited by workflow agent executors**. The workflow's own `full_conversation` between +executors does not pass through the agent, so it needs the separate **L3** hook. ### Consequences -- Good: **configuration parity**, since the same core strategies and hooks apply on the durable - runtime with no changes. Durable workflows inherit L1+L2, and L3 reuses the existing - `context_filter` seam. -- Good: **no silent data loss**, since the durable record is only reduced when the user opts into a - reducer. Capacity limits surface explicitly rather than truncating. +- Good: **configuration parity for context.** The same core strategies and hooks apply on the durable + runtime with no changes, and retention applies no context policy of its own. +- Good: **every configuration is protected from the capacity limit**, including external providers, + service-managed agents and agents with no context pipeline, because retention lives in the entity + rather than in the history provider. +- Good: **deletion is proportionate.** Under `auto` the amount removed is set by the budget, not by + how much a context strategy happened to exclude. - Neutral: a larger entity change than a bespoke compaction pass, and it must preserve the existing `ConversationHistory` consumer contract (`AgentRunHandle` response polling, audit/replay, TTL). -- Bad: L2 carries workaround code because upstream binds the store-rewrite hook to session state. - That code is deletable if the gap closes. -- Bad: an opt-in LLM-based reducer runs inside the entity operation and re-runs on retry, mitigated - by stable summary identity and optionally by Option 3 to move heavy summarization off the request - path. +- Bad: **L2 is Python-only today.** In .NET, `CompactionProvider` persists its `CompactionMessageIndex` + into `AgentSession.StateBag` with full `ChatMessage` copies, so a durable provider that also persists + the session would store the transcript twice. See "Core Interface Gaps". +- Bad: L2 carries workaround code in Python because upstream binds the store-rewrite hook to session + state. That code is deletable if the gap closes. +- Bad: retention under `auto` behaves differently above and below the watermark, which is harder to + explain than uniform behavior. Accepted because the alternative for those users is the entity + failing. +- Bad: an opt-in LLM-based reducer under `follow_compaction` runs inside the entity operation and + re-runs on retry, mitigated by stable summary identity and optionally by Option 3 to move heavy + summarization off the request path. Eviction under `auto` is deterministic and unaffected. ### Validation @@ -176,9 +258,14 @@ assert that annotations and message ids survive entity serialization, that an ex keeps a whole conversation under one key, and that a downstream workflow agent can reference the upstream conversation. -**Outstanding.** Three things are not covered yet. +**Outstanding.** Not covered yet. -- The .NET realization and its schema parity (gap 3). +- **Retention.** The `auto` and `keep_all` modes are designed but not built. Only the behavior now + called `follow_compaction` exists, under its former name. Nothing measures state size today, so an + entity approaching the scheduler limit gets no warning and no relief. +- The .NET realization and its schema parity (gap 3), and the .NET compaction-state blocker (gap 4). +- Blob offload (Option 7) against a real scheduler, and whether the Durable Functions Python path can + reach it at all. - An external history provider storing history beyond the built-in state-size limit. - Idempotency of an LLM-based reducer across simulated entity retries. @@ -201,15 +288,22 @@ The full argument is in **Decision Outcome** above. This is the summary. compaction never sees, reusing the existing `context_filter` seam. Only relevant to multi-agent workflows, and must reuse core grouping or a naive filter breaks atomic groups. **Adopted alongside Option 6 as L3.** -- **Option 5 - Auto-derive a store reducer.** Would bound durable storage automatically even for - filter-only configs, but conflates storage with context management and **silently truncates the - durable record**, breaking parity and the no-data-loss driver. **Rejected.** +- **Option 5 - Auto-derive a store reducer.** Would bound durable storage without an explicit + reducer, but as a *default* it only reaches agents that already configured compaction, since + nothing else marks messages excludable, and it treats a context decision as consent to delete. + **Adopted in opt-in form as the `follow_compaction` retention mode**, not as the default. - **Option 6 - Durable store as a history provider (chosen).** The user's configuration carries over unchanged, and the same abstraction makes external backends pluggable, so one seam delivers both the opt-in reducer and pluggable storage. Costs a larger entity change that must preserve the - `ConversationHistory` consumer contract (response polling, audit, TTL). L2 also does not come free, - because upstream binds the store-rewrite hook to session state, so the provider publishes a working - buffer and reconciles it itself (see "Core Interface Gaps"). + `ConversationHistory` consumer contract (response polling, audit, TTL). L2 also does not come free: + in Python the store-rewrite hook is bound to session state, so the provider publishes a working + buffer and reconciles it itself, and **in .NET L2 is blocked outright** until core can persist + compaction metadata without duplicating the transcript (see "Core Interface Gaps"). +- **Option 7 - Blob offload.** Raises the ceiling roughly tenfold with no data loss, needs no code + from this layer since the payload store is passed to the worker and client the caller already + builds, and mirrors what the Azure Storage backend does internally. But it is preview, needs a + storage account, does not remove the ceiling, and its Durable Functions support is .NET only + today. **Adopted as the first capacity answer, ahead of any deletion.** ## Cross-Cutting Design Details @@ -289,13 +383,37 @@ around them, but the cleaner fix is upstream. **.NET needs the same treatment, and looks deceptively fine.** Its `DurableAgentStateMessage` already has an `ExtensionData` property, but it is `[JsonExtensionData]`, System.Text.Json's - overflow bucket for *unmapped JSON properties*, not a mapping of `ChatMessage.AdditionalProperties` - where compaction annotations live. `FromChatMessage`/`ToChatMessage` copy neither - `AdditionalProperties` nor `MessageId` (which .NET does not have at all), so annotations are lost - at the **conversion** boundary rather than the JSON one. Anyone checking for "is extension data - persisted?" will see the property and wrongly conclude parity is done. - -4. **Provider cadence splits under per-service-call persistence.** With + overflow bucket for *unmapped JSON properties*, not a mapping of `ChatMessage.AdditionalProperties`. + `FromChatMessage`/`ToChatMessage` copy neither `AdditionalProperties` nor `MessageId`, so both are + lost at the **conversion** boundary rather than the JSON one. Anyone checking for "is extension + data persisted?" will see the property and wrongly conclude parity is done. + + Two clarifications, because the reason this matters is not the obvious one. `ChatMessage.MessageId` + **does** exist in the pinned Microsoft.Extensions.AI.Abstractions and is used throughout .NET, so + it only needs mapping, not inventing. And .NET does **not** keep exclusion state in + `AdditionalProperties` (it lives on `CompactionMessageGroup.IsExcluded`), so mapping these two + fields is necessary but not sufficient. What `AdditionalProperties` does carry is the summary + marker `_is_summary`, which is how a rebuilt index recognises an existing summary instead of + re-summarizing it. + +4. **.NET compaction state cannot be persisted without duplicating the transcript.** This is the + blocker behind "L2 is Python-only today". `CompactionProvider.State` is documented as living in + `AgentSession.StateBag`, holds `List`, and each group serializes its full + `ChatMessage` objects. Every run rewrites it wholesale. That leaves three unappealing choices for a + durable provider that also persists the session: + + | Choice | Consequence | + | --- | --- | + | Persist the session | The conversation is stored twice, in `ConversationHistory` and again in the state bag, so entity state roughly doubles instead of being bounded | + | Omit the provider state | Exclusions and summaries are discarded and summarization can re-run | + | Return only included messages | `CompactionMessageIndex.Update()` sees a trimmed front and rebuilds from scratch, losing the incremental state | + + None of this is inherent to the history-provider approach. It resolves if core can persist + lightweight compaction metadata keyed by `MessageId` rather than whole message copies. Until then + .NET can bound entity state only through the retention path, which is deliberately independent of + `CompactionProvider` and therefore unaffected. + +5. **Provider cadence splits under per-service-call persistence.** With `require_per_service_call_history_persistence=True`, the agent's once-per-run loop skips history providers because the per-service-call middleware drives `before_run`/`after_run` itself, once per **model call** instead of once per run. `CompactionProvider` is not a `HistoryProvider`, so it @@ -322,6 +440,14 @@ In-process workflows give a downstream `AgentExecutor` the upstream conversation `custom` + `context_filter`). The durable orchestrator previously flattened that to the **last message's text**, so a downstream agent lost everything earlier nodes produced. +**L3 is a weaker seam than L1 and L2, and should not be described as parity with them.** Core's +compaction system is agent-level, so a workflow agent node inherits L1 unchanged: the in-process +`AgentExecutor` holds its own `AgentSession` and passes it to `agent.run()`, so any `CompactionProvider` +on the agent runs exactly as it would standalone. The inter-executor conversation has no equivalent. +`context_filter` is a synchronous callable returning a filtered list, not a strategy that annotates +groups, so L3 reuses the same *strategy* at a different, plainer seam rather than reusing the same +hook. + Durable now projects the same conversation and delivers it to the agent entity: - The orchestrator reads the executor's `context_mode`/`context_filter` and projects @@ -333,6 +459,27 @@ Durable now projects the same conversation and delivers it to the agent entity: entity **drops messages whose id it has already recorded**, keeping at least the latest message so the agent always has an input. This relies on the persisted `messageId` described above. +**Dedup is tracked by position, not by stored identity.** Comparing against the ids currently in +`ConversationHistory` breaks the moment retention evicts any of them: their ids leave the comparison +set, the orchestrator re-sends them on the next visit because its own conversation is never evicted, +and the node re-records exactly what was just deleted. That oscillates rather than converges, since +the re-ingested volume is proportional to what was evicted. + +The entity therefore keeps a small map of `executor_id` to the highest conversation position it has +ingested, and drops anything at or below that mark. It is a handful of integers, it is unaffected by +deletion, and it is per executor rather than global because a fan-out gives two branches the same +position. Consequence worth stating: once a message is evicted the node stops seeing it, where the +broken behavior would re-feed it. That is intended. Re-ingesting evicted content defeats the +eviction. + +**Alternatives measured and rejected.** Not persisting the forwarded context, and treating the +orchestrator's conversation as authoritative, both looked cleaner on paper. Measuring what actually +reaches the model showed otherwise. Core in-process sends 11 messages on the third visit of a +`full`-mode cycle, with heavy duplication, while durable today sends 8, because this dedup removes +repeats before they reach the model. For `last_agent` the two are identical. So the current design +already matches core where core is sane and improves on it where core is not, and the alternatives +would have reordered the conversation or dropped context the node should keep. + Behavior difference that remains, by design: each agent node also keeps its **own durable history** (keyed by workflow instance + executor), so per-agent memory survives restarts and is compacted independently - a superset of the in-process behavior rather than a strict match. @@ -373,7 +520,9 @@ Two distinct decisions drive the entity, and conflating them caused bugs. 1. **Who supplies conversation context?** If the agent exposes core's context-provider pipeline, the providers do, so the entity passes a session and delivers **only the new messages**. This holds whether history lives in durable state, an external store, or the model service. -2. **Should durable state be bound?** Only when a `DurableHistoryProvider` is present. +2. **Should durable state be bound?** Retention decides this, at the entity, for every + configuration. It is deliberately not tied to whether a `DurableHistoryProvider` is present, + because the entity records the conversation either way. The entity therefore replays its own persisted history in exactly one case, an agent that does not expose the context pipeline at all (for example a fully custom agent). Routing external-store or @@ -459,11 +608,23 @@ store-by-default client and asserts recall. *Upstream fix:* expose the resolved ### Retention is a deployment policy, not agent configuration -Compaction annotates, it does not delete. Physically deleting excluded messages bounds durable -storage but is **lossy**, so it is opt-in via `prune_history` at **registration** (app-level default -with a per-agent override) rather than on the agent. This keeps the agent definition portable, since -the same agent runs in-memory where a retention policy would be meaningless, and it places the -setting next to its natural sibling, entity lifetime/TTL. +Compaction annotates, it does not delete. Deletion is configured at **registration** (an app-level +default with a per-agent override) rather than on the agent, so the agent definition stays portable: +the same agent runs in-memory where retention would be meaningless, and the setting sits next to its +natural sibling, entity lifetime and TTL. + +The three modes are described under "Retention" in the Decision Outcome. Two properties are worth +restating here, because they are what make retention safe to have on by default. + +- **It applies no context policy.** Retention decides what durable state can hold, never what the + model should read. Filtering the model's view remains entirely L1's job. What retention cannot + avoid is that a deleted message is gone for every reader, including the history provider that + loads context from `ConversationHistory`. Eviction therefore shortens the model's available + history as a consequence of deletion, not as a policy of its own, and only from the point where + the record would otherwise have stopped being writable at all. +- **An exclusion is not consent to delete.** `follow_compaction` is the only mode where a compaction + exclusion causes deletion, and it is opt-in. Under `auto` a user's exclusions are left untouched + and the amount deleted is set by the storage budget alone. ## Related Concern: Entity Lifetime (TTL) and Cleanup @@ -472,6 +633,12 @@ deleted, is a separate axis. It is out of scope for the decision above, but is r because it is the natural sibling of the retention setting introduced by this ADR, and because it has a notable cross-language parity gap in this repository. +**TTL does not substitute for retention.** The .NET mechanism is a sliding idle timer: every +interaction pushes `ExpirationTimeUtc` forward, so an actively used conversation never expires and +grows until it reaches the backend limit. TTL reclaims *abandoned* entities, which bounds how many +exist and what they cost in aggregate. It does nothing about how large a single live entity gets, +which is the failure this ADR's retention design addresses. + - **.NET agents:** `DurableAgentsOptions.DefaultTimeToLive` (default 14 days) provides a global TTL, with a per-agent override via `AddAIAgent(agent, ttl)`. Idle entities self-delete via an `ExpirationTimeUtc` + `CheckAndDeleteIfExpired` self-signal. From 5587159e179fb8ee66e8cb4827db4917e7b824d0 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 13 Aug 2026 20:13:39 -0500 Subject: [PATCH 27/32] feat: bound durable entity state so an agent stops failing at the backend limit Until now nothing measured how large entity state was getting. An agent in a long conversation grew until the scheduler refused the write, with no warning and no relief, and the only mitigation was a flag that did nothing unless compaction was already configured. Retention replaces prune_history with three modes. keep_all never deletes and lets the entity fail, which is the honest choice when the record matters more than availability. auto, the default, deletes only under storage pressure and only down to the low watermark. follow_compaction also deletes what compaction excluded, which is the old prune_history=True. It lives on the entity rather than the history provider because the entity records the conversation in every configuration. External providers, service-managed agents and agents with no context pipeline all accumulate state, and none of them could be protected by a provider-level hook. The eviction itself is almost entirely core's. TokenBudgetComposedStrategy with no strategies of its own goes straight to a deterministic oldest-group eviction that preserves system messages and keeps tool-call groups whole, and deletion reuses the prune path that already existed. What is new is the size check and converting a byte budget into a token budget, which calibrates from the measured ratio of content to serialized bytes rather than assuming an overhead constant. Two details worth recording. The budget is computed over a detached copy with context exclusions cleared, because the strategy budgets over included messages and a user's own sliding window would otherwise make an over-budget conversation look empty and evict nothing. And the user's strategy is deliberately not passed in, since early stop would satisfy the budget immediately and everything they had excluded for context reasons would be deleted. Writing the tests surfaced a real defect. Given a single turn larger than the budget, core's fallback drops everything, including the exchange that just completed, which would discard the result the caller is polling for. The newest exchange is now held back from eviction, grouped by correlation id so a request and its response are protected together. --- .../agent_framework_azurefunctions/_app.py | 45 ++-- .../_entities.py | 18 +- .../packages/azurefunctions/tests/test_app.py | 4 +- .../agent_framework_durabletask/_entities.py | 27 +- .../_history_provider.py | 71 +++-- .../agent_framework_durabletask/_retention.py | 235 +++++++++++++++++ .../agent_framework_durabletask/_worker.py | 36 ++- .../tests/test_durable_history_autoswap.py | 9 +- .../durabletask/tests/test_retention.py | 242 ++++++++++++++++++ 9 files changed, 634 insertions(+), 53 deletions(-) create mode 100644 python/packages/durabletask/agent_framework_durabletask/_retention.py create mode 100644 python/packages/durabletask/tests/test_retention.py diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py index c7c3723..0db5be9 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py @@ -44,6 +44,12 @@ execute_workflow_activity, plan_workflow_registration, ) +from agent_framework_durabletask._retention import ( + DEFAULT_MAX_STATE_BYTES, + DEFAULT_RETENTION, + RetentionMode, + resolve_retention, +) from agent_framework_durabletask._workflows.naming import ( SUBWORKFLOW_REQUEST_SEPARATOR, split_subworkflow_request_id, @@ -244,7 +250,9 @@ def __init__( poll_interval_seconds: float = DEFAULT_POLL_INTERVAL_SECONDS, enable_mcp_tool_trigger: bool = False, default_callback: AgentResponseCallbackProtocol | None = None, - prune_history: bool = False, + prune_history: bool | None = None, + retention: RetentionMode = DEFAULT_RETENTION, + max_state_bytes: int = DEFAULT_MAX_STATE_BYTES, ): """Initialize the AgentFunctionApp. @@ -264,10 +272,13 @@ def __init__( :param poll_interval_seconds: Delay in seconds between polling attempts. Defaults to ``DEFAULT_POLL_INTERVAL_SECONDS``. :param default_callback: Optional callback invoked for agents without specific callbacks. - :param prune_history: Default conversation-retention policy for agents hosted by this app - (including agents inside hosted workflows). When True, messages that compaction - excluded are physically deleted from durable state, bounding stored size. This is - lossy and off by default; ``add_agent`` can override it per agent. + :param prune_history: Deprecated. ``True`` maps to ``retention='follow_compaction'``. + :param retention: Default conversation retention for agents hosted by this app, including + agents inside hosted workflows. ``auto`` deletes only under storage pressure, + ``keep_all`` never deletes and lets the entity fail at the backend limit, and + ``follow_compaction`` also deletes what compaction excluded. ``add_agent`` can + override it per agent. + :param max_state_bytes: Budget for serialized entity state. :note: If no agents are provided, they can be added later using :meth:`add_agent`. """ @@ -288,7 +299,8 @@ def __init__( self.enable_http_endpoints = enable_http_endpoints self.enable_mcp_tool_trigger = enable_mcp_tool_trigger self.default_callback = default_callback - self._prune_history = prune_history + self._retention: RetentionMode = resolve_retention(retention, prune_history) + self._max_state_bytes = max_state_bytes try: retries = int(max_poll_retries) @@ -833,6 +845,7 @@ def add_agent( *, entity_id: str | None = None, prune_history: bool | None = None, + retention: RetentionMode | None = None, ) -> None: """Add an agent to the function app after initialization. @@ -849,8 +862,8 @@ def add_agent( durable entity (and the ``agents`` / ``get_agent`` key) matches the identity the orchestrator dispatches to. Mirrors ``DurableAIAgentWorker.add_agent(entity_id=...)``. - prune_history: Per-agent conversation-retention override. When None, the app-level - ``prune_history`` setting is used. + prune_history: Deprecated. ``True`` maps to ``retention='follow_compaction'``. + retention: Per-agent retention override. When None, the app-level setting is used. Raises: ValueError: If the agent doesn't have a 'name' attribute. @@ -899,7 +912,7 @@ def add_agent( ) effective_callback = callback or self.default_callback - effective_prune_history = self._prune_history if prune_history is None else prune_history + effective_retention: RetentionMode = self._retention if retention is None else retention self._setup_agent_functions( agent, @@ -907,7 +920,7 @@ def add_agent( effective_callback, effective_enable_http_endpoint, effective_enable_mcp_endpoint, - prune_history=effective_prune_history, + retention=resolve_retention(effective_retention, prune_history), ) logger.debug(f"[AgentFunctionApp] Agent '{registration_name}' added successfully") @@ -953,7 +966,7 @@ def _setup_agent_functions( enable_http_endpoint: bool, enable_mcp_tool_trigger: bool, *, - prune_history: bool = False, + retention: RetentionMode = DEFAULT_RETENTION, ) -> None: """Set up the HTTP trigger, entity, and MCP tool trigger for a specific agent. @@ -963,7 +976,7 @@ def _setup_agent_functions( callback: Optional callback to receive response updates enable_http_endpoint: Whether to create HTTP endpoint enable_mcp_tool_trigger: Whether to create MCP tool trigger - prune_history: Whether excluded messages are deleted from durable state. + retention: How much of the conversation durable state may discard. """ logger.debug(f"[AgentFunctionApp] Setting up functions for agent '{agent_name}'...") @@ -974,7 +987,7 @@ def _setup_agent_functions( "[AgentFunctionApp] HTTP run route disabled for agent '%s'", agent_name, ) - self._setup_agent_entity(agent, agent_name, callback, prune_history=prune_history) + self._setup_agent_entity(agent, agent_name, callback, retention=retention) if enable_mcp_tool_trigger: agent_description = agent.description @@ -1117,7 +1130,7 @@ def _setup_agent_entity( agent_name: str, callback: AgentResponseCallbackProtocol | None, *, - prune_history: bool = False, + retention: RetentionMode = DEFAULT_RETENTION, ) -> None: """Register the durable entity responsible for agent state. @@ -1125,7 +1138,7 @@ def _setup_agent_entity( agent: The agent instance agent_name: The agent name (used for both entity identification and function naming) callback: Optional callback for response updates - prune_history: Whether excluded messages are deleted from durable state. + retention: How much of the conversation durable state may discard. """ # Use the prefixed entity name for both registration and function naming entity_name_with_prefix = AgentSessionId.to_entity_name(agent_name) @@ -1138,7 +1151,7 @@ def entity_function(context: df.DurableEntityContext) -> None: - run_agent: (Deprecated) Execute the agent with a message - reset: Clear conversation history """ - entity_handler = create_agent_entity(agent, callback, prune_history=prune_history) + entity_handler = create_agent_entity(agent, callback, retention=retention) entity_handler(context) # Set function name for Azure Functions (used in function.json generation) diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py index 7678c16..5db3ef5 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py @@ -21,6 +21,7 @@ AgentResponseCallbackProtocol, run_agent_coroutine, ) +from agent_framework_durabletask._retention import DEFAULT_MAX_STATE_BYTES, DEFAULT_RETENTION, RetentionMode logger = logging.getLogger("agent_framework.azurefunctions") @@ -55,7 +56,8 @@ def create_agent_entity( agent: SupportsAgentRun, callback: AgentResponseCallbackProtocol | None = None, *, - prune_history: bool = False, + retention: RetentionMode = DEFAULT_RETENTION, + max_state_bytes: int = DEFAULT_MAX_STATE_BYTES, ) -> Callable[[df.DurableEntityContext], None]: """Factory function to create an agent entity class. @@ -64,8 +66,10 @@ def create_agent_entity( callback: Optional callback invoked during streaming and final responses Keyword Args: - prune_history: When True, messages that compaction excluded are physically deleted - from durable state. Lossy retention policy; off by default. + retention: How much of the conversation durable state may discard. ``auto`` deletes only + under storage pressure, ``keep_all`` never deletes, and ``follow_compaction`` also + deletes what compaction excluded. + max_state_bytes: Budget for serialized entity state. Returns: Entity function configured with the agent @@ -78,7 +82,13 @@ async def _entity_coroutine(context: df.DurableEntityContext) -> None: logger.debug("[entity_function] Operation: %s", context.operation_name) state_provider = AzureFunctionEntityStateProvider(context) - entity = AgentEntity(agent, callback, state_provider=state_provider, prune_history=prune_history) + entity = AgentEntity( + agent, + callback, + state_provider=state_provider, + retention=retention, + max_state_bytes=max_state_bytes, + ) operation = context.operation_name diff --git a/python/packages/azurefunctions/tests/test_app.py b/python/packages/azurefunctions/tests/test_app.py index 185b2c2..8560ad3 100644 --- a/python/packages/azurefunctions/tests/test_app.py +++ b/python/packages/azurefunctions/tests/test_app.py @@ -269,7 +269,7 @@ def test_agent_override_enables_http_route_when_app_disabled(self) -> None: app.add_agent(mock_agent, enable_http_endpoint=True) http_route_mock.assert_called_once_with("OverrideAgent") - agent_entity_mock.assert_called_once_with(mock_agent, "OverrideAgent", ANY, prune_history=False) + agent_entity_mock.assert_called_once_with(mock_agent, "OverrideAgent", None, retention="auto") assert app._agent_metadata["OverrideAgent"].http_endpoint_enabled is True def test_agent_override_disables_http_route_when_app_enabled(self) -> None: @@ -286,7 +286,7 @@ def test_agent_override_disables_http_route_when_app_enabled(self) -> None: app.add_agent(mock_agent, enable_http_endpoint=False) http_route_mock.assert_not_called() - agent_entity_mock.assert_called_once_with(mock_agent, "DisabledOverride", ANY, prune_history=False) + agent_entity_mock.assert_called_once_with(mock_agent, "DisabledOverride", None, retention="auto") assert app._agent_metadata["DisabledOverride"].http_endpoint_enabled is False def test_multiple_apps_independent(self) -> None: diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index dceed0b..6fd8244 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -40,6 +40,13 @@ unbind_durable_history, ) from ._models import RunRequest +from ._retention import ( + DEFAULT_MAX_STATE_BYTES, + DEFAULT_RETENTION, + RetentionMode, + enforce_budget, + prunes_excluded, +) logger = logging.getLogger("agent_framework.durabletask") @@ -199,13 +206,16 @@ def __init__( callback: AgentResponseCallbackProtocol | None = None, *, state_provider: AgentEntityStateProviderMixin, - prune_history: bool = False, + retention: RetentionMode = DEFAULT_RETENTION, + max_state_bytes: int = DEFAULT_MAX_STATE_BYTES, ) -> None: # Back the agent's conversation history with durable entity state so an agent that # already works in core runs durably without any configuration change. - self.agent = ensure_durable_history(agent, prune_history=prune_history) + self.agent = ensure_durable_history(agent, prune_history=prunes_excluded(retention)) self.callback = callback self._state_provider = state_provider + self._retention = retention + self._max_state_bytes = max_state_bytes logger.debug("[AgentEntity] Initialized with agent type: %s", type(agent).__name__) @@ -308,6 +318,7 @@ async def run( state_response = DurableAgentStateResponse.from_run_response(correlation_id, agent_run_response) self.state.data.conversation_history.append(state_response) self._capture_session(session) + await self._enforce_retention() self.persist_state() return agent_run_response @@ -326,6 +337,7 @@ async def run( error_state_response = DurableAgentStateResponse.from_run_response(correlation_id, error_response) error_state_response.is_error = True self.state.data.conversation_history.append(error_state_response) + await self._enforce_retention() self.persist_state() return error_response @@ -334,6 +346,17 @@ async def run( if binding_token is not None: unbind_durable_history(binding_token) + async def _enforce_retention(self) -> None: + """Bound durable state before it is persisted, unless the caller asked to keep everything. + + This lives on the entity rather than the history provider because the entity records the + conversation in every configuration, including external providers, service-managed agents + and agents with no context pipeline. Those are exactly the cases with no other mitigation. + """ + if self._retention == "keep_all": + return + await enforce_budget(self.state, max_state_bytes=self._max_state_bytes) + def _has_context_pipeline(self) -> bool: """Whether the agent exposes core's context-provider pipeline. diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py index a95f5de..53934a1 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -127,14 +127,10 @@ def _binding(self) -> DurableHistoryBinding | None: def _replayable_entries(self, binding: DurableHistoryBinding) -> Iterator[tuple[DurableAgentStateEntry, int]]: """Yield (entry, message_index) pairs that participate in model context.""" - for entry in binding.state_provider.state.data.conversation_history: - if isinstance(entry, DurableAgentStateResponse) and entry.is_error: - continue - if binding.correlation_id is not None and entry.correlation_id == binding.correlation_id: - # The in-flight request is delivered as run input, not as history. - continue - for index in range(len(entry.messages)): - yield entry, index + yield from replayable_entries( + binding.state_provider.state.data.conversation_history, + correlation_id=binding.correlation_id, + ) @staticmethod def _synthetic_message_id(entry: DurableAgentStateEntry, index: int) -> str: @@ -345,16 +341,57 @@ def _prune( Removal is by identity rather than index, since insertions earlier in this flush may have moved messages within their entry. """ - for entry, stored in pruned: - for index, candidate in enumerate(entry.messages): - if candidate is stored: - del entry.messages[index] - break + prune_messages(binding.state_provider.state.data.conversation_history, pruned) - history = binding.state_provider.state.data.conversation_history - remaining = [entry for entry in history if entry.messages] - if len(remaining) != len(history): - history[:] = remaining + +def replayable_entries( + history: list[DurableAgentStateEntry], + *, + correlation_id: str | None = None, +) -> Iterator[tuple[DurableAgentStateEntry, int]]: + """Yield (entry, message_index) pairs that participate in model context. + + Shared by the history provider and by retention, so both agree on which stored messages are + real conversation rather than bookkeeping. + + Args: + history: The entity's conversation history. + correlation_id: The in-flight request, which is delivered as run input rather than history. + + Yields: + Each replayable message as its owning entry and its index within that entry. + """ + for entry in history: + if isinstance(entry, DurableAgentStateResponse) and entry.is_error: + continue + if correlation_id is not None and entry.correlation_id == correlation_id: + continue + for index in range(len(entry.messages)): + yield entry, index + + +def prune_messages( + history: list[DurableAgentStateEntry], + pruned: list[tuple[DurableAgentStateEntry, DurableAgentStateMessage]], +) -> None: + """Physically remove the given messages, and any entries left empty. + + Removal is by identity rather than index, since an insertion elsewhere in the same pass may + have moved messages within their entry. + + Args: + history: The entity's conversation history, modified in place. + pruned: The messages to remove, each with the entry that owns it. + """ + for entry, stored in pruned: + for index, candidate in enumerate(entry.messages): + if candidate is stored: + del entry.messages[index] + break + + remaining = [entry for entry in history if entry.messages] + if len(remaining) != len(history): + history[:] = remaining def _service_stores_history(agent: Any) -> bool: diff --git a/python/packages/durabletask/agent_framework_durabletask/_retention.py b/python/packages/durabletask/agent_framework_durabletask/_retention.py new file mode 100644 index 0000000..af3d112 --- /dev/null +++ b/python/packages/durabletask/agent_framework_durabletask/_retention.py @@ -0,0 +1,235 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Bounding durable entity state so an agent does not simply stop working at the backend limit. + +Retention is a **capacity** concern, deliberately separate from compaction. Compaction decides what +the model should read. Retention decides what durable state can afford to hold. An exclusion made +for token cost is not consent to delete the record, so the two never share a decision. + +See ADR 0032, "Retention". +""" + +from __future__ import annotations + +import json +import logging +import warnings +from typing import Literal, cast + +from agent_framework import ( + CharacterEstimatorTokenizer, + Message, + TokenBudgetComposedStrategy, +) + +from ._durable_agent_state import ( + DurableAgentState, + DurableAgentStateEntry, + DurableAgentStateMessage, +) +from ._history_provider import EXCLUDED_KEY, prune_messages, replayable_entries + +logger = logging.getLogger("agent_framework.durabletask") + +RetentionMode = Literal["keep_all", "auto", "follow_compaction"] +"""How much of the conversation durable state is allowed to discard. + +``keep_all`` + Never delete. The entity may reach the backend limit and fail. The honest choice when the + complete record matters more than availability. +``auto`` + Delete only under storage pressure, and only down to the low watermark. The default. +``follow_compaction`` + Also delete whatever compaction excluded, every turn. +""" + +DEFAULT_RETENTION: RetentionMode = "auto" + +DEFAULT_MAX_STATE_BYTES = 1_048_576 +"""The Durable Task Scheduler message limit. Raise it when large payload offload is configured.""" + +HIGH_WATERMARK = 0.85 +"""Fraction of the budget that triggers eviction. + +Below 0.9 because the budget is approximate twice over, once in the byte-to-token estimate and once +because a message's non-text content is not counted when calibrating that estimate. +""" + +LOW_WATERMARK = 0.70 +"""Fraction of the budget to evict down to. + +The gap from the high watermark is hysteresis. Evicting to just under the trigger would evict again +on every subsequent turn. +""" + +_BYTES_PER_TOKEN = 4 +"""Matches ``CharacterEstimatorTokenizer``, which is a flat 4 characters per token.""" + +_MAX_PASSES = 3 +"""Eviction re-measures rather than trusting the estimate, but must not loop indefinitely.""" + + +def prunes_excluded(retention: RetentionMode) -> bool: + """Whether compaction exclusions should be deleted as they are made.""" + return retention == "follow_compaction" + + +def resolve_retention(retention: RetentionMode, prune_history: bool | None) -> RetentionMode: + """Fold the deprecated ``prune_history`` flag into the retention setting. + + Args: + retention: The retention mode the caller asked for. + prune_history: The deprecated flag, or None when it was not supplied. + + Returns: + The effective retention mode. + """ + if prune_history is None: + return retention + warnings.warn( + "prune_history is deprecated; use retention='follow_compaction' to delete what compaction " + "excluded, or retention='keep_all' to never delete.", + DeprecationWarning, + stacklevel=3, + ) + return "follow_compaction" if prune_history else retention + + +async def enforce_budget(state: DurableAgentState, *, max_state_bytes: int = DEFAULT_MAX_STATE_BYTES) -> int: + """Evict oldest conversation groups when persisted state approaches the backend limit. + + The measurement is exact rather than estimated. Serializing state at the 1 MB limit costs a few + milliseconds against a turn dominated by a model call, and ``to_dict()`` already runs on every + persist, so the incremental cost is small and only paid once per turn. + + Args: + state: The entity state, modified in place. + + Keyword Args: + max_state_bytes: The budget for serialized state. + + Returns: + How many messages were removed. Zero is the common case. + """ + high = int(max_state_bytes * HIGH_WATERMARK) + size = _serialized_size(state) + if size < high: + return 0 + + history = state.data.conversation_history + target = int(max_state_bytes * LOW_WATERMARK) + removed = 0 + + for attempt in range(_MAX_PASSES): + # Tighten on each pass, since the byte-to-token conversion is a heuristic and a first + # attempt can land short of the target. + evicted = await _evict_once(history, serialized_size=size, target_bytes=target >> attempt) + if not evicted: + break + removed += evicted + size = _serialized_size(state) + if size < high: + break + + if removed: + logger.warning( + "[Retention] Durable state reached %d bytes of a %d budget, so %d message(s) were " + "evicted oldest-first to %d bytes. Configure retention='keep_all' to disable this, or " + "raise max_state_bytes if large payload offload is enabled.", + high, + max_state_bytes, + removed, + size, + ) + elif size >= high: + logger.error( + "[Retention] Durable state is %d bytes against a %d budget and nothing could be " + "evicted. A single turn is likely larger than the budget itself, which retention " + "cannot resolve.", + size, + max_state_bytes, + ) + return removed + + +def _serialized_size(state: DurableAgentState) -> int: + """Measure the state exactly as it will be persisted.""" + return len(json.dumps(state.to_dict())) + + +async def _evict_once( + history: list[DurableAgentStateEntry], + *, + serialized_size: int, + target_bytes: int, +) -> int: + """Run one eviction pass, returning how many messages were removed. + + Core already knows how to drop oldest groups to a budget while preserving system messages and + keeping tool-call groups whole, so that judgement is borrowed rather than reimplemented. + """ + candidates: list[Message] = [] + origins: list[tuple[DurableAgentStateEntry, DurableAgentStateMessage]] = [] + protected = _newest_exchange(history) + for entry, index in replayable_entries(history): + if entry in protected: + # Never evict the exchange that just happened. Core's budget fallback will drop + # everything if the budget demands it, and losing the current turn would break + # response polling and discard the result the caller is waiting for. + continue + stored = entry.messages[index] + message = cast("Message", stored.to_chat_message()) + # The budget is computed over *included* messages, so a user's own compaction exclusions + # would make an over-budget conversation look empty. Clearing them here makes the budget + # reflect what is stored. This is a detached copy, so the persisted annotation is untouched. + message.additional_properties.pop(EXCLUDED_KEY, None) + candidates.append(message) + origins.append((entry, stored)) + + if not candidates: + return 0 + + strategy = TokenBudgetComposedStrategy( + token_budget=_token_budget(candidates, serialized_size=serialized_size, target_bytes=target_bytes), + tokenizer=CharacterEstimatorTokenizer(), + # No strategies, so this goes straight to core's deterministic oldest-group eviction. + # Passing the user's strategy would satisfy the budget immediately under early stop, and + # everything it had excluded for context reasons would then be deleted. + strategies=[], + ) + await strategy(candidates) + + evicted = [ + origins[position] + for position, message in enumerate(candidates) + if message.additional_properties.get(EXCLUDED_KEY) + ] + if not evicted: + return 0 + prune_messages(history, evicted) + return len(evicted) + + +def _newest_exchange(history: list[DurableAgentStateEntry]) -> list[DurableAgentStateEntry]: + """Return the entries belonging to the most recent exchange. + + Grouped by correlation id, so a request and the response it produced are protected together. + """ + if not history: + return [] + newest = history[-1].correlation_id + if newest is None: + return [history[-1]] + return [entry for entry in history if entry.correlation_id == newest] + + +def _token_budget(candidates: list[Message], *, serialized_size: int, target_bytes: int) -> int: + """Convert a byte budget into the token budget the strategy expects. + + Serialized state is larger than the text it contains, because of keys, escaping, ids and + annotations. Rather than assume an overhead constant, the ratio is measured from the state in + hand, so a conversation of long prose and one full of tool-call metadata are both handled. + """ + content_chars = sum(len(message.text or "") for message in candidates) + ratio = (content_chars / serialized_size) if serialized_size else 1.0 + return max(int(target_bytes * ratio) // _BYTES_PER_TOKEN, 1) diff --git a/python/packages/durabletask/agent_framework_durabletask/_worker.py b/python/packages/durabletask/agent_framework_durabletask/_worker.py index 63f4fe8..051812f 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_worker.py +++ b/python/packages/durabletask/agent_framework_durabletask/_worker.py @@ -19,6 +19,8 @@ from ._async_bridge import run_agent_coroutine from ._callbacks import AgentResponseCallbackProtocol from ._entities import AgentEntity, DurableTaskEntityStateProvider +from ._retention import DEFAULT_MAX_STATE_BYTES, DEFAULT_RETENTION, RetentionMode +from ._retention import resolve_retention as _resolve_retention from ._workflows.activity import execute_workflow_activity from ._workflows.dt_context import DurableTaskWorkflowContext from ._workflows.naming import ( @@ -79,20 +81,26 @@ def __init__( worker: TaskHubGrpcWorker, callback: AgentResponseCallbackProtocol | None = None, *, - prune_history: bool = False, + retention: RetentionMode = DEFAULT_RETENTION, + max_state_bytes: int = DEFAULT_MAX_STATE_BYTES, + prune_history: bool | None = None, ): """Initialize the worker wrapper. Args: worker: The durabletask worker instance to wrap callback: Optional callback for agent response notifications - prune_history: Default retention policy for registered agents. When True, messages - that compaction excluded are physically deleted from durable state, bounding - stored size. This is lossy and off by default. + retention: Default conversation retention for registered agents. ``auto`` deletes only + under storage pressure, ``keep_all`` never deletes and lets the entity fail at the + backend limit, and ``follow_compaction`` also deletes what compaction excluded. + max_state_bytes: Budget for serialized entity state. Raise it when large payload + offload is configured on the worker and client. + prune_history: Deprecated. ``True`` maps to ``follow_compaction``. """ self._worker = worker self._callback = callback - self._prune_history = prune_history + self._retention: RetentionMode = _resolve_retention(retention, prune_history) + self._max_state_bytes = max_state_bytes self._registered_agents: dict[str, SupportsAgentRun] = {} self._workflows: dict[str, Workflow] = {} # Every workflow whose orchestration has been registered (top-level plus nested @@ -108,6 +116,7 @@ def add_agent( callback: AgentResponseCallbackProtocol | None = None, *, entity_id: str | None = None, + retention: RetentionMode | None = None, prune_history: bool | None = None, ) -> None: """Register an agent with the worker. @@ -122,8 +131,8 @@ def add_agent( entity_id: Optional identity to register the entity under instead of ``agent.name``. Workflow hosting passes the executor's ``id`` so the entity matches the identity the orchestrator dispatches to. - prune_history: Per-agent retention override. When None, the worker-level - ``prune_history`` setting is used. + retention: Per-agent retention override. When None, the worker-level setting is used. + prune_history: Deprecated. ``True`` maps to ``follow_compaction``. Raises: ValueError: If the agent doesn't have a name or is already registered @@ -146,11 +155,13 @@ def add_agent( effective_callback = callback or self._callback # Create a configured entity class using the factory + effective_retention: RetentionMode = self._retention if retention is None else retention entity_class = self.__create_agent_entity( agent, effective_callback, entity_id=registration_name, - prune_history=(self._prune_history if prune_history is None else prune_history), + retention=_resolve_retention(effective_retention, prune_history), + max_state_bytes=self._max_state_bytes, ) # Register the entity class with the worker @@ -370,7 +381,8 @@ def __create_agent_entity( callback: AgentResponseCallbackProtocol | None = None, *, entity_id: str | None = None, - prune_history: bool = False, + retention: RetentionMode = DEFAULT_RETENTION, + max_state_bytes: int = DEFAULT_MAX_STATE_BYTES, ) -> type[DurableTaskEntityStateProvider]: """Factory function to create a DurableEntity class configured with an agent. @@ -383,7 +395,8 @@ def __create_agent_entity( entity_id: Optional identity to register the entity under instead of ``agent.name`` (used by workflow hosting to key entities by executor id). - prune_history: Whether excluded messages are physically deleted from durable state. + retention: How much of the conversation durable state may discard. + max_state_bytes: Budget for serialized entity state. Returns: A new DurableEntity subclass configured for this agent @@ -401,7 +414,8 @@ def __init__(self) -> None: agent=agent, callback=callback, state_provider=self, - prune_history=prune_history, + retention=retention, + max_state_bytes=max_state_bytes, ) logger.debug( "[ConfiguredAgentEntity] Initialized entity for agent: %s (entity name: %s)", diff --git a/python/packages/durabletask/tests/test_durable_history_autoswap.py b/python/packages/durabletask/tests/test_durable_history_autoswap.py index 063a4cf..5b9253d 100644 --- a/python/packages/durabletask/tests/test_durable_history_autoswap.py +++ b/python/packages/durabletask/tests/test_durable_history_autoswap.py @@ -212,10 +212,17 @@ def test_enabled_via_registration(self) -> None: def test_entity_forwards_the_flag(self) -> None: agent = _agent() - entity = AgentEntity(agent, state_provider=_InMemoryStateProvider(), prune_history=True) + entity = AgentEntity(agent, state_provider=_InMemoryStateProvider(), retention="follow_compaction") assert _history_providers(entity.agent)[0].prune_excluded is True + def test_other_retention_modes_do_not_prune_on_write(self) -> None: + """Only ``follow_compaction`` treats a compaction exclusion as consent to delete.""" + for mode in ("auto", "keep_all"): + entity = AgentEntity(_agent(), state_provider=_InMemoryStateProvider(), retention=mode) + + assert _history_providers(entity.agent)[0].prune_excluded is False, mode + def test_explicit_provider_configuration_wins(self) -> None: """A hand-configured provider is never overridden by the registration flag.""" explicit = DurableHistoryProvider(prune_excluded=False) diff --git a/python/packages/durabletask/tests/test_retention.py b/python/packages/durabletask/tests/test_retention.py new file mode 100644 index 0000000..f65cff4 --- /dev/null +++ b/python/packages/durabletask/tests/test_retention.py @@ -0,0 +1,242 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Tests for retention (ADR-0032, "Retention"). + +Retention bounds durable entity state so an agent does not simply stop working when it reaches the +backend limit. It is a capacity concern and deliberately separate from compaction: an exclusion made +for token cost is not consent to delete the record. +""" + +import json +from datetime import datetime, timezone +from typing import Any + +from agent_framework import Message + +from agent_framework_durabletask import ( + DurableAgentState, + DurableAgentStateMessage, + DurableAgentStateRequest, + DurableAgentStateResponse, +) +from agent_framework_durabletask._retention import ( + HIGH_WATERMARK, + LOW_WATERMARK, + enforce_budget, + prunes_excluded, + resolve_retention, +) + +BUDGET = 40_000 +"""Small enough to keep these tests fast, large enough to hold a realistic conversation.""" + + +def _state(turns: int, *, chars: int = 400, excluded_before: int = 0, excluded_recent: int = 0) -> DurableAgentState: + """Build entity state with the given number of user/assistant turns. + + Args: + turns: How many exchanges to record. + chars: Size of each message's text. + + Keyword Args: + excluded_before: Mark this many leading messages as compaction-excluded, as a user's own + sliding window would. + excluded_recent: Mark this many of the most recent messages as compaction-excluded, as a + tool-result strategy can do without touching the oldest turns. + + Returns: + The populated state. + """ + state = DurableAgentState() + now = datetime.now(tz=timezone.utc) + marked = 0 + for index in range(turns): + request = DurableAgentStateRequest( + correlation_id=f"c{index}", + created_at=now, + messages=[ + DurableAgentStateMessage.from_chat_message( + Message(role="user", contents=["u" * chars], message_id=f"u{index}") + ) + ], + ) + response = DurableAgentStateResponse( + correlation_id=f"c{index}", + created_at=now, + messages=[ + DurableAgentStateMessage.from_chat_message( + Message(role="assistant", contents=["a" * chars], message_id=f"a{index}") + ) + ], + ) + for entry in (request, response): + for stored in entry.messages: + if marked < excluded_before: + stored.extension_data = {"_excluded": True, "_excluded_reason": "sliding_window"} + marked += 1 + state.data.conversation_history.extend([request, response]) + + if excluded_recent: + stored_messages = [m for entry in state.data.conversation_history for m in entry.messages] + for stored in stored_messages[-excluded_recent:]: + stored.extension_data = {"_excluded": True, "_excluded_reason": "tool_result_compaction"} + return state + + +def _size(state: DurableAgentState) -> int: + return len(json.dumps(state.to_dict())) + + +def _message_ids(state: DurableAgentState) -> list[str]: + return [m.message_id or "" for entry in state.data.conversation_history for m in entry.messages] + + +class TestRetentionModes: + """The mode decides whether an exclusion may become a deletion.""" + + def test_only_follow_compaction_prunes_on_write(self) -> None: + assert prunes_excluded("follow_compaction") is True + assert prunes_excluded("auto") is False + assert prunes_excluded("keep_all") is False + + def test_deprecated_flag_maps_onto_a_mode(self) -> None: + import warnings + + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + assert resolve_retention("auto", True) == "follow_compaction" + assert any(issubclass(w.category, DeprecationWarning) for w in caught) + + def test_unset_flag_leaves_the_mode_alone(self) -> None: + assert resolve_retention("auto", None) == "auto" + assert resolve_retention("keep_all", None) == "keep_all" + + +class TestBudgetEnforcement: + """Nothing happens until state is genuinely close to the limit.""" + + async def test_below_the_watermark_nothing_is_touched(self) -> None: + state = _state(turns=4) + before = _message_ids(state) + + removed = await enforce_budget(state, max_state_bytes=BUDGET) + + assert removed == 0 + assert _message_ids(state) == before + + async def test_over_the_watermark_evicts_to_the_low_watermark(self) -> None: + state = _state(turns=60) + assert _size(state) > BUDGET * HIGH_WATERMARK, "the fixture must start over the trigger" + + removed = await enforce_budget(state, max_state_bytes=BUDGET) + + assert removed > 0 + assert _size(state) < BUDGET * HIGH_WATERMARK, "eviction did not get back under the trigger" + + async def test_the_newest_turn_survives(self) -> None: + """Evicting the turn that just happened would defeat the point of running it.""" + state = _state(turns=60) + + await enforce_budget(state, max_state_bytes=BUDGET) + + assert _message_ids(state)[-1] == "a59" + + async def test_eviction_is_hysteretic(self) -> None: + """Evicting to just under the trigger would evict again on every following turn.""" + state = _state(turns=60) + await enforce_budget(state, max_state_bytes=BUDGET) + + second = await enforce_budget(state, max_state_bytes=BUDGET) + + assert second == 0, "a second pass evicted again immediately, so there is no headroom" + + async def test_keep_all_is_the_caller_s_decision(self) -> None: + """``keep_all`` is enforced by the entity, so the budget helper itself always acts.""" + state = _state(turns=60) + + assert await enforce_budget(state, max_state_bytes=BUDGET) > 0 + + +class TestExclusionsAreNotConsentToDelete: + """A context decision must not silently become a storage decision.""" + + async def test_a_user_s_exclusions_survive_eviction(self) -> None: + """The budget is measured over a detached copy, so stored annotations are untouched. + + Exclusions are placed on recent messages here, which a tool-result strategy does, so they + sit inside the window eviction keeps. Had the annotation itself been the criterion they + would have gone regardless of where they were. + """ + state = _state(turns=60, excluded_recent=6) + + await enforce_budget(state, max_state_bytes=BUDGET) + + surviving = [ + stored + for entry in state.data.conversation_history + for stored in entry.messages + if (stored.extension_data or {}).get("_excluded") + ] + assert surviving, "every excluded message was evicted, so exclusion was treated as consent" + assert all((s.extension_data or {}).get("_excluded_reason") == "tool_result_compaction" for s in surviving) + + async def test_eviction_is_not_limited_to_what_compaction_excluded(self) -> None: + """The budget is computed over everything stored, not just the included messages. + + A user's own window can mark almost everything excluded. If those exclusions were left in + place the strategy would see a tiny included set, conclude it was already under budget, and + evict nothing while state kept growing. + """ + state = _state(turns=60, excluded_before=110) + + removed = await enforce_budget(state, max_state_bytes=BUDGET) + + assert removed > 0, "prior exclusions hid the real size and nothing was evicted" + + +class TestSingleOversizedTurn: + """Retention cannot save a conversation whose newest turn alone exceeds the budget.""" + + async def test_the_current_turn_is_never_evicted(self) -> None: + """Core's fallback will drop everything if asked, which would lose the result being polled.""" + state = _state(turns=1, chars=BUDGET * 2) + + removed = await enforce_budget(state, max_state_bytes=BUDGET) + + assert removed == 0 + assert _message_ids(state) == ["u0", "a0"], "the turn that just ran was evicted" + + async def test_an_oversized_newest_turn_does_not_take_the_history_with_it(self) -> None: + state = _state(turns=10) + state.data.conversation_history.extend(_state(turns=1, chars=BUDGET * 2).data.conversation_history) + + await enforce_budget(state, max_state_bytes=BUDGET) + + assert _message_ids(state)[-2:] == ["u0", "a0"], "the newest exchange must survive" + + +class TestStateShape: + """Eviction must leave durable state usable.""" + + async def test_empty_entries_are_removed(self) -> None: + state = _state(turns=60) + + await enforce_budget(state, max_state_bytes=BUDGET) + + assert all(entry.messages for entry in state.data.conversation_history) + + async def test_state_still_round_trips(self) -> None: + state = _state(turns=60) + + await enforce_budget(state, max_state_bytes=BUDGET) + + restored: Any = DurableAgentState.from_dict(state.to_dict()) + assert _message_ids(restored) == _message_ids(state) + + async def test_nothing_is_evicted_from_an_empty_conversation(self) -> None: + assert await enforce_budget(DurableAgentState(), max_state_bytes=BUDGET) == 0 + + +def test_watermarks_leave_room_to_work() -> None: + """The gap between them is what stops eviction running on every turn.""" + assert 0 < LOW_WATERMARK < HIGH_WATERMARK < 1 From c852c2756a876f60ad12e3df18fa20f81f7c6d5b Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 13 Aug 2026 20:49:52 -0500 Subject: [PATCH 28/32] feat: make workflow duplicate detection survive retention, and declare what we persist Two changes that only became necessary once retention could delete messages. Duplicate detection for workflow context compared incoming ids against the ids currently in history. Retention deletes oldest-first, which removes exactly those ids, and the orchestrator re-sends them because its own conversation is never evicted. The entity would then re-record precisely what had just been deleted, and since the re-ingested volume is proportional to what was evicted, that oscillates rather than settling. Detection is now by position. The entity keeps the highest chained-conversation position it has taken from each executor, which is a handful of integers, is unaffected by deletion, and is per executor rather than global because a fan-out hands two branches the same position. Once a message is evicted the node stops seeing it, which is intended: re-ingesting evicted content defeats the eviction. The id format and its parser now live together in naming.py instead of being an inline f-string. The shared schema also under-declared what this runtime persists. messageId and extensionData are both load-bearing for compaction and neither was declared, so a .NET implementer reading the contract had no way to know they must round-trip. Nothing failed validation, because the schema permits extra properties, which is exactly why it went unnoticed. Contrary to the review comment, a strict validator would not have rejected these payloads. The real defect was silent under-documentation. Session is now described as opaque and runtime-discriminated rather than pinning Python's shape, since .NET serializes conversationId plus stateBag and Python serializes session_id, service_session_id and state. Declaring either would invalidate the other. Schema version bumped to 1.2.0. Tests validate real persisted state rather than a synthetic dict, both in unit form and against the scheduler, so the code and the contract cannot drift apart quietly again. Worth recording that message ids are assigned when history is first loaded rather than when it is written, so a single-turn conversation legitimately has none. --- .../0032-durable-thread-compaction.md | 12 +- .../agent_framework_azurefunctions/_app.py | 6 + .../agent_framework_durabletask/_constants.py | 4 + .../_durable_agent_state.py | 14 +- .../agent_framework_durabletask/_entities.py | 46 +++++-- .../agent_framework_durabletask/_retention.py | 34 ++--- .../agent_framework_durabletask/_worker.py | 10 +- .../_workflows/naming.py | 47 +++++++ .../_workflows/orchestrator.py | 15 ++- .../test_13_dt_conversation_compaction.py | 26 ++++ .../tests/test_durable_agent_state.py | 8 +- .../durabletask/tests/test_state_schema.py | 123 ++++++++++++++++++ .../tests/test_workflow_context_parity.py | 63 +++++++++ python/pyproject.toml | 2 + schemas/durable-agent-entity-state.json | 25 ++-- 15 files changed, 384 insertions(+), 51 deletions(-) create mode 100644 python/packages/durabletask/tests/test_state_schema.py diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index f4b3e5e..b054332 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -381,6 +381,14 @@ around them, but the cleaner fix is upstream. compaction state, this had to be fixed for any of this to work. This one is ours rather than core's. The Python side now serializes it. + The shared schema also under-declared what is persisted. `messageId` and `extensionData` are both + load-bearing for compaction and neither appeared in `chatMessage`, so an implementer reading the + contract had no way to know they must round-trip. Nothing would have *failed* validation, since + the schema permits extra properties, which is precisely why it went unnoticed. They are declared + now, `session` is described as an opaque runtime-discriminated payload rather than pinning + Python's shape onto .NET, and a test validates real persisted state against the schema so the two + cannot drift apart again silently. + **.NET needs the same treatment, and looks deceptively fine.** Its `DurableAgentStateMessage` already has an `ExtensionData` property, but it is `[JsonExtensionData]`, System.Text.Json's overflow bucket for *unmapped JSON properties*, not a mapping of `ChatMessage.AdditionalProperties`. @@ -456,8 +464,8 @@ Durable now projects the same conversation and delivers it to the agent entity: the request entry's messages, so it is persisted like any other conversation content and is visible to compaction. - A node that runs more than once (a cycle) receives the whole upstream conversation again, so the - entity **drops messages whose id it has already recorded**, keeping at least the latest message so - the agent always has an input. This relies on the persisted `messageId` described above. + entity **drops the part it has already recorded**, keeping at least the latest message so the + agent always has an input. **Dedup is tracked by position, not by stored identity.** Comparing against the ids currently in `ConversationHistory` breaks the moment retention evicts any of them: their ids leave the comparison diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py index 0db5be9..2450854 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py @@ -252,6 +252,7 @@ def __init__( default_callback: AgentResponseCallbackProtocol | None = None, prune_history: bool | None = None, retention: RetentionMode = DEFAULT_RETENTION, + workflow_retention: RetentionMode | None = None, max_state_bytes: int = DEFAULT_MAX_STATE_BYTES, ): """Initialize the AgentFunctionApp. @@ -279,6 +280,9 @@ def __init__( ``follow_compaction`` also deletes what compaction excluded. ``add_agent`` can override it per agent. :param max_state_bytes: Budget for serialized entity state. + :param workflow_retention: Retention for agent nodes inside hosted workflows. When None, + ``retention`` applies. Worth setting separately, since a workflow node's entity lives + for one orchestration while a standalone agent's can live indefinitely. :note: If no agents are provided, they can be added later using :meth:`add_agent`. """ @@ -300,6 +304,7 @@ def __init__( self.enable_mcp_tool_trigger = enable_mcp_tool_trigger self.default_callback = default_callback self._retention: RetentionMode = resolve_retention(retention, prune_history) + self._workflow_retention: RetentionMode | None = workflow_retention self._max_state_bytes = max_state_bytes try: @@ -437,6 +442,7 @@ def _register_workflow_primitives(self, workflow: Workflow) -> None: agent_executor.agent, callback=self.default_callback, entity_id=workflow_scoped_executor_id(workflow.name, agent_executor.id), + retention=self._workflow_retention, ) for executor in plan.activity_executors: # Set up a Functions activity trigger for each non-agent executor, scoped diff --git a/python/packages/durabletask/agent_framework_durabletask/_constants.py b/python/packages/durabletask/agent_framework_durabletask/_constants.py index 445ca60..0e25e02 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_constants.py +++ b/python/packages/durabletask/agent_framework_durabletask/_constants.py @@ -137,6 +137,10 @@ class DurableStateFields: # Serialized AgentSession: the provider state bag plus any service-issued conversation id SESSION: Final[str] = "session" + # Highest chained-conversation position ingested from each workflow executor. Survives + # retention, which identity-based duplicate detection cannot. + INGESTED_POSITIONS: Final[str] = "ingestedPositions" + class ContentTypes: """Content type discriminator values for the $type field. diff --git a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py index dbecc32..10e74db 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py +++ b/python/packages/durabletask/agent_framework_durabletask/_durable_agent_state.py @@ -330,11 +330,16 @@ class DurableAgentStateData: bag plus any service-issued conversation id. Core treats session state as durable across turns, so it is persisted here rather than discarded with the per-operation session. + ingested_positions: Highest chained-conversation position taken from each workflow + executor. A workflow re-sends the whole conversation on every visit, and comparing + against stored ids stops working once retention deletes any of them, so the mark is + kept separately. extension_data: Optional dictionary for custom metadata (not part of core schema) """ conversation_history: list[DurableAgentStateEntry] session: dict[str, Any] | None + ingested_positions: dict[str, int] | None extension_data: dict[str, Any] | None def __init__( @@ -342,6 +347,7 @@ def __init__( conversation_history: list[DurableAgentStateEntry] | None = None, extension_data: dict[str, Any] | None = None, session: dict[str, Any] | None = None, + ingested_positions: dict[str, int] | None = None, ) -> None: """Initialize the data container. @@ -349,10 +355,13 @@ def __init__( conversation_history: Initial conversation history (defaults to empty list) extension_data: Optional custom metadata session: Optional serialized ``AgentSession`` from the previous turn + ingested_positions: Highest chained-conversation position taken from each workflow + executor, used to recognize context this entity has already recorded """ self.conversation_history = conversation_history or [] self.extension_data = extension_data self.session = session + self.ingested_positions = ingested_positions def to_dict(self) -> dict[str, Any]: result: dict[str, Any] = { @@ -362,6 +371,8 @@ def to_dict(self) -> dict[str, Any]: result[DurableStateFields.EXTENSION_DATA] = self.extension_data if self.session is not None: result[DurableStateFields.SESSION] = self.session + if self.ingested_positions: + result[DurableStateFields.INGESTED_POSITIONS] = self.ingested_positions return result @classmethod @@ -370,6 +381,7 @@ def from_dict(cls, data_dict: dict[str, Any]) -> DurableAgentStateData: conversation_history=_parse_history_entries(data_dict), extension_data=data_dict.get(DurableStateFields.EXTENSION_DATA), session=data_dict.get(DurableStateFields.SESSION), + ingested_positions=data_dict.get(DurableStateFields.INGESTED_POSITIONS), ) @@ -403,7 +415,7 @@ class DurableAgentState: """ # Durable Agent Schema version - SCHEMA_VERSION: str = "1.1.0" + SCHEMA_VERSION: str = "1.2.0" data: DurableAgentStateData schema_version: str = SCHEMA_VERSION diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 6fd8244..4d3c413 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -47,6 +47,7 @@ enforce_budget, prunes_excluded, ) +from ._workflows.naming import parse_workflow_message_id logger = logging.getLogger("agent_framework.durabletask") @@ -421,31 +422,58 @@ def _capture_session(self, session: Any) -> None: self.state.data.session = payload def _drop_already_stored(self, messages: list[DurableAgentStateMessage]) -> list[DurableAgentStateMessage]: - """Filter out upstream context messages this entity has already recorded. + """Filter out chained conversation this entity has already recorded. A workflow node that runs more than once (for example in a cycle) receives the whole - upstream conversation each time. Messages carrying an id that is already in this - entity's history are dropped so the conversation is not duplicated. The final message - is always kept so the agent still receives an input. + upstream conversation each time. Without filtering it re-records all of it on every visit. + + Filtering is by **position**, not by stored identity. The obvious check, "is this id + already in my history", stops working the moment retention evicts anything: those ids + leave the comparison set, the orchestrator re-sends them because its own conversation is + never evicted, and the entity re-records exactly what was deleted. That oscillates instead + of settling. A high-water mark per producing executor is unaffected by deletion, and is + per executor rather than global because a fan-out gives two branches the same position. + + Messages without a workflow id fall back to the identity check, which is enough for them + because nothing re-delivers them. + + The final message is always kept so the agent still receives an input. """ + ingested = dict(self.state.data.ingested_positions or {}) + seen: dict[str, int] = {} + kept: list[DurableAgentStateMessage] = [] + known_ids = { stored.message_id for entry in self.state.data.conversation_history for stored in entry.messages if stored.message_id } - if not known_ids: - return messages - deduped = [m for m in messages if not m.message_id or m.message_id not in known_ids] - if not deduped and messages: + for message in messages: + marker = parse_workflow_message_id(message.message_id) + if marker is not None: + executor, position = marker + seen[executor] = max(seen.get(executor, -1), position) + if position <= ingested.get(executor, -1): + continue + elif message.message_id and message.message_id in known_ids: + continue + kept.append(message) + + for executor, position in seen.items(): + ingested[executor] = max(ingested.get(executor, -1), position) + if ingested: + self.state.data.ingested_positions = ingested + + if not kept and messages: # Keep the newest message so the agent still has an input, but drop the id it shares # with the copy already in history. Two stored messages under one id collide in the # compaction position map, so annotations and pruning would target the wrong one. repeated = messages[-1] repeated.message_id = None return [repeated] - return deduped + return kept def _find_durable_history_provider(self) -> DurableHistoryProvider | None: """Return the agent's :class:`DurableHistoryProvider`, if it is configured with one.""" diff --git a/python/packages/durabletask/agent_framework_durabletask/_retention.py b/python/packages/durabletask/agent_framework_durabletask/_retention.py index af3d112..45b2c74 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_retention.py +++ b/python/packages/durabletask/agent_framework_durabletask/_retention.py @@ -118,7 +118,7 @@ async def enforce_budget(state: DurableAgentState, *, max_state_bytes: int = DEF history = state.data.conversation_history target = int(max_state_bytes * LOW_WATERMARK) - removed = 0 + removed: list[str] = [] for attempt in range(_MAX_PASSES): # Tighten on each pass, since the byte-to-token conversion is a heuristic and a first @@ -126,30 +126,32 @@ async def enforce_budget(state: DurableAgentState, *, max_state_bytes: int = DEF evicted = await _evict_once(history, serialized_size=size, target_bytes=target >> attempt) if not evicted: break - removed += evicted + removed.extend(evicted) size = _serialized_size(state) if size < high: break if removed: logger.warning( - "[Retention] Durable state reached %d bytes of a %d budget, so %d message(s) were " - "evicted oldest-first to %d bytes. Configure retention='keep_all' to disable this, or " - "raise max_state_bytes if large payload offload is enabled.", + "[Retention] Durable state passed %d bytes of a %d budget, so %d message(s) were " + "evicted oldest-first (%s .. %s), leaving %d bytes. Set retention='keep_all' to " + "disable this, or raise max_state_bytes if large payload offload is enabled.", high, max_state_bytes, - removed, + len(removed), + removed[0], + removed[-1], size, ) elif size >= high: logger.error( "[Retention] Durable state is %d bytes against a %d budget and nothing could be " - "evicted. A single turn is likely larger than the budget itself, which retention " - "cannot resolve.", + "evicted. The newest exchange is never evicted, so a single turn larger than the " + "budget cannot be resolved by retention.", size, max_state_bytes, ) - return removed + return len(removed) def _serialized_size(state: DurableAgentState) -> int: @@ -162,8 +164,8 @@ async def _evict_once( *, serialized_size: int, target_bytes: int, -) -> int: - """Run one eviction pass, returning how many messages were removed. +) -> list[str]: + """Run one eviction pass, returning the ids of the messages removed. Core already knows how to drop oldest groups to a budget while preserving system messages and keeping tool-call groups whole, so that judgement is borrowed rather than reimplemented. @@ -187,7 +189,7 @@ async def _evict_once( origins.append((entry, stored)) if not candidates: - return 0 + return [] strategy = TokenBudgetComposedStrategy( token_budget=_token_budget(candidates, serialized_size=serialized_size, target_bytes=target_bytes), @@ -200,14 +202,14 @@ async def _evict_once( await strategy(candidates) evicted = [ - origins[position] + (position, origins[position]) for position, message in enumerate(candidates) if message.additional_properties.get(EXCLUDED_KEY) ] if not evicted: - return 0 - prune_messages(history, evicted) - return len(evicted) + return [] + prune_messages(history, [origin for _, origin in evicted]) + return [candidates[position].message_id or "" for position, _ in evicted] def _newest_exchange(history: list[DurableAgentStateEntry]) -> list[DurableAgentStateEntry]: diff --git a/python/packages/durabletask/agent_framework_durabletask/_worker.py b/python/packages/durabletask/agent_framework_durabletask/_worker.py index 051812f..1581d36 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_worker.py +++ b/python/packages/durabletask/agent_framework_durabletask/_worker.py @@ -220,6 +220,8 @@ def configure_workflow( self, workflow: Workflow, callback: AgentResponseCallbackProtocol | None = None, + *, + retention: RetentionMode | None = None, ) -> None: """Register a :class:`Workflow` for automatic orchestration. @@ -245,6 +247,9 @@ def configure_workflow( across restarts and would break durable resume). Every nested sub-workflow must likewise be named. callback: Optional callback for agent response notifications. + retention: Retention for this workflow's agent nodes. When None, the worker-level + setting is used. Worth setting separately, since a workflow node's entity lives + for one orchestration while a standalone agent's can live indefinitely. Raises: ValueError: If the workflow (or a nested sub-workflow) name is missing, @@ -291,12 +296,13 @@ def configure_workflow( for hosted in hosted_workflows: if hosted.name.casefold() in self._registered_orchestrations: continue - self._register_single_workflow(hosted, callback) + self._register_single_workflow(hosted, callback, retention) def _register_single_workflow( self, workflow: Workflow, callback: AgentResponseCallbackProtocol | None, + retention: RetentionMode | None = None, ) -> None: """Register one workflow's durable primitives (no recursion into sub-workflows). @@ -316,7 +322,7 @@ def _register_single_workflow( for agent_executor in plan.agent_executors: scoped_id = workflow_scoped_executor_id(workflow.name, agent_executor.id) if scoped_id not in self._registered_agents: - self.add_agent(agent_executor.agent, callback=callback, entity_id=scoped_id) + self.add_agent(agent_executor.agent, callback=callback, entity_id=scoped_id, retention=retention) # Register non-agent executors as durable activities, scoped by workflow name. # WorkflowExecutor nodes are intentionally not registered as activities: their diff --git a/python/packages/durabletask/agent_framework_durabletask/_workflows/naming.py b/python/packages/durabletask/agent_framework_durabletask/_workflows/naming.py index b1b9072..7780857 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_workflows/naming.py +++ b/python/packages/durabletask/agent_framework_durabletask/_workflows/naming.py @@ -31,12 +31,15 @@ "DURABLE_NAME_PREFIX", "MAX_EXECUTOR_ID_LENGTH", "SUBWORKFLOW_REQUEST_SEPARATOR", + "WORKFLOW_INPUT_EXECUTOR_ID", "is_auto_generated_workflow_name", + "parse_workflow_message_id", "qualify_subworkflow_request_id", "split_subworkflow_request_id", "validate_executor_id", "validate_workflow_name", "workflow_executor_activity_name", + "workflow_message_id", "workflow_name_from_orchestrator", "workflow_orchestrator_name", "workflow_scoped_executor_id", @@ -47,6 +50,50 @@ # ``AgentSessionId.ENTITY_NAME_PREFIX``. DURABLE_NAME_PREFIX = "dafx-" +# Identifies the workflow's own input in the conversation chained between agent nodes. It has no +# producing executor, so it carries a reserved id in that position. +WORKFLOW_INPUT_EXECUTOR_ID = "input" + +_WORKFLOW_MESSAGE_ID_PREFIX = "wf_" +_WORKFLOW_MESSAGE_ID_RE = re.compile(rf"^{_WORKFLOW_MESSAGE_ID_PREFIX}(?P.+)_(?P\d+)$") + + +def workflow_message_id(executor_id: str, position: int) -> str: + """Build the id for a message the workflow itself puts in the chained conversation. + + Core leaves ``message_id`` unset, so without this an agent node cannot tell context it has + already recorded from genuinely new input. The position is the message's index in the chained + conversation, which is fixed once the message joins it and is reproduced identically when the + orchestrator replays. + + Args: + executor_id: The node that produced the message, or ``WORKFLOW_INPUT_EXECUTOR_ID``. + position: The message's index in the chained conversation. + + Returns: + An id unique within one workflow run. + """ + return f"{_WORKFLOW_MESSAGE_ID_PREFIX}{executor_id}_{position}" + + +def parse_workflow_message_id(message_id: str | None) -> tuple[str, int] | None: + """Recover the producing executor and conversation position from a message id. + + Args: + message_id: The id to parse, if the message has one. + + Returns: + The executor id and position, or None when the id was not produced by + :func:`workflow_message_id`. + """ + if not message_id: + return None + match = _WORKFLOW_MESSAGE_ID_RE.match(message_id) + if match is None: + return None + return match.group("executor"), int(match.group("position")) + + # Separator used to qualify a nested sub-workflow's pending HITL request when it is # bubbled up to the top-level instance (one top-level addressing surface). A qualified id # is a path of ``{executorId}~{ordinal}`` hops ending in the leaf's bare request id, diff --git a/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py b/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py index 94ce56c..2dce5f3 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py +++ b/python/packages/durabletask/agent_framework_durabletask/_workflows/orchestrator.py @@ -51,8 +51,10 @@ from .context import WorkflowOrchestrationContext from .naming import ( + WORKFLOW_INPUT_EXECUTOR_ID, qualify_subworkflow_request_id, workflow_executor_activity_name, + workflow_message_id, workflow_orchestrator_name, workflow_scoped_executor_id, ) @@ -83,11 +85,6 @@ SOURCE_ORCHESTRATOR = "__orchestrator__" SOURCE_HITL_RESPONSE = "__hitl_response__" -# Identifies the workflow's own input in the conversation forwarded between agent nodes. Agent -# entities use message ids to recognize context they have already recorded, so every message the -# workflow puts in that conversation needs one. -WORKFLOW_INPUT_MESSAGE_ID = "wf_input_0" - # A WorkflowExecutor node runs its inner workflow as a durable child orchestration. # The parent wraps the node's input in SUBWORKFLOW_INPUT_KEY (defined alongside the # trust-boundary sanitizer in serialization.py) so the child orchestrator can tell a @@ -237,14 +234,18 @@ def build_agent_executor_response( full_conversation.extend(previous_message.full_conversation) elif isinstance(previous_message, str): full_conversation.append( - Message(role="user", contents=[previous_message], message_id=WORKFLOW_INPUT_MESSAGE_ID) + Message( + role="user", + contents=[previous_message], + message_id=workflow_message_id(WORKFLOW_INPUT_EXECUTOR_ID, 0), + ) ) # Core leaves message_id unset, and a node that runs more than once receives this # conversation again every time. Without an id the entity cannot tell the repeat from new # input, so it re-records the whole conversation on each visit and state grows without bound. # The position is fixed once a message joins the conversation and the orchestrator rebuilds # the same sequence on replay, so deriving the id from it is both unique and replay-safe. - assistant_message.message_id = f"wf_{executor_id}_{len(full_conversation)}" + assistant_message.message_id = workflow_message_id(executor_id, len(full_conversation)) full_conversation.append(assistant_message) return AgentExecutorResponse( diff --git a/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py index d9df72f..54777a6 100644 --- a/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py +++ b/python/packages/durabletask/tests/integration_tests/test_13_dt_conversation_compaction.py @@ -10,6 +10,8 @@ - the full conversation record is retained in storage even though the model sees less. """ +import json +from pathlib import Path from typing import Any, Protocol import pytest @@ -104,6 +106,30 @@ def test_session_is_persisted_and_scoped(self) -> None: # source_id the sample's provider keeps after the durable swap. assert "in_memory" not in slices, f"durable history slice leaked into the session: {slices}" + def test_persisted_state_matches_the_shared_schema(self) -> None: + """Real scheduler round-tripped state must satisfy the cross-language contract. + + Unit tests validate a synthetic dict. This validates what the entity actually wrote and + the scheduler actually stored, which is where drift between the two would show up. + """ + jsonschema = pytest.importorskip("jsonschema") + schema_path = Path(__file__).resolve().parents[5] / "schemas" / "durable-agent-entity-state.json" + schema = json.loads(schema_path.read_text(encoding="utf-8")) + + agent = self.agent_client.get_agent("Historian") + session = agent.create_session() + assert agent.run("Name a city.", session=session) is not None + # A second turn, because message ids are assigned when history is first loaded rather + # than when it is written. After one turn there is nothing to load and nothing to stamp. + assert agent.run("Name another.", session=session) is not None + + state = self._read_state(session.durable_session_id) + jsonschema.Draft202012Validator(schema).validate(state.to_dict()) + + # The fields compaction depends on must actually be present, not merely permitted. + stored = [m for entry in state.data.conversation_history for m in entry.messages] + assert any(m.message_id for m in stored), "no message carried an id through real storage" + def test_recent_context_survives_compaction(self) -> None: """A fact inside the retained window is still answerable after several turns.""" agent = self.agent_client.get_agent("Historian") diff --git a/python/packages/durabletask/tests/test_durable_agent_state.py b/python/packages/durabletask/tests/test_durable_agent_state.py index d3a36c9..3c78a81 100644 --- a/python/packages/durabletask/tests/test_durable_agent_state.py +++ b/python/packages/durabletask/tests/test_durable_agent_state.py @@ -156,7 +156,7 @@ class TestDurableAgentState: def test_schema_version(self) -> None: """Test that schema version is set correctly.""" state = DurableAgentState() - assert state.schema_version == "1.1.0" + assert state.schema_version == "1.2.0" def test_to_dict_serialization(self) -> None: """Test that to_dict produces correct structure.""" @@ -165,13 +165,13 @@ def test_to_dict_serialization(self) -> None: assert "schemaVersion" in data assert "data" in data - assert data["schemaVersion"] == "1.1.0" + assert data["schemaVersion"] == "1.2.0" assert "conversationHistory" in data["data"] def test_from_dict_deserialization(self) -> None: """Test that from_dict restores state correctly.""" original_data = { - "schemaVersion": "1.1.0", + "schemaVersion": "1.2.0", "data": { "conversationHistory": [ { @@ -191,7 +191,7 @@ def test_from_dict_deserialization(self) -> None: state = DurableAgentState.from_dict(original_data) - assert state.schema_version == "1.1.0" + assert state.schema_version == "1.2.0" assert len(state.data.conversation_history) == 1 assert isinstance(state.data.conversation_history[0], DurableAgentStateRequest) diff --git a/python/packages/durabletask/tests/test_state_schema.py b/python/packages/durabletask/tests/test_state_schema.py new file mode 100644 index 0000000..ffc631a --- /dev/null +++ b/python/packages/durabletask/tests/test_state_schema.py @@ -0,0 +1,123 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""The persisted state must match the shared cross-language schema. + +``schemas/durable-agent-entity-state.json`` is the contract between the Python and .NET hosting +layers. Nothing enforced it before, so fields this runtime persisted (``messageId`` and +``extensionData``, both load-bearing for context management) went undeclared and a .NET +implementer reading the schema would not have known to round-trip them. +""" + +import json +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + +import pytest +from agent_framework import Message + +from agent_framework_durabletask import ( + DurableAgentState, + DurableAgentStateMessage, + DurableAgentStateRequest, + DurableAgentStateResponse, +) + +jsonschema = pytest.importorskip("jsonschema") + +SCHEMA_PATH = Path(__file__).resolve().parents[4] / "schemas" / "durable-agent-entity-state.json" + + +@pytest.fixture(scope="module") +def schema() -> dict[str, Any]: + return json.loads(SCHEMA_PATH.read_text(encoding="utf-8")) + + +def _validate(payload: dict[str, Any], schema: dict[str, Any]) -> None: + jsonschema.Draft202012Validator(schema).validate(payload) + + +def _populated_state() -> DurableAgentState: + """Build state exercising every field this runtime persists.""" + now = datetime.now(tz=timezone.utc) + request = DurableAgentStateRequest( + correlation_id="c0", + created_at=now, + messages=[ + DurableAgentStateMessage.from_chat_message( + Message(role="user", contents=["hello"], message_id="wf_input_0") + ) + ], + ) + response = DurableAgentStateResponse( + correlation_id="c0", + created_at=now, + messages=[ + DurableAgentStateMessage.from_chat_message( + Message(role="assistant", contents=["hi"], message_id="wf_writer_1") + ) + ], + ) + # Annotations are what carry compaction state across a round-trip. + response.messages[0].extension_data = {"_excluded": True, "_excluded_reason": "sliding_window"} + + state = DurableAgentState() + state.data.conversation_history.extend([request, response]) + state.data.session = {"type": "session", "session_id": "@dafx-writer@run-1", "state": {"compaction": {}}} + state.data.ingested_positions = {"input": 0, "writer": 1} + return state + + +def test_the_schema_itself_is_valid(schema: dict[str, Any]) -> None: + jsonschema.Draft202012Validator.check_schema(schema) + + +def test_empty_state_validates(schema: dict[str, Any]) -> None: + _validate(DurableAgentState().to_dict(), schema) + + +def test_populated_state_validates(schema: dict[str, Any]) -> None: + _validate(_populated_state().to_dict(), schema) + + +def test_message_identity_and_annotations_are_declared(schema: dict[str, Any]) -> None: + """Both are load-bearing for compaction, so an implementer must be told to round-trip them.""" + properties = schema["$defs"]["chatMessage"]["properties"] + + assert "messageId" in properties + assert "extensionData" in properties + + +def test_the_ingestion_watermark_is_declared(schema: dict[str, Any]) -> None: + """It is how a repeated workflow node recognizes context it already recorded.""" + assert "ingestedPositions" in schema["$defs"]["data"]["properties"] + + +def test_session_is_left_opaque(schema: dict[str, Any]) -> None: + """The two runtimes serialize sessions differently, so the shared schema must not fix a shape. + + .NET produces ``conversationId`` plus ``stateBag``. Python produces ``session_id``, + ``service_session_id`` and ``state``. Declaring either one would make the other invalid. + """ + session = schema["$defs"]["data"]["properties"]["session"] + + assert "properties" not in session, "the schema pins one runtime's session shape" + + dotnet_shaped = { + "schemaVersion": DurableAgentState.SCHEMA_VERSION, + "data": {"conversationHistory": [], "session": {"conversationId": "abc", "stateBag": {}}}, + } + _validate(dotnet_shaped, schema) + + +def test_state_survives_a_round_trip_through_the_schema(schema: dict[str, Any]) -> None: + """Serialize, validate, restore, and confirm the compaction-critical fields came back.""" + payload = _populated_state().to_dict() + _validate(payload, schema) + + restored = DurableAgentState.from_dict(payload) + stored = restored.data.conversation_history[1].messages[0] + + assert stored.message_id == "wf_writer_1" + assert (stored.extension_data or {}).get("_excluded") is True + assert restored.data.ingested_positions == {"input": 0, "writer": 1} diff --git a/python/packages/durabletask/tests/test_workflow_context_parity.py b/python/packages/durabletask/tests/test_workflow_context_parity.py index f7a48d8..3ad74c4 100644 --- a/python/packages/durabletask/tests/test_workflow_context_parity.py +++ b/python/packages/durabletask/tests/test_workflow_context_parity.py @@ -20,6 +20,7 @@ from agent_framework_durabletask import ( AgentEntity, AgentEntityStateProviderMixin, + DurableAgentState, DurableAgentStateRequest, RunRequest, ) @@ -262,6 +263,68 @@ def _deliver_to_a(context: list[Message], correlation_id: str) -> int: assert second == 2, f"expected only the two new messages, got {second} of 5 delivered" +class TestDedupSurvivesRetention: + """Duplicate detection must not depend on the messages still being there. + + Retention deletes oldest-first, which removes exactly the ids an identity check relies on. The + orchestrator's own conversation is never evicted, so it re-sends them, and an entity comparing + against stored ids would re-record precisely what was just deleted. + """ + + def _deliver(self, entity: AgentEntity, context: list[Message], correlation_id: str) -> int: + request = RunRequest( + message=context[-1].text or "", + correlation_id=correlation_id, + context_messages=[m.to_dict() for m in context], + ) + entry = DurableAgentStateRequest.from_run_request(request) + entry.messages = entity._drop_already_stored(entry.messages) + entity.state.data.conversation_history.append(entry) + return len(entry.messages) + + def test_evicted_context_is_not_re_ingested(self) -> None: + provider = _InMemoryStateProvider() + entity = AgentEntity(_stub_agent(), state_provider=provider) + + conversation: Any = "start" + conversation = build_agent_executor_response("A", "a1", None, conversation) + conversation = build_agent_executor_response("B", "b1", None, conversation) + self._deliver(entity, list(conversation.full_conversation), "corr-1") + + # Retention deletes the oldest messages, taking their ids with them. + entity.state.data.conversation_history.clear() + + conversation = build_agent_executor_response("A", "a2", None, conversation) + conversation = build_agent_executor_response("B", "b2", None, conversation) + recorded = self._deliver(entity, list(conversation.full_conversation), "corr-2") + + assert recorded == 2, f"expected only the two new messages after eviction, got {recorded} of 5" + + def test_the_mark_is_kept_per_executor(self) -> None: + """A fan-out gives two branches the same position, so one global mark would conflate them.""" + provider = _InMemoryStateProvider() + entity = AgentEntity(_stub_agent(), state_provider=provider) + + conversation: Any = "start" + conversation = build_agent_executor_response("A", "a1", None, conversation) + conversation = build_agent_executor_response("B", "b1", None, conversation) + self._deliver(entity, list(conversation.full_conversation), "corr-1") + + marks = entity.state.data.ingested_positions or {} + assert set(marks) == {"input", "A", "B"}, f"expected a mark per producing executor, got {marks}" + + def test_the_mark_round_trips_through_durable_state(self) -> None: + provider = _InMemoryStateProvider() + entity = AgentEntity(_stub_agent(), state_provider=provider) + + conversation: Any = build_agent_executor_response("A", "a1", None, "start") + self._deliver(entity, list(conversation.full_conversation), "corr-1") + entity.persist_state() + + restored = DurableAgentState.from_dict(provider._get_state_dict()) + assert restored.data.ingested_positions == entity.state.data.ingested_positions + + class TestCoreSessionIdentity: """The id handed to core must identify one entity, not one workflow run.""" diff --git a/python/pyproject.toml b/python/pyproject.toml index 3aba926..879cd2f 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -47,6 +47,8 @@ dev = [ ] test = [ "azure-monitor-opentelemetry", + # Validates that persisted entity state matches the shared cross-language schema. + "jsonschema", "mcp[ws]", "redis", # Model provider packages used by the sample workers that the integration diff --git a/schemas/durable-agent-entity-state.json b/schemas/durable-agent-entity-state.json index 1f5e081..5f6f907 100644 --- a/schemas/durable-agent-entity-state.json +++ b/schemas/durable-agent-entity-state.json @@ -142,7 +142,15 @@ "type": "array", "items": { "$ref": "#/$defs/chatContentItem" } }, - "createdAt": { "type": "string", "format": "date-time", "description": "When this message was created (RFC 3339)." } + "createdAt": { "type": "string", "format": "date-time", "description": "When this message was created (RFC 3339)." }, + "messageId": { + "type": "string", + "description": "Stable identity for this message. Context management reconciles its results back onto stored messages by this value, so an implementation that drops it on round-trip silently loses compaction state. Assigned by the runtime when the producer left it unset." + }, + "extensionData": { + "type": "object", + "description": "Message-level metadata, carrying the annotations context management writes (for example exclusion markers and summary markers). Must round-trip: discarding it loses compaction state rather than failing loudly." + } }, "required": ["role"] }, @@ -203,15 +211,12 @@ }, "session": { "type": "object", - "description": "Serialized agent session carried between turns: the per-provider state bag and any service-issued conversation id. The agent's own history provider slice is excluded, since conversationHistory is the record of truth.", - "properties": { - "session_id": { "type": "string" }, - "service_session_id": { "type": ["string", "null"] }, - "state": { - "type": "object", - "description": "Provider state keyed by context provider source id." - } - } + "description": "Serialized agent session carried between turns, holding the per-provider state bag and any service-issued conversation id. The shape is the hosting runtime's own and is deliberately not fixed here: .NET serializes conversationId plus stateBag, Python serializes session_id, service_session_id and state. Treat it as opaque and discriminate on the properties present. The agent's own history provider slice is excluded, since conversationHistory is the record of truth." + }, + "ingestedPositions": { + "type": "object", + "description": "Highest chained-conversation position this entity has taken from each workflow executor, keyed by executor id. A workflow re-sends the whole conversation on every visit, so this is what lets a repeated node recognize context it already recorded. Kept separately from the messages because retention may delete them.", + "additionalProperties": { "type": "integer", "minimum": 0 } } } } From 15f77b4c6fc4a3da51fbf81d44891a5b71460f4e Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 13 Aug 2026 21:17:23 -0500 Subject: [PATCH 29/32] test: drive retention through the real entity, and document the modes The existing retention tests call enforce_budget directly, which proves the eviction algorithm but not that anything calls it. These drive a real Agent through AgentEntity.run for twenty turns against a small budget, so the assertion covers the wiring: session load, durable history, agent run, save, enforcement, persist, reload. The keep_all case is the control. It asserts state grows past the limit on the same run, so the bounded assertion cannot pass because the conversation was small. The trimmed assertion exists for the same reason, since a bound holds trivially if nothing accumulates. The fake client answers based on the question rather than a call counter, because the entity retries through a non-streaming fallback and a counter would drift. It returns a ResponseStream so the streaming path the entity actually prefers is the one under test. Also covers the deprecation shim end to end, which was only tested at the resolve_retention level, and the sample README described prune_history, which is no longer how this is configured. --- .../durabletask/tests/test_retention.py | 119 +++++++++++++++++- .../13_conversation_compaction/README.md | 21 +++- 2 files changed, 134 insertions(+), 6 deletions(-) diff --git a/python/packages/durabletask/tests/test_retention.py b/python/packages/durabletask/tests/test_retention.py index f65cff4..a8c914a 100644 --- a/python/packages/durabletask/tests/test_retention.py +++ b/python/packages/durabletask/tests/test_retention.py @@ -8,12 +8,23 @@ """ import json +from collections.abc import AsyncIterator from datetime import datetime, timezone -from typing import Any - -from agent_framework import Message +from typing import Any, cast + +from agent_framework import ( + Agent, + BaseChatClient, + ChatResponse, + ChatResponseUpdate, + Content, + Message, + ResponseStream, +) from agent_framework_durabletask import ( + AgentEntity, + AgentEntityStateProviderMixin, DurableAgentState, DurableAgentStateMessage, DurableAgentStateRequest, @@ -111,6 +122,25 @@ def test_unset_flag_leaves_the_mode_alone(self) -> None: assert resolve_retention("auto", None) == "auto" assert resolve_retention("keep_all", None) == "keep_all" + def test_the_deprecated_flag_still_works_through_the_worker(self) -> None: + """Callers who set prune_history=True must keep the behavior they had.""" + import warnings + + from agent_framework_durabletask import DurableAIAgentWorker + + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + worker = DurableAIAgentWorker(cast(Any, object()), prune_history=True) + + assert worker._retention == "follow_compaction" + assert any(issubclass(w.category, DeprecationWarning) for w in caught) + + def test_the_default_is_auto(self) -> None: + """Which is the deliberate behavior change: previously nothing bounded storage.""" + from agent_framework_durabletask import DurableAIAgentWorker + + assert DurableAIAgentWorker(cast(Any, object()))._retention == "auto" + class TestBudgetEnforcement: """Nothing happens until state is genuinely close to the limit.""" @@ -240,3 +270,86 @@ async def test_nothing_is_evicted_from_an_empty_conversation(self) -> None: def test_watermarks_leave_room_to_work() -> None: """The gap between them is what stops eviction running on every turn.""" assert 0 < LOW_WATERMARK < HIGH_WATERMARK < 1 + + +class _VerboseClient(BaseChatClient): + """A client whose answers are long enough to reach the budget in a handful of turns.""" + + def __init__(self, *, reply_chars: int = 4_000) -> None: + super().__init__() + self._reply_chars = reply_chars + + def _inner_get_response(self, *, messages: Any, stream: bool, options: Any, **kwargs: Any) -> Any: + del options, kwargs + # Keyed off the question rather than a counter, so a retried call answers the same thing. + asked = next( + (m.text for m in reversed(list(messages)) if str(getattr(m.role, "value", m.role)) == "user"), + "?", + ) + body = f"answering:{asked} " + ("x" * self._reply_chars) + if stream: + + async def _updates() -> AsyncIterator[ChatResponseUpdate]: + yield ChatResponseUpdate(role="assistant", contents=[Content.from_text(text=body)]) + + return ResponseStream(_updates(), finalizer=ChatResponse.from_updates) + + async def _response() -> ChatResponse: + return ChatResponse(messages=[Message(role="assistant", contents=[body])]) + + return _response() + + +class _EntityState(AgentEntityStateProviderMixin): + def __init__(self) -> None: + self._state_dict: dict[str, Any] = {} + + def _get_state_dict(self) -> dict[str, Any]: + return self._state_dict + + def _set_state_dict(self, state: dict[str, Any]) -> None: + # The real provider hands state to the SDK, which serializes it eagerly. + json.dumps(state) + self._state_dict = state + + def _get_session_id_from_entity(self) -> str: + return "retention-e2e" + + +class TestTheWholeLoopStaysUnderBudget: + """Drives the real entity, not just enforce_budget, because the value is in the wiring.""" + + LIMIT = 60_000 + TURNS = 20 + + async def _drive(self, **entity_kwargs: Any) -> tuple[_EntityState, list[str]]: + client = _VerboseClient() + agent = Agent(client=cast(Any, client), name="verbose") + provider = _EntityState() + entity = AgentEntity(agent, state_provider=provider, **entity_kwargs) + + replies: list[str] = [] + for turn in range(self.TURNS): + result = await entity.run({"message": f"question {turn}", "correlationId": f"corr-{turn}"}) + replies.append(result.text) + return provider, replies + + async def test_state_stays_bounded_across_many_turns(self) -> None: + provider, _ = await self._drive(max_state_bytes=self.LIMIT) + assert len(json.dumps(provider._get_state_dict())) <= self.LIMIT + + async def test_every_turn_still_gets_its_own_answer(self) -> None: + """Eviction must not disturb the response the caller is waiting on.""" + _, replies = await self._drive(max_state_bytes=self.LIMIT) + assert [r.split(" x")[0] for r in replies] == [f"answering:question {i}" for i in range(self.TURNS)] + + async def test_history_is_actually_trimmed_not_just_small(self) -> None: + """Without this the bounded assertion above could pass for the wrong reason.""" + provider, _ = await self._drive(max_state_bytes=self.LIMIT) + kept = len(DurableAgentState.from_dict(provider._get_state_dict()).data.conversation_history) + assert 0 < kept < self.TURNS * 2 + + async def test_keep_all_lets_it_grow_past_the_limit(self) -> None: + """Proves the run is genuinely over budget, so the bounded case is a real result.""" + provider, _ = await self._drive(retention="keep_all", max_state_bytes=self.LIMIT) + assert len(json.dumps(provider._get_state_dict())) > self.LIMIT diff --git a/python/samples/13_conversation_compaction/README.md b/python/samples/13_conversation_compaction/README.md index e54e5c1..66882f1 100644 --- a/python/samples/13_conversation_compaction/README.md +++ b/python/samples/13_conversation_compaction/README.md @@ -31,9 +31,24 @@ Registering that agent with the durable runtime changes nothing about how you co - **Context stays bounded.** Only the messages the strategy keeps are sent to the model, so a long conversation does not grow the per-turn context without limit. -The full conversation remains in durable storage, and compaction bounds what the *model* sees. To -also bound what is *stored*, opt in at registration with `add_agent(agent, prune_history=True)`, -which is lossy and therefore off by default. +The full conversation remains in durable storage, and compaction bounds what the *model* sees. + +### Retention: what durable storage is allowed to discard + +Compaction and retention answer different questions. Compaction decides what the model should read. +Retention decides what durable state can afford to hold, and an exclusion made to save tokens is not +consent to delete the record. Set it at registration with `add_agent(agent, retention=...)`, or +app-wide on the worker. + +| Mode | Behavior | +| --- | --- | +| `auto` (default) | Deletes only when state approaches the backend limit, and only enough to get back under it. Nothing changes for a conversation that never gets close. | +| `keep_all` | Never deletes. The entity may reach the limit and fail. Choose this when the complete record matters more than staying available. | +| `follow_compaction` | Also deletes whatever compaction excluded, every turn. The most aggressive, and the old `prune_history=True`. | + +`auto` exists because the alternative is an agent that simply stops working mid-conversation, with +no warning. It evicts oldest-first, keeps system messages and tool-call groups intact, never touches +the exchange that just completed, and logs what it removed. ### Client-side vs service-managed history From 428aef68055e1675e47e922c8de09ec519aee8c8 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Thu, 13 Aug 2026 21:31:24 -0500 Subject: [PATCH 30/32] docs: settle whether blob offload can reach the Durable Functions Python path It cannot, in either version, and the ADR was guessing. 1.x is what this package pins, and it does not depend on the durabletask SDK at all. The host extension owns persistence, so there is no Python-side seam to configure and no reference to payloads anywhere in the package. The earlier wording called this dotnet-only support, which described the symptom but not the reason. The 2.x preview does depend on durabletask, and DurableFunctionsWorker subclasses TaskHubGrpcWorker, whose constructor takes payload_store. But its own __init__ takes no parameters and hardcodes the super call, and the client takes only a connection string. Neither forwards kwargs, so the capability is inherited and unreachable. Its comment even lists the payload store among the base state it relies on. This matters for the decision rather than being trivia. Blob offload stays the first capacity answer on the durabletask path, where the caller builds the worker and client and can pass a payload store with no code from us. On Functions it is not an answer at present, which is why retention has to exist rather than being deferred to a ceiling raise. Recorded as gap 6 with the upstream ask. The outstanding retention note was also stale, since all three modes are now built and tested. --- .../0032-durable-thread-compaction.md | 38 +++++++++++++++---- 1 file changed, 30 insertions(+), 8 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index b054332..56bdc7b 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -215,7 +215,13 @@ likely to hit the limit. It would leave every other configuration exactly as exp changing behavior for users who were never at risk. **Why not rely on blob offload alone.** It raises the ceiling roughly tenfold and does not remove it. -It is preview, it needs a storage account, and its Functions support is currently .NET only. +It is preview, it needs a storage account, and on the Durable Functions Python path it is currently +unreachable. `azure-functions-durable` 1.x does not depend on the durabletask SDK at all, because the +host extension owns persistence, so there is no Python-side seam to configure. The 2.x preview does +depend on `durabletask>=1.9.0`, whose worker and client both accept `payload_store`, but its +`DurableFunctionsWorker.__init__` takes no parameters and hardcodes the `super().__init__` call, and +`DurableFunctionsClient.__init__` takes only a connection string. Neither forwards `**kwargs`, so the +inherited capability cannot be reached without an upstream change (gap 6). **Service-managed storage** is out of scope, mirroring ADR-0019. When the model provider owns the conversation the client holds no history to compact. See "Service-managed conversations". @@ -260,12 +266,13 @@ upstream conversation. **Outstanding.** Not covered yet. -- **Retention.** The `auto` and `keep_all` modes are designed but not built. Only the behavior now - called `follow_compaction` exists, under its former name. Nothing measures state size today, so an - entity approaching the scheduler limit gets no warning and no relief. +- **Retention.** All three modes are built and covered, including an end-to-end test that drives a + real agent through the entity for twenty turns against a small budget, with `keep_all` as the + control proving the same run exceeds it. What is not yet covered is a conversation crossing the + real scheduler limit against a real backend, rather than a lowered one in-process. - The .NET realization and its schema parity (gap 3), and the .NET compaction-state blocker (gap 4). -- Blob offload (Option 7) against a real scheduler, and whether the Durable Functions Python path can - reach it at all. +- Blob offload (Option 7) against a real scheduler. Whether the Durable Functions Python path can + reach it is now answered: it cannot, in either 1.x or the 2.x preview (gap 6). - An external history provider storing history beyond the built-in state-size limit. - Idempotency of an LLM-based reducer across simulated entity retries. @@ -302,8 +309,9 @@ The full argument is in **Decision Outcome** above. This is the summary. - **Option 7 - Blob offload.** Raises the ceiling roughly tenfold with no data loss, needs no code from this layer since the payload store is passed to the worker and client the caller already builds, and mirrors what the Azure Storage backend does internally. But it is preview, needs a - storage account, does not remove the ceiling, and its Durable Functions support is .NET only - today. **Adopted as the first capacity answer, ahead of any deletion.** + storage account, does not remove the ceiling, and is unreachable on the Durable Functions Python + path in both 1.x and the 2.x preview (gap 6). **Adopted as the first capacity answer on the + durabletask path, ahead of any deletion.** ## Cross-Cutting Design Details @@ -431,6 +439,20 @@ around them, but the cleaner fix is upstream. rather than live. It is recorded because the symptom would be missing annotations rather than an error. +6. **Blob offload is unreachable on the Durable Functions Python path.** Not a core gap but an + upstream one, recorded here because it is what forces this layer to own a capacity answer at all. + `azure-functions-durable` 1.x, which this package pins (`>=1.3.1,<2`), does not depend on the + durabletask SDK, since the host extension owns persistence. There is no Python-side seam to + configure and the word payload does not appear in the package. The 2.x preview (`2.0.0b1`, + `2.0.0b2`, both requiring Python 3.13+) does depend on `durabletask>=1.9.0`, and + `DurableFunctionsWorker` subclasses `TaskHubGrpcWorker`, whose constructor accepts + `payload_store`. But `DurableFunctionsWorker.__init__` takes no parameters and hardcodes its + `super().__init__` arguments, and `DurableFunctionsClient.__init__` takes only a connection + string. Neither forwards `**kwargs`, so the inherited capability is unreachable. The durabletask + path has no such problem, because the caller constructs the worker and client and can pass + `payload_store` directly. *Upstream fix:* expose `payload_store` on `DurableFunctionsWorker` and + `DurableFunctionsClient`. + Two further core gaps are recorded with the decisions they affect: the process-local **state type registry** (see "The session is persisted, not just its conversation id") and the absence of a public way to ask whether **the service owns history for a run** (see "Service-managed conversations"). Both From d2ef652d5fd061bae840f12d7bbb2760b22678f5 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 14 Aug 2026 15:36:29 -0500 Subject: [PATCH 31/32] docs: tighten ADR 0032 around the actual decision Separate agent compaction, workflow context projection, and durable retention so each mechanism has one clear responsibility. Correct the claims about external history providers, prospective .NET retention, service-managed context, and the workflow context seam. Keep the schema, .NET compaction-state, blob offload, watermark, and position-dedup findings that answer review feedback. Remove the duplicate option recap, implementation detours, and the out-of-scope TTL plan. The ADR drops from 7,220 words to 3,914 while retaining the decision, tradeoffs, validation, and unresolved gaps. --- .../0032-durable-thread-compaction.md | 663 +++++------------- 1 file changed, 188 insertions(+), 475 deletions(-) diff --git a/docs/decisions/0032-durable-thread-compaction.md b/docs/decisions/0032-durable-thread-compaction.md index 56bdc7b..3aa4553 100644 --- a/docs/decisions/0032-durable-thread-compaction.md +++ b/docs/decisions/0032-durable-thread-compaction.md @@ -1,5 +1,4 @@ --- -# These are optional elements. Feel free to remove any of them. status: proposed contact: ahmedmuhsin date: 2026-07-27 @@ -10,9 +9,8 @@ informed: # Thread Compaction for Durable Agents and Workflows -> **How to read this.** Everything through "Pros and Cons of the Options" is the proposed design. -> Everything after it records how that design was prototyped in Python and what the prototype -> surfaced. **.NET is not implemented yet.** +> **How to read this.** The decision comes first. The later sections record the Python prototype +> and the gaps it exposed. **.NET is not implemented yet.** > > **Naming.** .NET's `ChatHistoryProvider` and Python's `HistoryProvider` are the same concept. The > decision sections use the .NET name, and the implementation sections use the Python one. @@ -23,8 +21,8 @@ Long-running **durable** agents and workflows accumulate conversation history in storage and replay it on every turn. Durable agents persist a full `ConversationHistory` in entity state (`AgentEntity` → `DurableAgentState`). Durable workflows persist inter-executor messages (`AgentExecutor.full_conversation`) as checkpointed envelopes. An in-memory agent keeps its -history in process RAM, where it disappears when the process recycles. This history is instead -**persisted, reloaded every turn, and permanent**. +history in process RAM, where it disappears when the process recycles. Durable history instead +survives restarts and is reloaded on later turns. It helps to separate **three distinct pressures**, because they have different owners. @@ -32,7 +30,7 @@ It helps to separate **three distinct pressures**, because they have different o | --- | --- | --- | --- | | **Context window**, the model's max input per call | the model | **Yes**, identical in core and durable | Compaction (in-run filter) | | **Token cost / latency**, resending history each turn | tokens billed / round-trip | **Yes**, same mechanism | Compaction (in-run filter) | -| **Storage capacity**, the cumulative persisted state | backend state-size limit | **No**, durable-only | Storage backend (built-in limit or external store) | +| **Storage capacity**, the cumulative persisted state | backend state-size limit | **No**, durable-only | Backend offload and durable retention | The first two are per-operation and identical in both runtimes. The third is cumulative. `ConversationHistory` is one blob appended to every turn and re-persisted whole, so it is bounded by @@ -43,9 +41,8 @@ backend stops working at the limit and the other degrades toward it, while a cor only by RAM and resets on restart. **Storage capacity is an infrastructure concern, not a context-window concern.** It is relieved -first by raising the ceiling (blob offload, an external store) and only then by deleting. The two -are kept separate throughout this document, because a tool for bounding what the model reads is not -a tool for bounding what the backend holds. +first by raising the ceiling where blob offload is available and only then by deleting. A tool for +bounding what the model reads is not a tool for bounding what the backend holds. Core MAF already has a compaction system ([ADR-0019](https://github.com/microsoft/agent-framework/blob/main/docs/decisions/0019-python-context-compaction-strategy.md), .NET `Microsoft.Agents.AI.Compaction`, Python `agent_framework._compaction`) with **two hooks**. @@ -69,11 +66,11 @@ Both the in-run filter's incremental state and the store reducer were thrown awa The goal is **configuration parity**: a user's core compaction config must carry over to a durable entity or workflow **unchanged**, reusing the same strategies and hooks on the durable runtime, -without a parallel durable-only API. +without a parallel durable compaction API. Storage retention is a separate deployment policy because +it has no meaning for an in-memory agent. -**How should core compaction (both hooks) be reused on the durable runtime, in both .NET and -Python, so that the model input is bounded identically to core and the persisted store can be -bounded when the user opts into it?** +**How should core compaction be reused on the durable runtime, in both .NET and Python, so the same +agent configuration bounds model input and the persisted store can be bounded separately?** ## Decision Drivers @@ -83,8 +80,8 @@ bounded when the user opts into it?** - **Reuse existing core hooks.** Do not reinvent triggers, strategies or grouping. Reuse the in-run filter and the store reducer. - **Separate storage capacity from context management.** Bound the model input with compaction - (parity with core), and relieve persisted-storage capacity with infrastructure (backend limits, - external stores) rather than by silently trimming. + (parity with core), raise backend capacity where possible, and use observable deletion only as a + fallback. - **Deleting is a last resort, and never silent.** Entity state is a state bag, not an immutable system of record, so deleting from it is legitimate. But deletion should happen only when capacity demands it, should remove no more than capacity demands, and should always be observable. @@ -98,133 +95,86 @@ bounded when the user opts into it?** which is a plain callable rather than the compaction system. That difference is real and is called out rather than papered over. - **Defer when the model provider owns the conversation.** When the chat client keeps history on the - service (a `ConversationId` or `service_session_id` is set), the client holds nothing to compact. - "Service" here means the model provider. The durable entity is not the service in this sense, even - though it is also storage someone else manages. + service, the client holds no history for core compaction. "Service" here means the model provider, + not the durable entity. The entity's own durable record still follows its retention policy. ## Considered Options -- **Option 1, in-run filter only.** Register the core `CompactionProvider` / `compaction_strategy` - on the inner agent and change nothing else in the durable layer. +- **Option 1, in-run filter only (rejected).** Reuse the agent's core compaction without changing + durable history. This bounds model input but not persisted state. - **Option 2, bespoke pre-write compaction in the agent entity.** Add durable-specific code that - compacts `ConversationHistory` inside the entity operation before checkpoint. -- **Option 3, on-storage maintenance compaction.** Compact persisted history from a separate - entity signal or operation, decoupled from the request path. -- **Option 4, workflow-level compaction hook.** Apply a strategy at the `AgentExecutor` - `context_mode` / `context_filter` boundary that governs the `full_conversation` chained between - agent executors. -- **Option 5, auto-derive a durable store reducer.** When only an in-run filter is configured, - automatically derive a lossy store reducer (`strategy.AsChatReducer()`) so durable storage is - bounded even without an explicit reducer. + compacts `ConversationHistory` inside the entity operation before checkpoint. Rejected because it + duplicates core's store-reducer behavior. +- **Option 3, on-storage maintenance compaction (deferred).** Compact persisted history from a + separate entity operation. This may suit expensive summarization but does not prevent in-turn + growth. +- **Option 4, workflow context projection (chosen).** Honor `AgentExecutor.context_mode` and + `context_filter` for the `full_conversation` chained between executors. +- **Option 5, auto-derive a durable store reducer (rejected as default).** Derive a lossy reducer + from a configured in-run strategy. The explicit equivalent is `follow_compaction`. The default + must also protect agents with no compaction strategy. - **Option 6, durable store as a `ChatHistoryProvider` (chosen).** Back the durable entity's persisted conversation with a core `ChatHistoryProvider` implementation, so both core hooks apply on the durable runtime from the user's unchanged configuration. The in-run filter runs in the - agent pipeline (L1), and a user-configured reducer or strategy bounds the store (L2, opt-in). The - same seam makes external storage backends (Cosmos, Valkey, blob) pluggable for capacity. -- **Option 7, offload large payloads to blob storage.** Raise the ceiling instead of reducing the - content, using the Durable Task Scheduler [large payload + agent pipeline (L1), and a user-configured reducer or strategy can bound the durable store (L2, + opt-in). External history providers also rejoin the context pipeline, but the entity still keeps + its own conversation record, so they do not currently remove the need for retention. +- **Option 7, offload large payloads to blob storage (chosen where available).** Raise the ceiling + instead of reducing content, using the Durable Task Scheduler [large payload extension](https://learn.microsoft.com/azure/durable-task/scheduler/durable-task-scheduler-large-payloads). Non-lossy, and the same technique the Azure Storage backend has always used internally. ## Decision Outcome Chosen option: **Option 6, express durable conversation storage as a core `ChatHistoryProvider`**, -combined with the workflow hook (Option 4). This makes core's two compaction hooks apply on the -durable runtime with **no config change**, and cleanly separates context management from storage -capacity. +combined with workflow context projection (Option 4). The two solve different surfaces. -Compaction applies at **three layers**, mapped directly onto the core hooks. Retention, described -below, is a fourth and separate concern: it bounds storage and never touches the model input. - -| Layer | Core mechanism reused | Lossy? | Role | -| --- | --- | --- | --- | -| **L1, in-run filter** | `CompactionProvider` in the agent pipeline | No | Always-on. Bounds the **model input** (context window, token cost). Identical to core. | -| **L2, store reducer** | the store-rewrite hook applied to the durable provider's store | Yes | **Opt-in** (`follow_compaction`). Bounds the **persisted store** from the user's own strategy. Python only today, since the hook is bound to session state upstream and .NET cannot persist its compaction state without duplicating the transcript (see "Core Interface Gaps"). | -| **L3, workflow hook** | the same strategy at the `AgentExecutor` `context_filter` seam | Yes | Bounds the inter-executor `full_conversation`. A plainer seam than L1 and L2. | - -**Two accumulation surfaces.** - -| Surface | Where it accumulates | Covered by | +| Surface | Mechanism | Behavior | | --- | --- | --- | -| **In-agent** | the agent's model input, and the persisted `AgentEntity` store | L1 for the model input, retention for the store, L2 when opted into | -| **Inter-executor (workflow)** | `AgentExecutor.full_conversation`, checkpointed as envelopes | L3 | +| **L1, agent context** | The user's configured core `CompactionProvider` / `compaction_strategy` | Non-lossy projection of model input. The same agent configuration works durably. | +| **L2, eager store pruning** | Core compaction annotations plus `retention="follow_compaction"` | Opt-in deletion of messages the user's strategy excluded. Python only today because .NET is blocked by duplicated compaction state (gap 4). | +| **L3, workflow context** | Existing `AgentExecutor.context_mode` / `context_filter` projection | Controls `full_conversation` passed between executors. This is not a core compaction hook. | +| **Capacity fallback** | Durable retention | Bounds entity state independently of whether compaction is configured. | -**Strict parity for context, capacity handled separately.** Durable honors exactly the compaction -hooks the user configured, so the model input is identical in both runtimes. It does **not** infer a -storage policy from a context policy: an exclusion means "do not send this to the model", never -"this is safe to delete". Those are two of the three pressures above and conflating them would let a -token-cost decision quietly destroy records the user never agreed to lose. +This gives agent-level **configuration parity**, not byte-for-byte parity in every workflow cycle. +Durable workflow nodes intentionally deduplicate repeated upstream context before persisting it. The +L3 section explains the measured difference. -Capacity is therefore its own axis, with three answers applied in order. - -1. **Raise the ceiling first.** Blob offload (Option 7) or an external provider. Non-lossy. -2. **Honor an explicit retention choice.** `follow_compaction` is the user authorizing exclusion to - mean deletion (Option 5, in opt-in form). -3. **Evict as a last resort.** Under storage pressure, delete the minimum needed to stay alive. +Capacity is handled in this order: raise the ceiling non-lossily where blob offload is available, +honor an explicit `follow_compaction` choice, then evict under pressure. An exclusion normally means +only "do not send this to the model". It means "delete this" only under `follow_compaction`. ### Retention -One setting, because a single question ("who deleted my message?") should have a single answer. - | Mode | Behavior | | --- | --- | -| `keep_all` | Never delete. The entity may reach the backend limit and fail. The honest choice when the complete record matters more than availability. | -| `auto` **(default)** | Delete only under storage pressure, and only down to the low watermark. | -| `follow_compaction` | Delete whatever compaction excluded, every turn. The previous `prune_history=True`. | - -**How `auto` works.** After the turn is recorded and before the state is persisted, the entity -serializes the state and measures it. Under the high watermark, nothing happens. Over it, the entity -builds a detached view of the stored messages **with context exclusions cleared**, hands it to core's -`TokenBudgetComposedStrategy` with no strategies of its own, and deletes whatever that marks. - -Each part earns its place. - -- **The entity triggers it, not the history provider.** `AgentEntity` appends to - `ConversationHistory` in every configuration, including external providers, service-managed agents - and agents with no context pipeline. A trigger inside the provider would protect only the - configurations that already have `follow_compaction` available, and miss the ones with no other - mitigation. -- **Exclusions are cleared on the detached view.** The strategy budgets over *included* messages, so - leaving a user's exclusions in place makes an over-budget conversation look empty and nothing is - evicted. Clearing them makes the budget reflect what is stored. The stored annotations are - untouched, so the user's context decisions survive. -- **No strategies are passed to the budget strategy.** With `early_stop`, a configured sliding window - would satisfy the budget immediately and everything it had excluded would be deleted, which is the - over-deletion this design exists to avoid. An empty strategy list goes straight to core's - deterministic oldest-group eviction, which preserves system messages and keeps tool-call groups - intact. -- **Deletion reuses the existing prune path**, which removes by identity and drops entries left - empty. No second deletion mechanism exists. -- **No summarization.** A model call on the request path re-runs on retry and can diverge. Eviction - is deterministic. - -**Values.** `max_state_bytes` defaults to `1_048_576`, the scheduler limit, and should be raised when -blob offload is configured. The high watermark is `0.85` and the low watermark `0.70`. The gap is -hysteresis: evicting to just under the trigger would evict again every subsequent turn. `0.85` rather -than `0.90` because the budget is approximate twice over, once in the byte-to-token estimate and once -because reasoning content is stripped from the candidate view. The byte budget converts to a token -budget using the ratio of content characters to serialized bytes measured on the spot, rather than a -guessed overhead constant. - -Measuring costs about 8 ms on a conversation at the 1 MB limit, against a turn dominated by a model -call, and `to_dict()` already runs on every persist regardless. +| `keep_all` | Never delete. The entity may reach the backend limit and fail. | +| `auto` **(default)** | Delete only under storage pressure, targeting the low watermark. | +| `follow_compaction` | Delete whatever compaction excluded every turn, then use the same pressure eviction as `auto` if the remaining state is still too large. | + +**How pressure eviction works.** After the turn is recorded and before the state is persisted, the +entity measures its serialized state. `auto` uses only this path. `follow_compaction` uses it after +eager pruning. Below the high watermark, nothing happens. Above it, the entity targets the low +watermark using detached message copies with existing exclusions cleared and +`TokenBudgetComposedStrategy(strategies=[])`. Clearing exclusions makes the budget reflect what is +stored, while the empty strategy list bypasses the user's context policy and uses core's +deterministic oldest-group fallback. System messages, atomic tool groups, and the newest exchange +are protected. The entity remeasures after each pass and logs what it removes. + +`max_state_bytes` defaults to `1_048_576`. High and low watermarks of `0.85` and `0.70` provide +hysteresis and room for estimation error. Measuring a 1 MB prototype state took about 8 ms. **Why not simply reduce the store by default.** A default-on reducer only helps agents that already -configured compaction, because nothing else marks messages excludable, and those are the agents least -likely to hit the limit. It would leave every other configuration exactly as exposed as before while -changing behavior for users who were never at risk. +configured compaction, because nothing else marks messages excludable. It would leave every other +configuration exposed while changing behavior for only a subset of users. **Why not rely on blob offload alone.** It raises the ceiling roughly tenfold and does not remove it. -It is preview, it needs a storage account, and on the Durable Functions Python path it is currently -unreachable. `azure-functions-durable` 1.x does not depend on the durabletask SDK at all, because the -host extension owns persistence, so there is no Python-side seam to configure. The 2.x preview does -depend on `durabletask>=1.9.0`, whose worker and client both accept `payload_store`, but its -`DurableFunctionsWorker.__init__` takes no parameters and hardcodes the `super().__init__` call, and -`DurableFunctionsClient.__init__` takes only a connection string. Neither forwards `**kwargs`, so the -inherited capability cannot be reached without an upstream change (gap 6). +It is also unreachable on the Durable Functions Python path today (gap 6). Retention is therefore +the fallback that works on every host. -**Service-managed storage** is out of scope, mirroring ADR-0019. When the model provider owns the -conversation the client holds no history to compact. See "Service-managed conversations". +**Service-managed model context** is outside compaction scope, mirroring ADR-0019. When the model +provider owns the conversation, the client holds no history to compact. Entity retention still +applies. See "Service-managed conversations". **Why workflows largely come "for free."** Durable workflow agent execution (`DurableExecutorDispatcher.ExecuteAgentAsync`) runs an agent through the same @@ -234,101 +184,49 @@ executors does not pass through the agent, so it needs the separate **L3** hook. ### Consequences -- Good: **configuration parity for context.** The same core strategies and hooks apply on the durable - runtime with no changes, and retention applies no context policy of its own. -- Good: **every configuration is protected from the capacity limit**, including external providers, - service-managed agents and agents with no context pipeline, because retention lives in the entity - rather than in the history provider. -- Good: **deletion is proportionate.** Under `auto` the amount removed is set by the budget, not by - how much a context strategy happened to exclude. -- Neutral: a larger entity change than a bespoke compaction pass, and it must preserve the existing - `ConversationHistory` consumer contract (`AgentRunHandle` response polling, audit/replay, TTL). -- Bad: **L2 is Python-only today.** In .NET, `CompactionProvider` persists its `CompactionMessageIndex` - into `AgentSession.StateBag` with full `ChatMessage` copies, so a durable provider that also persists - the session would store the transcript twice. See "Core Interface Gaps". -- Bad: L2 carries workaround code in Python because upstream binds the store-rewrite hook to session - state. That code is deletable if the gap closes. -- Bad: retention under `auto` behaves differently above and below the watermark, which is harder to - explain than uniform behavior. Accepted because the alternative for those users is the entity - failing. -- Bad: an opt-in LLM-based reducer under `follow_compaction` runs inside the entity operation and - re-runs on retry, mitigated by stable summary identity and optionally by Option 3 to move heavy - summarization off the request path. Eviction under `auto` is deterministic and unaffected. +- **Configuration parity.** Existing agent compaction configuration works durably without changing + the agent. Retention does not choose the current model projection. +- **Broad capacity protection.** Because pressure eviction lives in the entity, it covers external + providers, service-managed agents, and agents with no context pipeline. A single oversized newest + exchange can still fail because the current result is never evicted. +- **Proportionate deletion.** Under `auto`, the budget decides how much to remove. The user's context + strategy does not. +- **Larger entity change.** The history-provider design must preserve response polling and the + entity's conversation record. +- **Python-only eager pruning.** .NET would duplicate the transcript if it persisted current + compaction state (gap 4), so a .NET implementation could use pressure retention but not L2 yet. +- **Core workarounds.** Python must publish and reconcile a working buffer because core binds + store-side compaction to session state (gaps 1 and 2). +- **Threshold behavior.** `auto` changes behavior only near the capacity limit. This is less uniform + than always pruning, but it avoids changing unaffected conversations. ### Validation -**Done (Python).** Unit tests cover the provider substitution rules, compaction annotations -surviving a state round-trip, summary insertion, pruning, service-managed skip, session-state -persistence, and workflow context projection. Integration tests run against a real scheduler and -assert that annotations and message ids survive entity serialization, that an external provider -keeps a whole conversation under one key, and that a downstream workflow agent can reference the -upstream conversation. +**Done (Python).** Unit tests cover provider substitution, annotation round-trips, synthetic summary +insertion and reconciliation, all retention modes, session persistence, and workflow projection and +deduplication. The retention test drives a real agent through twenty turns against a reduced budget. +`keep_all` is the control proving the same run exceeds it. Scheduler integration covers persisted +annotations and message ids, external-provider session identity, schema conformance, and downstream +workflow context. **Outstanding.** Not covered yet. -- **Retention.** All three modes are built and covered, including an end-to-end test that drives a - real agent through the entity for twenty turns against a small budget, with `keep_all` as the - control proving the same run exceeds it. What is not yet covered is a conversation crossing the - real scheduler limit against a real backend, rather than a lowered one in-process. +- Retention crossing the real scheduler limit against a live backend, rather than a reduced budget + in process. - The .NET realization and its schema parity (gap 3), and the .NET compaction-state blocker (gap 4). -- Blob offload (Option 7) against a real scheduler. Whether the Durable Functions Python path can - reach it is now answered: it cannot, in either 1.x or the 2.x preview (gap 6). -- An external history provider storing history beyond the built-in state-size limit. +- Blob offload (Option 7) against a real scheduler. It remains unreachable through Durable Functions + Python 1.x and the 2.x preview (gap 6). - Idempotency of an LLM-based reducer across simulated entity retries. -## Pros and Cons of the Options - -The full argument is in **Decision Outcome** above. This is the summary. - -- **Option 1 - In-run filter only.** Existing core feature, almost no new code, bounds the model - input including long tool loops, and applies to workflow agent executors too. But it is non-lossy - by design, so the persisted store keeps growing and the filter's incremental state is discarded - and recomputed every turn. -- **Option 2 - Bespoke pre-write compaction in the entity.** Directly bounds persisted state, but is - new durable-only code duplicating what core's store-reducer path already does, needs a - `DurableAgentStateMessage` ⇄ message conversion, and gives no external-storage pluggability. -- **Option 3 - On-storage maintenance compaction.** Keeps expensive summarization off the request - path and maps to ADR-0019's "on existing storage" point, and can layer on top of Option 6 later - without rework. Adds scheduling machinery, leaves a window where state is un-compacted, and does - not bound in-turn growth. -- **Option 4 - Workflow-level hook.** Bounds the inter-executor `full_conversation` that agent-level - compaction never sees, reusing the existing `context_filter` seam. Only relevant to multi-agent - workflows, and must reuse core grouping or a naive filter breaks atomic groups. **Adopted - alongside Option 6 as L3.** -- **Option 5 - Auto-derive a store reducer.** Would bound durable storage without an explicit - reducer, but as a *default* it only reaches agents that already configured compaction, since - nothing else marks messages excludable, and it treats a context decision as consent to delete. - **Adopted in opt-in form as the `follow_compaction` retention mode**, not as the default. -- **Option 6 - Durable store as a history provider (chosen).** The user's configuration carries over - unchanged, and the same abstraction makes external backends pluggable, so one seam delivers both - the opt-in reducer and pluggable storage. Costs a larger entity change that must preserve the - `ConversationHistory` consumer contract (response polling, audit, TTL). L2 also does not come free: - in Python the store-rewrite hook is bound to session state, so the provider publishes a working - buffer and reconciles it itself, and **in .NET L2 is blocked outright** until core can persist - compaction metadata without duplicating the transcript (see "Core Interface Gaps"). -- **Option 7 - Blob offload.** Raises the ceiling roughly tenfold with no data loss, needs no code - from this layer since the payload store is passed to the worker and client the caller already - builds, and mirrors what the Azure Storage backend does internally. But it is preview, needs a - storage account, does not remove the ceiling, and is unreachable on the Durable Functions Python - path in both 1.x and the 2.x preview (gap 6). **Adopted as the first capacity answer on the - durabletask path, ahead of any deletion.** - ## Cross-Cutting Design Details -- **Reducer trigger.** Honor the configured `ReducerTriggerEvent`. `AfterMessageAdded` - (compact-on-write, before checkpoint) is the natural durable default so the checkpoint is already - bounded. `BeforeMessagesRetrieval` also works (reduce-on-load, then persist). -- **Determinism & idempotency.** An opt-in lossy reducer runs inside the entity operation and - re-runs on retry. Give any generated summary a **stable identity** (derived from the ids of the - messages it replaces) so retries do not re-summarize or duplicate. Reduced content becomes - **permanent** durable state (same indirect-prompt-injection caution core flags on - `ChatReducerCompactionStrategy` / `SummarizationCompactionStrategy`). -- **Message-list correctness.** Reuse core grouping so atomic tool-call/result and reasoning - pairings are preserved at every layer. -- **Token counting.** Triggers must work without a live model call, so use the estimator tokenizer - (`CharacterEstimatorTokenizer` / equivalent) unless a real tokenizer is supplied. -- **Placement.** The durable history provider backs `AgentEntity` in both languages. L3 lives in the - `AgentExecutor` context handling. +- Honor the user's configured reducer trigger. Durable registration must not change compaction + cadence. +- Reuse core grouping so tool-call/result and reasoning groups remain atomic. +- Pressure eviction is deterministic and uses the estimator tokenizer without a model call. Any + future LLM reducer must give summaries stable identities and be tested across retries. +- The durable history provider belongs in `AgentEntity`. Workflow projection belongs at the + existing `AgentExecutor.context_mode` / `context_filter` seam. ## Core Interface Gaps for Pluggable History Providers @@ -338,79 +236,33 @@ whose store is not session state (Cosmos, Valkey, durable), not just this one. T around them, but the cleaner fix is upstream. 1. **Store-side compaction is bound to session state rather than to the provider.** `CompactionProvider` - has two hooks and only one of them is coupled. - - - `before_strategy` runs on messages already in the invocation context, whichever provider loaded - them. Every provider gets this, so **in-run context bounding already works for external stores**. - - `after_strategy` is documented as operating on "the accumulated messages stored by a history - provider in session state", and "requires `history_source_id` to locate the messages in session - state". It reads `session.state[history_source_id]["messages"]` and mutates that list in place, - treating mutation as persistence - which only holds when the store *is* session state. + has two hooks, but only `before_strategy` works with any provider because it acts on invocation + context. `after_strategy` mutates `session.state[history_source_id]["messages"]` and assumes that + mutation rewrites storage. External providers can therefore bound model input but cannot use + core to rewrite their stores. .NET similarly exposes `IChatReducer` only on + `InMemoryChatHistoryProvider`. - So the missing capability is narrower than it first appears: an external provider can bound what - the model sees, but cannot have the framework rewrite its store. - - Whether that is a defect depends on **who owns the store**. For a user-owned store (Cosmos, Redis) - the framework arguably *should not* rewrite it implicitly. For a framework-owned store (in-memory, - and durable entity state) rewriting is squarely in scope. Durable is the first framework-owned - store that is not session state, which is what turns this from a defensible omission into a real - problem. - - It is also unresolved rather than decided. ADR-0019 names three compaction points (in-run, - pre-write, on existing storage), explicitly scopes in "local storage (e.g. `InMemoryHistoryProvider`, - Redis, Cosmos)", and then leaves the mechanism open: - - > Should pre-write and existing-storage compaction share one unified configuration/setup to reduce - > duplicate strategy wiring, and then either: each write overrides the full storage, or only new - > messages are compacted while a separate interface can be called to compact the existing storage? - - That question shipped unanswered, and the languages then diverged on where the hook lives. .NET - puts store reduction on the provider (`IChatReducer`) but only on `InMemoryChatHistoryProvider`, - and `CosmosChatHistoryProvider` has none. Python puts it in `CompactionProvider` reaching into - session state. **Neither language offers it to external providers.** - - *Workaround:* the provider publishes its loaded messages as a working buffer under the expected - session-state key. *Upstream fix:* bind the store-rewrite hook to the provider abstraction instead - of to session state as a storage mechanism, since .NET's shape generalizes and Python's does not. + *Workaround:* the durable provider publishes a working buffer under the session-state key core + expects. *Upstream fix:* put store-rewrite compaction on the provider abstraction. 2. **`save_messages()` is append-only.** The other half of the same open question. It receives only - the newly produced messages, so mutations that compaction applies to *already stored* messages - (setting `_excluded`, inserting a summary) have no defined path back to the store. - *Workaround (implemented):* the provider overrides `after_run` and reconciles the working buffer - itself **by `message_id`**, updating annotations on known messages and inserting ones compaction - added. This required persisting `messageId` in durable state, which also gives summaries the - **stable identity** the idempotency requirement needs. *Upstream fix:* add an explicit - replace/flush operation alongside append so every external provider does not have to re-implement - this reconciliation. - -3. **Message-level metadata was not persisted (durable schema).** `DurableAgentStateMessage.to_dict()` - dropped `extension_data` while `from_dict()` read it, a write-lossy asymmetry that silently - discarded compaction annotations on every state round-trip. Since annotations are what carry - compaction state, this had to be fixed for any of this to work. This one is ours rather than - core's. The Python side now serializes it. - - The shared schema also under-declared what is persisted. `messageId` and `extensionData` are both - load-bearing for compaction and neither appeared in `chatMessage`, so an implementer reading the - contract had no way to know they must round-trip. Nothing would have *failed* validation, since - the schema permits extra properties, which is precisely why it went unnoticed. They are declared - now, `session` is described as an opaque runtime-discriminated payload rather than pinning - Python's shape onto .NET, and a test validates real persisted state against the schema so the two - cannot drift apart again silently. - - **.NET needs the same treatment, and looks deceptively fine.** Its `DurableAgentStateMessage` - already has an `ExtensionData` property, but it is `[JsonExtensionData]`, System.Text.Json's - overflow bucket for *unmapped JSON properties*, not a mapping of `ChatMessage.AdditionalProperties`. - `FromChatMessage`/`ToChatMessage` copy neither `AdditionalProperties` nor `MessageId`, so both are - lost at the **conversion** boundary rather than the JSON one. Anyone checking for "is extension - data persisted?" will see the property and wrongly conclude parity is done. - - Two clarifications, because the reason this matters is not the obvious one. `ChatMessage.MessageId` - **does** exist in the pinned Microsoft.Extensions.AI.Abstractions and is used throughout .NET, so - it only needs mapping, not inventing. And .NET does **not** keep exclusion state in - `AdditionalProperties` (it lives on `CompactionMessageGroup.IsExcluded`), so mapping these two - fields is necessary but not sufficient. What `AdditionalProperties` does carry is the summary - marker `_is_summary`, which is how a rebuilt index recognises an existing summary instead of - re-summarizing it. + new messages, so changes to existing messages and inserted summaries have no path back to storage. + *Workaround:* the durable provider reconciles its working buffer **by `message_id`** during + `after_run`. *Upstream fix:* add an explicit replace/flush operation alongside append. + +3. **Message-level metadata was not persisted (durable schema).** Python wrote + `extension_data` asymmetrically, so annotations disappeared on round-trip. This is fixed. The + shared schema now declares `messageId` and `extensionData` as round-trip-required, describes + `session` as runtime-discriminated, and has a conformance test. A validator would not previously + have rejected these fields because the schema permits extra properties. The defect was an + under-declared contract. + + .NET still loses `ChatMessage.AdditionalProperties` and `MessageId` in + `FromChatMessage`/`ToChatMessage`. Its `[JsonExtensionData]` property is only an overflow bucket for + unmapped JSON. `ChatMessage.MessageId` **does** exist in the pinned package and needs mapping. + Exclusions themselves live on `CompactionMessageGroup.IsExcluded`, while + `AdditionalProperties` carries the `_is_summary` marker, so mapping both fields is necessary but + not sufficient for .NET compaction parity. 4. **.NET compaction state cannot be persisted without duplicating the transcript.** This is the blocker behind "L2 is Python-only today". `CompactionProvider.State` is documented as living in @@ -426,42 +278,25 @@ around them, but the cleaner fix is upstream. None of this is inherent to the history-provider approach. It resolves if core can persist lightweight compaction metadata keyed by `MessageId` rather than whole message copies. Until then - .NET can bound entity state only through the retention path, which is deliberately independent of - `CompactionProvider` and therefore unaffected. + a .NET implementation can bound entity state through the retention path, which is independent of + `CompactionProvider`. 5. **Provider cadence splits under per-service-call persistence.** With - `require_per_service_call_history_persistence=True`, the agent's once-per-run loop skips history - providers because the per-service-call middleware drives `before_run`/`after_run` itself, once per - **model call** instead of once per run. `CompactionProvider` is not a `HistoryProvider`, so it - stays on the once-per-run path. The pair is therefore split across two cadences, and compaction - annotates the buffer *after* the history provider last flushed it, so annotations would not reach - storage until the following flush. Only `HarnessAgent` sets this flag today, so this is latent - rather than live. It is recorded because the symptom would be missing annotations rather than an - error. + `require_per_service_call_history_persistence=True`, history providers run per model call while + `CompactionProvider` remains once per run. Compaction can then annotate after the last history + flush, delaying persistence until the next flush. Only `HarnessAgent` enables this today, so the + gap is latent. 6. **Blob offload is unreachable on the Durable Functions Python path.** Not a core gap but an - upstream one, recorded here because it is what forces this layer to own a capacity answer at all. - `azure-functions-durable` 1.x, which this package pins (`>=1.3.1,<2`), does not depend on the - durabletask SDK, since the host extension owns persistence. There is no Python-side seam to - configure and the word payload does not appear in the package. The 2.x preview (`2.0.0b1`, - `2.0.0b2`, both requiring Python 3.13+) does depend on `durabletask>=1.9.0`, and - `DurableFunctionsWorker` subclasses `TaskHubGrpcWorker`, whose constructor accepts - `payload_store`. But `DurableFunctionsWorker.__init__` takes no parameters and hardcodes its - `super().__init__` arguments, and `DurableFunctionsClient.__init__` takes only a connection - string. Neither forwards `**kwargs`, so the inherited capability is unreachable. The durabletask - path has no such problem, because the caller constructs the worker and client and can pass - `payload_store` directly. *Upstream fix:* expose `payload_store` on `DurableFunctionsWorker` and - `DurableFunctionsClient`. - -Two further core gaps are recorded with the decisions they affect: the process-local **state type -registry** (see "The session is persisted, not just its conversation id") and the absence of a public -way to ask whether **the service owns history for a run** (see "Service-managed conversations"). Both -forced this layer to re-implement logic core already has. - -Consequence for ordering: core runs `before_run` forward and `after_run` in **reverse**. With -`[history, compaction]`, compaction annotates the buffer *before* the history provider flushes it -(convenient), but it sees history only as of the **previous** turn - so context reaches a steady -state rather than shrinking immediately. This is expected, not a defect. + upstream gap. Version 1.x, which this package pins (`>=1.3.1,<2`), does not use the durabletask + SDK, so it has no Python-side payload-store seam. The 2.x previews (`2.0.0b1` and `2.0.0b2`, Python + 3.13+) depend on `durabletask>=1.9.0`, but `DurableFunctionsWorker` and + `DurableFunctionsClient` do not expose the base types' `payload_store` parameter. The direct + durabletask path does. *Upstream fix:* expose `payload_store` on both Functions types. + +Two more core gaps are described where they matter: the process-local state-type registry under +session persistence, and the lack of a public resolved history-ownership decision under +service-managed conversations. ## L3 Realization: Workflow Context Parity @@ -470,61 +305,33 @@ In-process workflows give a downstream `AgentExecutor` the upstream conversation `custom` + `context_filter`). The durable orchestrator previously flattened that to the **last message's text**, so a downstream agent lost everything earlier nodes produced. -**L3 is a weaker seam than L1 and L2, and should not be described as parity with them.** Core's -compaction system is agent-level, so a workflow agent node inherits L1 unchanged: the in-process -`AgentExecutor` holds its own `AgentSession` and passes it to `agent.run()`, so any `CompactionProvider` -on the agent runs exactly as it would standalone. The inter-executor conversation has no equivalent. -`context_filter` is a synchronous callable returning a filtered list, not a strategy that annotates -groups, so L3 reuses the same *strategy* at a different, plainer seam rather than reusing the same -hook. - -Durable now projects the same conversation and delivers it to the agent entity: - -- The orchestrator reads the executor's `context_mode`/`context_filter` and projects - `full_conversation` accordingly. -- The projection travels as `RunRequest.context_messages` (serialized `Message` values) and becomes - the request entry's messages, so it is persisted like any other conversation content and is - visible to compaction. -- A node that runs more than once (a cycle) receives the whole upstream conversation again, so the - entity **drops the part it has already recorded**, keeping at least the latest message so the - agent always has an input. - -**Dedup is tracked by position, not by stored identity.** Comparing against the ids currently in -`ConversationHistory` breaks the moment retention evicts any of them: their ids leave the comparison -set, the orchestrator re-sends them on the next visit because its own conversation is never evicted, -and the node re-records exactly what was just deleted. That oscillates rather than converges, since -the re-ingested volume is proportional to what was evicted. - -The entity therefore keeps a small map of `executor_id` to the highest conversation position it has -ingested, and drops anything at or below that mark. It is a handful of integers, it is unaffected by -deletion, and it is per executor rather than global because a fan-out gives two branches the same -position. Consequence worth stating: once a message is evicted the node stops seeing it, where the -broken behavior would re-feed it. That is intended. Re-ingesting evicted content defeats the -eviction. - -**Alternatives measured and rejected.** Not persisting the forwarded context, and treating the -orchestrator's conversation as authoritative, both looked cleaner on paper. Measuring what actually -reaches the model showed otherwise. Core in-process sends 11 messages on the third visit of a -`full`-mode cycle, with heavy duplication, while durable today sends 8, because this dedup removes -repeats before they reach the model. For `last_agent` the two are identical. So the current design -already matches core where core is sane and improves on it where core is not, and the alternatives -would have reordered the conversation or dropped context the node should keep. - -Behavior difference that remains, by design: each agent node also keeps its **own durable history** -(keyed by workflow instance + executor), so per-agent memory survives restarts and is compacted -independently - a superset of the in-process behavior rather than a strict match. +Agent-level compaction needs no workflow-specific work: `AgentExecutor` passes its own session to +`agent.run()`, so the agent's `CompactionProvider` runs normally. Inter-executor context has no core +compaction hook. Durable instead honors the existing `context_mode` and invokes `context_filter` for +`custom` mode, then sends the projection as `RunRequest.context_messages`. Those messages become part +of the request entry and are visible to agent-level compaction. -## Zero-Configuration Registration +Cycles need deduplication because a node receives the accumulated upstream conversation again on +each visit. The orchestrator stamps each forwarded message as `wf_{executor}_{position}`. The entity +stores the highest ingested position per executor and drops older positions, keeping the newest +message as input when everything repeats. Per-executor watermarks are required because fan-out +branches can share a position. -The parity goal is only met if a user can take an agent that **already works in core**, register it -with `AgentFunctionApp` (or the worker, or as a workflow node), and get durable behavior with **no -edits to the agent**. Requiring them to add a durable-specific provider would just relocate the -configuration burden. +Stored-id comparison is insufficient: retention removes old ids, after which a cycle would re-ingest +exactly what was evicted and oscillate instead of converging. The small position map survives +deletion. Once content is evicted, the node no longer sees it. Re-ingesting it would defeat +retention. -So the durable entity substitutes the history provider at construction time - covering every -registration path, since both the worker and the Azure Functions host build the same entity. The -agent is never mutated: when a substitution is needed, a shallow copy with its own provider list is -used, so the caller's agent still behaves normally in-process. +This intentionally differs from core in one measured case. On the third visit of a `full`-mode +cycle, core in-process sends 11 messages with repeated context while durable sends 8 after dedup. In +`last_agent` mode they are identical. Each durable node also keeps history keyed by workflow instance +and executor, so its memory survives restarts independently of the workflow envelope. + +## Zero-Configuration Registration + +Registration must not require edits to an agent that already works in core. The entity therefore +substitutes history at construction time. It shallow-copies the agent when substitution is needed, +so the caller's instance remains unchanged. | User configured | Durable behavior | | --- | --- | @@ -536,92 +343,42 @@ used, so the caller's agent still behaves normally in-process. Preserving `source_id` is the load-bearing detail. `CompactionProvider` locates history through `history_source_id` (default `"in_memory"`), so a provider swapped in under the same id is invisible -to the rest of the user's configuration. Because the injected provider is a `HistoryProvider` with -`load_messages=True`, core's own auto-injection sees a provider present and stands down, leaving no -duplicate provider. - -An explicit `DurableHistoryProvider` remains supported as an advanced escape hatch, and takes -precedence over anything the runtime would inject. +to the rest of the configuration. An explicit `DurableHistoryProvider` takes precedence. -### When the entity manages history itself - -Two distinct decisions drive the entity, and conflating them caused bugs. +### Entity Context Ownership 1. **Who supplies conversation context?** If the agent exposes core's context-provider pipeline, the providers do, so the entity passes a session and delivers **only the new messages**. This holds whether history lives in durable state, an external store, or the model service. -2. **Should durable state be bound?** Retention decides this, at the entity, for every - configuration. It is deliberately not tied to whether a `DurableHistoryProvider` is present, - because the entity records the conversation either way. +2. **Who bounds entity state?** Retention does, for every configuration, because the entity records + the conversation even when another provider owns model context. The entity therefore replays its own persisted history in exactly one case, an agent that does not -expose the context pipeline at all (for example a fully custom agent). Routing external-store or -service-backed agents down that path was incorrect, because it either bypassed their provider -entirely or re-sent history the service already had. - -**Consequence:** passing a session is what re-engages the pipeline, so external history providers -(Cosmos, Redis, file) now function under the durable runtime. Previously they were silently -ignored because no session was ever created. They get the in-run filter like any other provider. -What they do not get is the framework rewriting their store, which no language offers today (core -interface gap 1 above). - -That session must also carry the entity's **stable** session id rather than a generated one. -External providers key their storage on `session.session_id`, so a per-operation id would make them -read and write a different key every turn - the conversation would silently restart each time with -nothing to indicate a problem. +expose the context pipeline. Passing a session re-engages external providers and core's in-run +filter. It does not let core rewrite an external store (gap 1). The session id is derived from the +full entity identity, name plus key, so workflow nodes cannot share an external-provider key. ### The session is persisted, not just its conversation id -Core documents the per-provider `state` dict handed to `before_run`/`after_run` as durable for the -life of the session, and persists it through `AgentSession.to_dict()`. The entity builds a fresh -session per operation, so anything providers keep there was previously discarded at the end of every -turn: tool approval rules and **queued approval requests**, todo lists, background-task state, memory -extraction state. On .NET the same bag (`AgentSessionStateBag`) is a first-class part of the -`AIContextProvider` contract via `StateKeys`, so the gap is wider there. - -That is a poor fit for a durable runtime whose headline scenario is long-running human-in-the-loop: -an approval flow that spans turns cannot work if the pending requests are dropped between them. - -So the entity persists the **whole serialized session** rather than individual fields. Two -consequences: +Providers use session state for data that must survive turns, including pending approvals. Because +the entity creates a session per operation, it persists the **whole serialized session** rather than +selecting fields. Two details prevent duplication and type loss: - The service-issued conversation id needs no bespoke field of its own - it is already part of - `AgentSession.to_dict()`. This replaces a hand-rolled `serviceSessionId` state field and its - capture/restore helpers with one general mechanism that matches core's own serialization contract. + `AgentSession.to_dict()`. - The durable history provider's own slice is **excluded** before persisting. It is derived from - `conversationHistory` on every turn, so storing it would duplicate the transcript and let the copy - drift from the record of truth. + `conversationHistory`, so storing it would duplicate the transcript. Restore applies the stored state onto a session created by the agent's own `create_session()`, so -the agent's session type is preserved. - -**Restoring values as their own types.** Core deserializes state through a type registry that it -seeds with exactly one entry (`Message`). Anything else must be registered explicitly, and the -registry is process-local. `to_dict`-based types are never auto-registered, and only Pydantic models -are, and then only as a side effect of serializing. A durable entity routinely restores in a process -that never serialized the value, so state would come back as plain dicts instead of its own classes. - -Before restoring, the entity therefore registers the serializable types **already loaded in the -process**. Nothing is imported from persisted data, so this cannot load code the application has not -already loaded itself, and that is sufficient in practice, because whoever put a value in the state -bag had to import its class to construct it. The walk is over `SerializationMixin` subclasses and -costs tens of microseconds. - -Residual gaps, both better fixed in core. - -- Pydantic values in state are keyed by `cls.__name__.lower()` and are not covered, since walking - every `BaseModel` subclass in the process would be broad and collision-prone. -- Core could seed the registry with the state types it ships, which would make this unnecessary. - `register_state_type()` is already public and its documentation names cold-start restore as the - motivating case, yet nothing calls it today. +the agent's session type is preserved. Core's state-type registry is process-local, so the entity +pre-registers serializable types already loaded in the process before restore. Pydantic state remains +a core gap because broad subclass discovery would be collision-prone. ### Service-managed conversations When the model service stores the conversation, it identifies the thread with an id. The entity creates a fresh session per operation, so that id is **persisted in durable state and restored on -the next turn** (as part of the serialized session, above). Without it the service would start a new -thread every turn. The durable history provider additionally no-ops (neither loading nor flushing) -for service-managed sessions. +the next turn** as part of the serialized session. Without it, every turn would start a new thread. Whether the service owns history is decided with **core's precedence, not the client class alone**. An explicit `store` in the agent's options wins, and only when it is unset does the client's @@ -631,63 +388,22 @@ API) are routinely put back into client-side mode with `store=False`. Consulting runtime never persists, silently losing the conversation between turns. Core resolves this rule inside `Agent._run` and does not expose the result, so this layer -**re-derives it** and can drift from core if the rule changes, with silent conversation loss as the -symptom, which is exactly the bug this rule was written to fix. The unit tests here only pin *our* -logic. The end-to-end net is the compaction sample, which runs `store=False` against a -store-by-default client and asserts recall. *Upstream fix:* expose the resolved decision. +**re-derives it** and can drift if core changes. *Upstream fix:* expose the resolved decision. The +integration sample covers `store=False` against a store-by-default client. ### Retention is a deployment policy, not agent configuration Compaction annotates, it does not delete. Deletion is configured at **registration** (an app-level default with a per-agent override) rather than on the agent, so the agent definition stays portable: -the same agent runs in-memory where retention would be meaningless, and the setting sits next to its -natural sibling, entity lifetime and TTL. - -The three modes are described under "Retention" in the Decision Outcome. Two properties are worth -restating here, because they are what make retention safe to have on by default. - -- **It applies no context policy.** Retention decides what durable state can hold, never what the - model should read. Filtering the model's view remains entirely L1's job. What retention cannot - avoid is that a deleted message is gone for every reader, including the history provider that - loads context from `ConversationHistory`. Eviction therefore shortens the model's available - history as a consequence of deletion, not as a policy of its own, and only from the point where - the record would otherwise have stopped being writable at all. -- **An exclusion is not consent to delete.** `follow_compaction` is the only mode where a compaction - exclusion causes deletion, and it is opt-in. Under `auto` a user's exclusions are left untouched - and the amount deleted is set by the storage budget alone. - -## Related Concern: Entity Lifetime (TTL) and Cleanup - -Compaction bounds the *size* of a conversation. Entity **lifetime**, when the persisted state is -deleted, is a separate axis. It is out of scope for the decision above, but is recorded here -because it is the natural sibling of the retention setting introduced by this ADR, and because it -has a notable cross-language parity gap in this repository. - -**TTL does not substitute for retention.** The .NET mechanism is a sliding idle timer: every -interaction pushes `ExpirationTimeUtc` forward, so an actively used conversation never expires and -grows until it reaches the backend limit. TTL reclaims *abandoned* entities, which bounds how many -exist and what they cost in aggregate. It does nothing about how large a single live entity gets, -which is the failure this ADR's retention design addresses. - -- **.NET agents:** `DurableAgentsOptions.DefaultTimeToLive` (default 14 days) provides a global TTL, - with a per-agent override via `AddAIAgent(agent, ttl)`. Idle entities self-delete via an - `ExpirationTimeUtc` + `CheckAndDeleteIfExpired` self-signal. -- **.NET workflows:** workflow agent executors are auto-registered *without* a TTL - (`DurableWorkflowOptions` calls `AddAIAgent(agent)`) and inherit the global default. There is **no - workflow-scoped TTL option**, and each agent-node invocation spawns a fresh, single-use entity that - then lingers for the full default (14 days) - far longer than needed for throwaway per-node state. -- **Python (agents *and* workflows):** there is **no TTL/cleanup mechanism at all** - no global - default, no per-agent option, no `expirationTimeUtc` in the state schema, and no deletion. Entities - persist indefinitely until manually deleted. This is a **.NET/Python parity gap**. - -Follow-ups (tracked separately from the compaction decision): - -1. **Port the TTL mechanism to Python** - a global default TTL, per-agent override, an - `expirationTimeUtc` state field (for cross-language schema parity), and idle-based self-deletion. -2. **Expose a configurable global TTL consistently** across both languages, for agents and workflows. -3. **Give workflow-spawned agent entities a sensible lifetime** - a short workflow-scoped default TTL, - or deterministic cleanup when the workflow completes, instead of the 14-day agent default (with an - idle-TTL backstop for workflows that pause or never reach a terminal state). +the same agent runs in-memory where retention has no meaning. `auto` applies no context policy, but +deletion necessarily shortens future available history. Only `follow_compaction` treats a compaction +exclusion as permission to delete. Under `auto`, the storage budget alone chooses what is removed. + +## Out of Scope: Entity Lifetime + +Idle TTL and cleanup bound how many abandoned entities remain. They do not bound an actively used +entity because each interaction extends its lifetime. Cross-language TTL parity is a separate +decision. ## More Information @@ -700,6 +416,3 @@ Follow-ups (tracked separately from the compaction decision): - Relevant durable code: `AgentEntity` and `DurableAgentState` (durable agents), `DurableExecutorDispatcher.ExecuteAgentAsync` (durable workflow agent execution), and `AgentExecutor` (`context_mode` / `context_filter`, `full_conversation`). -- Suggested realization order: express the durable store as a `ChatHistoryProvider` (Option 6) → - verify L1 filter parity → wire L3 workflow hook → add external storage backends → evaluate - Option 3 for heavy summarization. From b58621978e055efebeb59789f7acd6dd68d545f3 Mon Sep 17 00:00:00 2001 From: Ahmed Muhsin Date: Fri, 14 Aug 2026 15:40:33 -0500 Subject: [PATCH 32/32] refactor: remove the unshipped prune_history alias prune_history was introduced and replaced within this draft branch, so deprecating it would preserve an API that no release ever exposed. Remove the flag from worker and Functions registration, delete the compatibility resolver, and use retention modes directly. follow_compaction remains the explicit policy for deleting compaction exclusions. Clarify that follow_compaction still runs pressure eviction when eager pruning is insufficient. Add an end-to-end control with no compaction strategy, proving the shared fallback keeps state bounded. --- .../agent_framework_azurefunctions/_app.py | 14 +++----- .../_entities.py | 4 +-- .../agent_framework_durabletask/_entities.py | 2 +- .../_history_provider.py | 8 ++--- .../agent_framework_durabletask/_retention.py | 25 ++------------ .../agent_framework_durabletask/_worker.py | 12 +++---- .../tests/test_durable_history_autoswap.py | 8 ++--- .../durabletask/tests/test_retention.py | 34 +++++-------------- .../13_conversation_compaction/README.md | 2 +- .../14_conversation_compaction/README.md | 5 +-- .../function_app.py | 4 +-- 11 files changed, 36 insertions(+), 82 deletions(-) diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py index 2450854..5271786 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_app.py @@ -48,7 +48,6 @@ DEFAULT_MAX_STATE_BYTES, DEFAULT_RETENTION, RetentionMode, - resolve_retention, ) from agent_framework_durabletask._workflows.naming import ( SUBWORKFLOW_REQUEST_SEPARATOR, @@ -250,7 +249,6 @@ def __init__( poll_interval_seconds: float = DEFAULT_POLL_INTERVAL_SECONDS, enable_mcp_tool_trigger: bool = False, default_callback: AgentResponseCallbackProtocol | None = None, - prune_history: bool | None = None, retention: RetentionMode = DEFAULT_RETENTION, workflow_retention: RetentionMode | None = None, max_state_bytes: int = DEFAULT_MAX_STATE_BYTES, @@ -273,12 +271,12 @@ def __init__( :param poll_interval_seconds: Delay in seconds between polling attempts. Defaults to ``DEFAULT_POLL_INTERVAL_SECONDS``. :param default_callback: Optional callback invoked for agents without specific callbacks. - :param prune_history: Deprecated. ``True`` maps to ``retention='follow_compaction'``. :param retention: Default conversation retention for agents hosted by this app, including agents inside hosted workflows. ``auto`` deletes only under storage pressure, ``keep_all`` never deletes and lets the entity fail at the backend limit, and - ``follow_compaction`` also deletes what compaction excluded. ``add_agent`` can - override it per agent. + ``follow_compaction`` first deletes what compaction excluded, then uses the same + pressure eviction as ``auto`` if that is not enough. ``add_agent`` can override it + per agent. :param max_state_bytes: Budget for serialized entity state. :param workflow_retention: Retention for agent nodes inside hosted workflows. When None, ``retention`` applies. Worth setting separately, since a workflow node's entity lives @@ -303,7 +301,7 @@ def __init__( self.enable_http_endpoints = enable_http_endpoints self.enable_mcp_tool_trigger = enable_mcp_tool_trigger self.default_callback = default_callback - self._retention: RetentionMode = resolve_retention(retention, prune_history) + self._retention: RetentionMode = retention self._workflow_retention: RetentionMode | None = workflow_retention self._max_state_bytes = max_state_bytes @@ -850,7 +848,6 @@ def add_agent( enable_mcp_tool_trigger: bool | None = None, *, entity_id: str | None = None, - prune_history: bool | None = None, retention: RetentionMode | None = None, ) -> None: """Add an agent to the function app after initialization. @@ -868,7 +865,6 @@ def add_agent( durable entity (and the ``agents`` / ``get_agent`` key) matches the identity the orchestrator dispatches to. Mirrors ``DurableAIAgentWorker.add_agent(entity_id=...)``. - prune_history: Deprecated. ``True`` maps to ``retention='follow_compaction'``. retention: Per-agent retention override. When None, the app-level setting is used. Raises: @@ -926,7 +922,7 @@ def add_agent( effective_callback, effective_enable_http_endpoint, effective_enable_mcp_endpoint, - retention=resolve_retention(effective_retention, prune_history), + retention=effective_retention, ) logger.debug(f"[AgentFunctionApp] Agent '{registration_name}' added successfully") diff --git a/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py b/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py index 5db3ef5..46de237 100644 --- a/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py +++ b/python/packages/azurefunctions/agent_framework_azurefunctions/_entities.py @@ -67,8 +67,8 @@ def create_agent_entity( Keyword Args: retention: How much of the conversation durable state may discard. ``auto`` deletes only - under storage pressure, ``keep_all`` never deletes, and ``follow_compaction`` also - deletes what compaction excluded. + under storage pressure, ``keep_all`` never deletes, and ``follow_compaction`` first + deletes what compaction excluded, then uses pressure eviction if needed. max_state_bytes: Budget for serialized entity state. Returns: diff --git a/python/packages/durabletask/agent_framework_durabletask/_entities.py b/python/packages/durabletask/agent_framework_durabletask/_entities.py index 4d3c413..13db529 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_entities.py +++ b/python/packages/durabletask/agent_framework_durabletask/_entities.py @@ -212,7 +212,7 @@ def __init__( ) -> None: # Back the agent's conversation history with durable entity state so an agent that # already works in core runs durably without any configuration change. - self.agent = ensure_durable_history(agent, prune_history=prunes_excluded(retention)) + self.agent = ensure_durable_history(agent, prune_excluded=prunes_excluded(retention)) self.callback = callback self._state_provider = state_provider self._retention = retention diff --git a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py index 53934a1..75f72ba 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_history_provider.py +++ b/python/packages/durabletask/agent_framework_durabletask/_history_provider.py @@ -411,7 +411,7 @@ def _service_stores_history(agent: Any) -> bool: return bool(getattr(client, "STORES_BY_DEFAULT", False)) -def ensure_durable_history(agent: SupportsAgentRun, *, prune_history: bool = False) -> SupportsAgentRun: +def ensure_durable_history(agent: SupportsAgentRun, *, prune_excluded: bool = False) -> SupportsAgentRun: """Back an agent's conversation history with durable entity state. Lets a user register an agent that already works in core and get durable behavior with no @@ -435,7 +435,7 @@ def ensure_durable_history(agent: SupportsAgentRun, *, prune_history: bool = Fal agent: The agent being registered with the durable runtime. Keyword Args: - prune_history: When True, the injected provider physically deletes messages that + prune_excluded: When True, the injected provider physically deletes messages that compaction excluded, bounding durable storage. This is a **lossy retention policy** and is off by default. It only affects providers this function creates. @@ -465,7 +465,7 @@ def ensure_durable_history(agent: SupportsAgentRun, *, prune_history: bool = Fal updated = [ DurableHistoryProvider( source_id=InMemoryHistoryProvider.DEFAULT_SOURCE_ID, - prune_excluded=prune_history, + prune_excluded=prune_excluded, ), *provider_list, ] @@ -473,7 +473,7 @@ def ensure_durable_history(agent: SupportsAgentRun, *, prune_history: bool = Fal replacement = DurableHistoryProvider( source_id=existing.source_id, skip_excluded=existing.skip_excluded, - prune_excluded=prune_history, + prune_excluded=prune_excluded, ) updated = [replacement if p is existing else p for p in provider_list] else: diff --git a/python/packages/durabletask/agent_framework_durabletask/_retention.py b/python/packages/durabletask/agent_framework_durabletask/_retention.py index 45b2c74..ed41bcd 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_retention.py +++ b/python/packages/durabletask/agent_framework_durabletask/_retention.py @@ -13,7 +13,6 @@ import json import logging -import warnings from typing import Literal, cast from agent_framework import ( @@ -40,7 +39,8 @@ ``auto`` Delete only under storage pressure, and only down to the low watermark. The default. ``follow_compaction`` - Also delete whatever compaction excluded, every turn. + Delete whatever compaction excluded every turn, then use the same pressure eviction as + ``auto`` if the remaining state is still too large. """ DEFAULT_RETENTION: RetentionMode = "auto" @@ -74,27 +74,6 @@ def prunes_excluded(retention: RetentionMode) -> bool: return retention == "follow_compaction" -def resolve_retention(retention: RetentionMode, prune_history: bool | None) -> RetentionMode: - """Fold the deprecated ``prune_history`` flag into the retention setting. - - Args: - retention: The retention mode the caller asked for. - prune_history: The deprecated flag, or None when it was not supplied. - - Returns: - The effective retention mode. - """ - if prune_history is None: - return retention - warnings.warn( - "prune_history is deprecated; use retention='follow_compaction' to delete what compaction " - "excluded, or retention='keep_all' to never delete.", - DeprecationWarning, - stacklevel=3, - ) - return "follow_compaction" if prune_history else retention - - async def enforce_budget(state: DurableAgentState, *, max_state_bytes: int = DEFAULT_MAX_STATE_BYTES) -> int: """Evict oldest conversation groups when persisted state approaches the backend limit. diff --git a/python/packages/durabletask/agent_framework_durabletask/_worker.py b/python/packages/durabletask/agent_framework_durabletask/_worker.py index 1581d36..81b301c 100644 --- a/python/packages/durabletask/agent_framework_durabletask/_worker.py +++ b/python/packages/durabletask/agent_framework_durabletask/_worker.py @@ -20,7 +20,6 @@ from ._callbacks import AgentResponseCallbackProtocol from ._entities import AgentEntity, DurableTaskEntityStateProvider from ._retention import DEFAULT_MAX_STATE_BYTES, DEFAULT_RETENTION, RetentionMode -from ._retention import resolve_retention as _resolve_retention from ._workflows.activity import execute_workflow_activity from ._workflows.dt_context import DurableTaskWorkflowContext from ._workflows.naming import ( @@ -83,7 +82,6 @@ def __init__( *, retention: RetentionMode = DEFAULT_RETENTION, max_state_bytes: int = DEFAULT_MAX_STATE_BYTES, - prune_history: bool | None = None, ): """Initialize the worker wrapper. @@ -92,14 +90,14 @@ def __init__( callback: Optional callback for agent response notifications retention: Default conversation retention for registered agents. ``auto`` deletes only under storage pressure, ``keep_all`` never deletes and lets the entity fail at the - backend limit, and ``follow_compaction`` also deletes what compaction excluded. + backend limit, and ``follow_compaction`` first deletes what compaction excluded, + then uses the same pressure eviction as ``auto`` if that is not enough. max_state_bytes: Budget for serialized entity state. Raise it when large payload offload is configured on the worker and client. - prune_history: Deprecated. ``True`` maps to ``follow_compaction``. """ self._worker = worker self._callback = callback - self._retention: RetentionMode = _resolve_retention(retention, prune_history) + self._retention: RetentionMode = retention self._max_state_bytes = max_state_bytes self._registered_agents: dict[str, SupportsAgentRun] = {} self._workflows: dict[str, Workflow] = {} @@ -117,7 +115,6 @@ def add_agent( *, entity_id: str | None = None, retention: RetentionMode | None = None, - prune_history: bool | None = None, ) -> None: """Register an agent with the worker. @@ -132,7 +129,6 @@ def add_agent( ``agent.name``. Workflow hosting passes the executor's ``id`` so the entity matches the identity the orchestrator dispatches to. retention: Per-agent retention override. When None, the worker-level setting is used. - prune_history: Deprecated. ``True`` maps to ``follow_compaction``. Raises: ValueError: If the agent doesn't have a name or is already registered @@ -160,7 +156,7 @@ def add_agent( agent, effective_callback, entity_id=registration_name, - retention=_resolve_retention(effective_retention, prune_history), + retention=effective_retention, max_state_bytes=self._max_state_bytes, ) diff --git a/python/packages/durabletask/tests/test_durable_history_autoswap.py b/python/packages/durabletask/tests/test_durable_history_autoswap.py index 5b9253d..fee1bc1 100644 --- a/python/packages/durabletask/tests/test_durable_history_autoswap.py +++ b/python/packages/durabletask/tests/test_durable_history_autoswap.py @@ -192,8 +192,8 @@ def test_entity_construction_does_not_mutate_the_agent(self) -> None: assert isinstance(_history_providers(agent)[0], InMemoryHistoryProvider) -class TestPruneHistoryOptIn: - """Pruning is a deployment-level retention policy, set at registration.""" +class TestFollowCompactionRetention: + """Follow-compaction retention physically deletes exclusions.""" def test_off_by_default(self) -> None: agent = _agent() @@ -205,7 +205,7 @@ def test_off_by_default(self) -> None: def test_enabled_via_registration(self) -> None: agent = _agent(context_providers=[InMemoryHistoryProvider()]) - prepared = ensure_durable_history(agent, prune_history=True) + prepared = ensure_durable_history(agent, prune_excluded=True) assert _history_providers(prepared)[0].prune_excluded is True @@ -228,7 +228,7 @@ def test_explicit_provider_configuration_wins(self) -> None: explicit = DurableHistoryProvider(prune_excluded=False) agent = _agent(context_providers=[explicit]) - prepared = ensure_durable_history(agent, prune_history=True) + prepared = ensure_durable_history(agent, prune_excluded=True) assert _history_providers(prepared)[0] is explicit assert explicit.prune_excluded is False diff --git a/python/packages/durabletask/tests/test_retention.py b/python/packages/durabletask/tests/test_retention.py index a8c914a..0034d89 100644 --- a/python/packages/durabletask/tests/test_retention.py +++ b/python/packages/durabletask/tests/test_retention.py @@ -35,7 +35,6 @@ LOW_WATERMARK, enforce_budget, prunes_excluded, - resolve_retention, ) BUDGET = 40_000 @@ -110,31 +109,6 @@ def test_only_follow_compaction_prunes_on_write(self) -> None: assert prunes_excluded("auto") is False assert prunes_excluded("keep_all") is False - def test_deprecated_flag_maps_onto_a_mode(self) -> None: - import warnings - - with warnings.catch_warnings(record=True) as caught: - warnings.simplefilter("always") - assert resolve_retention("auto", True) == "follow_compaction" - assert any(issubclass(w.category, DeprecationWarning) for w in caught) - - def test_unset_flag_leaves_the_mode_alone(self) -> None: - assert resolve_retention("auto", None) == "auto" - assert resolve_retention("keep_all", None) == "keep_all" - - def test_the_deprecated_flag_still_works_through_the_worker(self) -> None: - """Callers who set prune_history=True must keep the behavior they had.""" - import warnings - - from agent_framework_durabletask import DurableAIAgentWorker - - with warnings.catch_warnings(record=True) as caught: - warnings.simplefilter("always") - worker = DurableAIAgentWorker(cast(Any, object()), prune_history=True) - - assert worker._retention == "follow_compaction" - assert any(issubclass(w.category, DeprecationWarning) for w in caught) - def test_the_default_is_auto(self) -> None: """Which is the deliberate behavior change: previously nothing bounded storage.""" from agent_framework_durabletask import DurableAIAgentWorker @@ -338,6 +312,14 @@ async def test_state_stays_bounded_across_many_turns(self) -> None: provider, _ = await self._drive(max_state_bytes=self.LIMIT) assert len(json.dumps(provider._get_state_dict())) <= self.LIMIT + async def test_follow_compaction_falls_back_to_pressure_eviction(self) -> None: + """With nothing to prune, only the shared pressure fallback can bound this run.""" + provider, _ = await self._drive(retention="follow_compaction", max_state_bytes=self.LIMIT) + state = DurableAgentState.from_dict(provider._get_state_dict()) + + assert len(json.dumps(provider._get_state_dict())) <= self.LIMIT + assert 0 < len(state.data.conversation_history) < self.TURNS * 2 + async def test_every_turn_still_gets_its_own_answer(self) -> None: """Eviction must not disturb the response the caller is waiting on.""" _, replies = await self._drive(max_state_bytes=self.LIMIT) diff --git a/python/samples/13_conversation_compaction/README.md b/python/samples/13_conversation_compaction/README.md index 66882f1..b8a5733 100644 --- a/python/samples/13_conversation_compaction/README.md +++ b/python/samples/13_conversation_compaction/README.md @@ -44,7 +44,7 @@ app-wide on the worker. | --- | --- | | `auto` (default) | Deletes only when state approaches the backend limit, and only enough to get back under it. Nothing changes for a conversation that never gets close. | | `keep_all` | Never deletes. The entity may reach the limit and fail. Choose this when the complete record matters more than staying available. | -| `follow_compaction` | Also deletes whatever compaction excluded, every turn. The most aggressive, and the old `prune_history=True`. | +| `follow_compaction` | Deletes whatever compaction excluded every turn. If that does not free enough space, it also uses the same pressure eviction as `auto`. | `auto` exists because the alternative is an agent that simply stops working mid-conversation, with no warning. It evicts oldest-first, keeps system messages and tool-call groups intact, never touches diff --git a/python/samples/azure_functions/14_conversation_compaction/README.md b/python/samples/azure_functions/14_conversation_compaction/README.md index c3b1db7..98c4449 100644 --- a/python/samples/azure_functions/14_conversation_compaction/README.md +++ b/python/samples/azure_functions/14_conversation_compaction/README.md @@ -32,8 +32,9 @@ app = AgentFunctionApp(agents=[agent], enable_health_check=True) ``` The full conversation remains in durable storage, and compaction bounds what the *model* sees. To -also bound what is *stored*, opt in at registration with `AgentFunctionApp(..., prune_history=True)`, -which is lossy and therefore off by default. +also delete what compaction excluded, use +`AgentFunctionApp(..., retention="follow_compaction")`. It deletes exclusions every turn, then +uses the same pressure eviction as `auto` if the remaining state is still too large. ### Client-side vs service-managed history diff --git a/python/samples/azure_functions/14_conversation_compaction/function_app.py b/python/samples/azure_functions/14_conversation_compaction/function_app.py index 2a34af4..715eb4a 100644 --- a/python/samples/azure_functions/14_conversation_compaction/function_app.py +++ b/python/samples/azure_functions/14_conversation_compaction/function_app.py @@ -68,8 +68,8 @@ def _create_agent() -> Any: # 2. Register the agent with AgentFunctionApp so Azure Functions exposes the required triggers. -# Pass prune_history=True here to also delete compacted-out messages from durable storage. -# That is lossy, so the full record is kept by default. +# Set retention="follow_compaction" here to delete compacted-out messages immediately, with +# pressure eviction as a fallback if the remaining state is still too large. app = AgentFunctionApp(agents=[_create_agent()], enable_health_check=True, max_poll_retries=50) """