Skip to content

feat(runtime): bound model-visible tool observations #99

Description

@yuuiiikii

Problem

LangGraph currently forwards complete tool results to the model. Large JSON payloads, lists, and long text can consume most of the context window, making agent behavior, latency, and token cost less predictable. The full result is still useful for audit and debugging, so the model-facing view should be bounded without discarding the persisted raw result.

Proposal

Add an optional runtime setting for model-visible tool observations:

{
  "tool_observation": {
    "max_tokens": 2000
  }
}

When enabled, the LangGraph adapter should:

  • produce a deterministic, valid JSON preview for mappings, lists, nested values, and long strings;
  • preserve high-signal fields such as status, error, code, message, and id when possible;
  • expose truncation metadata, an estimated token count, and omitted item counts;
  • keep the complete tool result in the existing ToolCall event/persistence path;
  • treat max_tokens: 0 or an omitted setting as disabled for backward compatibility.

This first version is intentionally bounded preview/truncation, not semantic summarization. It should not call another LLM.

Scope

Python Agent Runtime and the LangGraph adapter only. No database migration, /v1 API change, Memory behavior, Webhook, or frontend work is required.

Acceptance Criteria

  • Existing configurations behave exactly as before when the setting is disabled.
  • Enabled observations remain valid JSON and respect the configured budget as closely as the existing token estimator allows.
  • Preview output is deterministic for the same input and budget.
  • Raw tool results remain available for audit/debugging.
  • Tests cover normal results, nested/long results, malformed edge cases, disabled mode, and budget boundaries.
  • Documentation includes the configuration example and the distinction between model-visible preview and persisted raw output.

If this direction fits the project roadmap, I can provide the implementation with focused tests and documentation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions