fix: expose output schemas to agents - #696
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the workflow engine’s structured-output behavior so API-backed agent tasks are explicitly shown the resolved output contract (routing or JSON Schema) in their task instructions, while keeping intermediate agent turns/tool calls unrestricted. It also ensures schema details are repeated during structured-output repair turns, adds regression assertions, and documents the behavior.
Changes:
- Add the resolved
output_schemacontract to API-backed agent prompt instructions, scoped to the final response only. - Refactor structured-output messaging to reuse a shared expectation builder and include the full JSON Schema in repair prompts.
- Strengthen regression tests and document how agent nodes receive and apply output contracts.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| lib/components/fabro-workflow/src/handler/structured_output.rs | Refactors expectation text generation and adds agent prompt augmentation that embeds the final-output contract (including full JSON Schema). |
| lib/components/fabro-workflow/src/handler/llm/api.rs | Tightens a repair-turn regression test to assert the JSON Schema content is actually present in the request payload. |
| lib/components/fabro-workflow/src/handler/agent.rs | Parses output_schema once per execution, injects the contract into the agent prompt, and updates tests to assert prompt contents. |
| docs/public/reference/dot-language.mdx | Documents that API-backed agent nodes receive the resolved output contract and that it applies only to the final response. |
| docs/public/agents/outputs.mdx | Adds documentation clarifying agent-node task instructions include the resolved contract without restricting intermediate tool use. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Both contract helpers dispatched on OutputSchemaKind, so they belong on the type. Moves expectation() and agent_prompt() into an impl block and drops the free functions. Splits the combined agent test: assertions no longer run inside the backend's run(), where a failure surfaces as a panic from execute(). Adds coverage for the Routing branch of the contract, which was previously untested. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Ran a simplification pass over the branch. Changed
Checked, no change needed
Worth a second look before merge
|
One conflict, in StructuredOutputError::repair_message. main (#709) added a `previous_error` parameter and richer validation-error rendering; this branch had replaced the inline expectation match with OutputSchemaKind::expectation(). Resolved by keeping both: main's new signature and section assembly, calling schema.expectation() for the expectation text. The method already supersedes main's inline match and carries this branch's intent of embedding the resolved JSON Schema instead of naming it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Summary
Why
One-shot prompt nodes pass the schema to the provider as a response format. Multi-turn agent nodes validated the final response but did not show the schema to the model. This could cause an agent to ask the user which object shape to return.
Applying a provider response format to every agent turn would interfere with tool use on some providers and is unsupported by others. The task-level final-output contract exposes the schema without changing intermediate agent turns.
Testing