Add structured JSON helper APIs#265
Merged
Merged
Conversation
Contributor
|
Chat app preview removed for |
There was a problem hiding this comment.
Pull request overview
Adds a new public API layer for “structured JSON output” generation, providing a typed helper (LlamaStructuredOutput<T>) plus an engine convenience (LlamaEngine.createStructuredJson(...)) that finalizes a streamed response by parsing/validating JSON and then decoding into an application type. This aligns with issue #126’s goal of making strict JSON / JSON Schema outputs ergonomic and safer.
Changes:
- Added
LlamaStructuredOutput<T>with JSON-object and JSON-schema helpers plus final-output parsing/validation and a stream finalization extension (parseStructuredJson). - Added
LlamaEngine.createStructuredJson(...)to apply structured response formats and decode the final validated value. - Updated README + website guides and added changelog entries; added unit tests for schema validation, malformed output, streaming finalization, and engine integration.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| website/docs/guides/generation-and-streaming.md | Adds a “Structured JSON output” guide section and usage example. |
| website/docs/guides/chat-template-and-parsing.md | Updates guidance for responseFormat and structured JSON helpers. |
| website/docs/changelog/recent-releases.md | Adds an Unreleased entry for structured JSON helpers. |
| test/unit/core/models/inference/structured_output_test.dart | Adds unit tests for schema validation, parsing failures, and streaming finalization. |
| test/unit/core/engine/engine_test.dart | Adds test coverage for createStructuredJson forwarding grammar + decoding output. |
| README.md | Documents structured JSON output usage and updates section numbering. |
| lib/src/core/models/inference/structured_output.dart | Implements LlamaStructuredOutput<T>, schema subset validation, output validation, and stream parsing extension. |
| lib/src/core/engine/engine.dart | Adds createStructuredJson(...) and references structured output helpers in docs. |
| lib/llamadart.dart | Exports the new structured output API. |
| CHANGELOG.md | Adds an Unreleased entry describing the new helpers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds ergonomic structured JSON helpers for issue #126:
LlamaStructuredOutput<T>for JSON object, object-shaped JSON Schema, and value-shaped JSON Schema outputsparseStructuredJson(...)LlamaEngine.createStructuredJson(...)for typed structured JSON callsFixes #126.
Validation
dart test test/unit/core/models/inference/structured_output_test.dartpassed on VM and Chromedart test test/unit/core/engine/engine_test.dartpassed on VM and Chromedart analyze lib/src/core/models/inference/structured_output.dart test/unit/core/models/inference/structured_output_test.dartpassedgit diff --checkpasseddart run .dart_tool/e2e/structured_output_real_model_smoke.dart /opt/UnitySrc/personal/llama/llamadart/models/qwen2.5-0.5b-instruct-q4_k_m.ggufRESULT structured_output_real_model {"answer":"ok","count":3}Notes
Full root
dart analyzeis currently blocked by unrelated existing example package setup errors aroundexample/basic_appimports/dependencies. The touched files analyze cleanly.