Skip to content

Add structured JSON helper APIs#265

Merged
leehack merged 3 commits into
mainfrom
issue-126-structured-json-helpers
Jul 4, 2026
Merged

Add structured JSON helper APIs#265
leehack merged 3 commits into
mainfrom
issue-126-structured-json-helpers

Conversation

@leehack

@leehack leehack commented Jul 4, 2026

Copy link
Copy Markdown
Owner

Summary

Adds ergonomic structured JSON helpers for issue #126:

  • exports LlamaStructuredOutput<T> for JSON object, object-shaped JSON Schema, and value-shaped JSON Schema outputs
  • adds final-output parsing and validation, including streamed finalization with parseStructuredJson(...)
  • adds LlamaEngine.createStructuredJson(...) for typed structured JSON calls
  • documents usage in README and website generation guides

Fixes #126.

Validation

  • dart test test/unit/core/models/inference/structured_output_test.dart passed on VM and Chrome
  • dart test test/unit/core/engine/engine_test.dart passed on VM and Chrome
  • dart analyze lib/src/core/models/inference/structured_output.dart test/unit/core/models/inference/structured_output_test.dart passed
  • git diff --check passed
  • Direct real-model E2E passed:
    • dart run .dart_tool/e2e/structured_output_real_model_smoke.dart /opt/UnitySrc/personal/llama/llamadart/models/qwen2.5-0.5b-instruct-q4_k_m.gguf
    • result: RESULT structured_output_real_model {"answer":"ok","count":3}

Notes

Full root dart analyze is currently blocked by unrelated existing example package setup errors around example/basic_app imports/dependencies. The touched files analyze cleanly.

@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-265.

@leehack leehack marked this pull request as ready for review July 4, 2026 17:59
Copilot AI review requested due to automatic review settings July 4, 2026 17:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new public API layer for “structured JSON output” generation, providing a typed helper (LlamaStructuredOutput<T>) plus an engine convenience (LlamaEngine.createStructuredJson(...)) that finalizes a streamed response by parsing/validating JSON and then decoding into an application type. This aligns with issue #126’s goal of making strict JSON / JSON Schema outputs ergonomic and safer.

Changes:

  • Added LlamaStructuredOutput<T> with JSON-object and JSON-schema helpers plus final-output parsing/validation and a stream finalization extension (parseStructuredJson).
  • Added LlamaEngine.createStructuredJson(...) to apply structured response formats and decode the final validated value.
  • Updated README + website guides and added changelog entries; added unit tests for schema validation, malformed output, streaming finalization, and engine integration.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
website/docs/guides/generation-and-streaming.md Adds a “Structured JSON output” guide section and usage example.
website/docs/guides/chat-template-and-parsing.md Updates guidance for responseFormat and structured JSON helpers.
website/docs/changelog/recent-releases.md Adds an Unreleased entry for structured JSON helpers.
test/unit/core/models/inference/structured_output_test.dart Adds unit tests for schema validation, parsing failures, and streaming finalization.
test/unit/core/engine/engine_test.dart Adds test coverage for createStructuredJson forwarding grammar + decoding output.
README.md Documents structured JSON output usage and updates section numbering.
lib/src/core/models/inference/structured_output.dart Implements LlamaStructuredOutput<T>, schema subset validation, output validation, and stream parsing extension.
lib/src/core/engine/engine.dart Adds createStructuredJson(...) and references structured output helpers in docs.
lib/llamadart.dart Exports the new structured output API.
CHANGELOG.md Adds an Unreleased entry describing the new helpers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/src/core/models/inference/structured_output.dart
Comment thread lib/src/core/engine/engine.dart Outdated
Comment thread README.md Outdated
Comment thread website/docs/guides/chat-template-and-parsing.md Outdated
Comment thread website/docs/guides/generation-and-streaming.md Outdated
@leehack leehack merged commit 1bc7e98 into main Jul 4, 2026
11 checks passed
@leehack leehack deleted the issue-126-structured-json-helpers branch July 4, 2026 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(generation): add structured JSON generation helpers

2 participants