Skip to content

Fix initializer callback propagation in OrtGraphToProto subgraphs - #34159

Merged
Chi Lo (chilo-ms) merged 1 commit into
microsoft:mainfrom
SiluPanda:fix/graph-subgraph-initializer-callback-34137
Nov 12, 2026
Merged

Chi Lo (chilo-ms) merged 1 commit into
microsoft:mainfrom
SiluPanda:fix/graph-subgraph-initializer-callback-34137

Conversation

@SiluPanda

Copy link
Copy Markdown
Contributor

Description

Forward the initializer-data callback into OrtGraphToProto subgraph recursion, keeping the same mutable callback state throughout a traversal. Preserve the empty-callback path and both public function signatures.

The recursive call currently omits the callback, so nested initializers silently stay inline and nested callback errors are never reported. Passing the std::function by value instead would copy mutable captures into each branch, potentially reusing external-data offsets. A conditional std::ref wrapper shares the active callback during synchronous recursion without wrapping an empty function.

Adds generated, CPU-only regressions in the existing EP graph suite:

  • root, sibling and depth-two initializers, including same-named initializers with different values;
  • a mutable value-captured offset, checking every external-data location/offset/length and the corresponding bytes;
  • external-file reload and actual inference for every If branch;
  • both GraphProto/ModelProto overloads, omitted/empty callbacks, and exact nested error propagation with no subsequent callbacks.

Validation

Built and tested on macOS arm64, Apple Clang 17, CPU-only Release shared-library build, based on a8246c0d094f1ab0d6bf236228e5e67683321cd7. No minimal build or operator exclusions were used.

source .venv/bin/activate
python tools/ci_build/build.py \
  --build_dir build/MacOS --config Release --update --build --parallel 4 \
  --target onnxruntime_ep_graph_test --build_shared_lib \
  --cmake_generator Ninja --skip_submodule_sync --skip_tests \
  --cmake_extra_defines FETCHCONTENT_TRY_FIND_PACKAGE_MODE=NEVER
# Target-only builds do not run the general test target's fixture-copy step.
cmake -E copy_directory onnxruntime/test/testdata build/MacOS/Release/testdata
cd build/MacOS/Release
./onnxruntime_ep_graph_test --gtest_filter='EpGraphTest.*'
  • Before: both added regressions fail against the unmodified header. The nested fixture serializes eight initializers but invokes the callback once; nested rejection incorrectly returns success.
  • After: all 20 EP graph tests pass, including the two added regressions; no skips.
  • Mutation checks: forwarding the callback by value fails the offset/data/output assertions; unconditionally wrapping the empty callback fails the inline serialization control. The intended fix was restored, rebuilt and the full suite rerun afterward.
  • lintrunner and git diff --check pass.
Selected output from the actual local runs

Baseline:

Expected equality of these values:
  callback_calls
    Which is: 1
  inline_initializers.size()
    Which is: 8

Value of: status.IsOK()
  Actual: true
Expected: false

[  PASSED  ] 0 tests.
[  FAILED  ] 2 tests

Restored final patch:

[==========] 20 tests from 1 test suite ran. (48 ms total)
[  PASSED  ] 20 tests.

Windows execution has not been tested locally. Existing macOS telemetry duplicate-class and model-fixture warnings were not suppressed.

Motivation and Context

Fixes #34137. Thanks to Zarana Kiran Desai (@zadesai) for the reproducer and the reference-forwarding approach.

This fixes the existing header-only EP utility. #33921 concerns FLOAT16 tensor attributes, while #34014 adds a separate serialization C API; neither fixes this callback omission.

AI assistance: OpenAI Codex assisted with implementation, independent source review, and execution of the reported local tests. The results above are from those actual automated runs.

Share the active callback through synchronous subgraph recursion without wrapping an empty function. Preserve mutable callback state and propagate nested errors. Add nested If serialization, external-data round-trip, and error regressions for both overloads.

Assisted-by: OpenAI Codex
Signed-off-by: Silu Panda <31051721+SiluPanda@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 12, 2026 19:45
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the changes that propagate initializer callbacks into nested subgraphs while sharing callback state.

No open issues 🎉

What changed in this PR

Fixes #34137 in the header-only OrtGraphToProto utility by propagating initializer callbacks into nested subgraphs while sharing mutable callback state. Public signatures and serialization without a callback remain unchanged.

Changes:

  • Forward nonempty callbacks by reference during recursive serialization and document traversal-wide state sharing.
  • Add CPU regressions covering nested initializer storage, external-data reload and inference, both overloads, empty callbacks, and nested error propagation.
File Description
onnxruntime/​test/​ep_graph/​test_ep_graph.cc Adds generated nested-graph tests for callback state, external data, inference, and errors.
include/​onnxruntime/​core/​providers/​utils/​ort_graph_to_proto.h Forwards callbacks into subgraphs and documents shared callback state.

Review effort: Balanced

@SiluPanda

Copy link
Copy Markdown
Contributor Author

Hi Adrian Lizarraga (@adrianlizarraga) Chi Lo (@chilo-ms), could you please review this fix and approve the CI runs if you have access? They currently show action_required before any jobs start. CLA has passed, and all 20 EP graph tests pass locally, with both new regressions failing before the fix. Thanks for your time!

@chilo-ms
Chi Lo (chilo-ms) enabled auto-merge (squash) November 12, 2026 21:20
@chilo-ms
Chi Lo (chilo-ms) merged commit 22b26d3 into microsoft:main Nov 12, 2026
107 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OrtGraphToProto does not propagate initializer callback into subgraphs

3 participants