Skip to content

Model Package Support (Phase 2)#28271

Draft
chilo-ms wants to merge 15 commits intomainfrom
chi/model_package_2
Draft

Model Package Support (Phase 2)#28271
chilo-ms wants to merge 15 commits intomainfrom
chi/model_package_2

Conversation

@chilo-ms
Copy link
Copy Markdown
Contributor

This pull request introduces a new public C API for loading, inspecting, and creating sessions from ONNX Runtime model packages.

Following are the new C APIs:

namespace OrtModelPackageAPI {

ORT_API(const OrtModelPackageApi*, GetModelPackageApi);

ORT_API(void, ReleaseModelPackageOptions, _Frees_ptr_opt_ OrtModelPackageOptions*);
ORT_API_STATUS_IMPL(CreateModelPackageOptionsFromSessionOptions,
                    _In_ const OrtEnv* env,
                    _In_ const OrtSessionOptions* session_options,
                    _Outptr_ OrtModelPackageOptions** out);

ORT_API(void, ReleaseModelPackageContext, _Frees_ptr_opt_ OrtModelPackageContext*);
ORT_API_STATUS_IMPL(CreateModelPackageContext,
                    _In_ const OrtEnv* env,
                    _In_ const ORTCHAR_T* package_root,
                    _In_ const OrtModelPackageOptions* options,
                    _Outptr_ OrtModelPackageContext** out);

ORT_API_STATUS_IMPL(ModelPackageContext_GetComponentModelCount,
                    _In_ const OrtModelPackageContext* ctx,
                    _Out_ size_t* out_count);
ORT_API_STATUS_IMPL(ModelPackageContext_GetComponentModelName,
                    _In_ const OrtModelPackageContext* ctx,
                    _In_ size_t component_index,
                    _Outptr_ const char** out_name);
ORT_API_STATUS_IMPL(ModelPackageContext_GetSelectedVariantFileCount,
                    _In_ const OrtModelPackageContext* ctx,
                    _In_ const char* component_name,
                    _Out_ size_t* out_count);
ORT_API_STATUS_IMPL(ModelPackageContext_GetSelectedVariantFileIdentifier,
                    _In_ const OrtModelPackageContext* ctx,
                    _In_ const char* component_name,
                    _In_ size_t index,
                    _Outptr_ const char** out_file_identifier);

ORT_API_STATUS_IMPL(ModelPackageGetFileSessionOptions,
                    _In_ const OrtModelPackageContext* context,
                    _In_ const char* component_name,
                    _In_opt_ const char* file_identifier,
                    _Outptr_result_buffer_maybenull_(*num_entries) const char* const** option_keys,
                    _Outptr_result_buffer_maybenull_(*num_entries) const char* const** option_values,
                    _Out_ size_t* num_entries);

ORT_API_STATUS_IMPL(ModelPackageGetFileProviderOptions,
                    _In_ const OrtModelPackageContext* context,
                    _In_ const char* component_name,
                    _In_opt_ const char* file_identifier,
                    _Outptr_result_buffer_maybenull_(*num_entries) const char* const** option_keys,
                    _Outptr_result_buffer_maybenull_(*num_entries) const char* const** option_values,
                    _Out_ size_t* num_entries);

ORT_API_STATUS_IMPL(CreateSession,
                    _In_ const OrtEnv* env,
                    _In_ OrtModelPackageContext* ctx,
                    _In_ const char* component_name,
                    _In_opt_ const char* file_identifier,
                    _In_opt_ const OrtSessionOptions* session_options,
                    _Outptr_ OrtSession** session);

}

Please see the variant metadata schema:
https://github.com/microsoft/onnxruntime/blob/jambayk/model-package/docs/design_docs/model_package_api.md#variant-metadata-schema

The most important changes are:

Public C API Additions

  • Introduced new API types and functions for model package support, including OrtModelPackageApi, OrtModelPackageOptions, and OrtModelPackageContext, along with a new GetModelPackageApi() method on the main OrtApi struct. These allow clients to enumerate models in a package, query variant metadata, and create sessions for specific package components. [1] [2] [3] [4]

Documentation

  • Added a detailed API usage example and design rationale to model_package/README.md, outlining the typical workflow for using the new model package APIs and describing session/variant selection precedence.

Internal Data Structure Refactoring

  • Refactored model package and variant metadata structures to support multiple files per variant, richer compatibility metadata, and per-file session/provider options. Introduced new types: VariantEpCompatibilityInfo, VariantModelInfo, ModelVariantInfo, ComponentModelInfo, and ModelPackageInfo.
  • Updated the ModelPackageContext class to provide query APIs for enumerating component models, variants, and file-level metadata, as well as for resolving file paths and retrieving session/provider options for selected variants.

Manifest Parsing

  • Updated the manifest/metadata parser to handle the new schema, supporting multiple models per variant and richer compatibility constraints.

These changes collectively provide a robust foundation for supporting ONNX model packages in the ONNX Runtime C API, enabling advanced workflows such as variant selection based on execution provider compatibility and custom session/provider options.### Description

Motivation and Context

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant