RHIDP-14063: migrate inference providers from llama stack to lightspeed core

[Jdubrick]

What does this PR do?:

• Migrates the inference providers from the Llama Stack config.yaml to Lightspeed Core's lightspeed-stack.yaml
• Updates LCORE image to a version that has the new inference section in the config file
• Updates rolling demo sync to instead patch the LCORE config with the allowed_models for openai
• Improvements made to the Docker compose setup to allow for a *.local.yaml file to avoid changes to the git-tracked config file when adding providers
• Doc updates for enabling providers with the new migration

Which issue(s) this PR fixes:

https://redhat.atlassian.net/browse/RHIDP-14063

PR acceptance criteria:

Testing and documentation do not need to be complete in order for this PR to be approved. We just need to ensure tracking issues are opened and linked to this PR, if they are not in the PR scope due to various constraints.

• Tested and Verified

• Documentation (READMEs, Product Docs, Blogs, Education Modules, etc.)

How to test changes / Special notes to the reviewer:

[qodo-code-review]

PR Summary by Qodo

Migrate inference provider configuration from Llama Stack to Lightspeed Core

✨ Enhancement ⚙️ Configuration changes 📝 Documentation 🕐 20-40 Minutes

AI Description

• Move inference provider registration from Llama Stack profile config into lightspeed-stack.yaml.
• Update GitOps sync to generate and ship a Lightspeed Stack ConfigMap with patched providers.
• Improve local compose workflow with .local.yaml overrides and updated provider docs.

Diagram

graph TD
  WF["GitHub Actions sync"] --> GEN["generate-gitops-manifests.sh"] --> GITOPS["GitOps chart templates"]
  LS["lightspeed-stack.yaml"] --> GEN
  LL["llama-stack config.yaml"] --> GEN
  DEV["Local dev"] --> MK["Makefile/compose"] --> LCORE["Lightspeed Core"]
  LOC["lightspeed-stack.local.yaml"] --> MK

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Use yq (or a YAML-aware templating step) instead of awk patching

• ➕ Much less brittle than regex/line-match insertion (resilient to whitespace/reordering)
• ➕ Easier to validate schema and fail fast on malformed YAML
• ➕ Clearer intent: patch specific keys/arrays rather than injecting text blocks
• ➖ Adds a tooling dependency (yq) to the GitOps generation path
• ➖ Requires reworking the current simple line-anchor approach

2. Keep inference provider enablement purely in Helm values (no manifest-time patching)

• ➕ Avoids modifying generated YAML content in scripts
• ➕ Makes GitOps diffs more transparent (values-driven)
• ➕ Easier to vary per-environment without code changes
• ➖ May require chart/template changes in ai-rolling-demo-gitops
• ➖ Could increase coupling between this repo’s config structure and the chart implementation

Recommendation: The PR’s direction (centralize inference provider config in Lightspeed Core) is the right long-term move and simplifies the Llama Stack profile. The main concern is the awk-based YAML injection in generate-gitops-manifests.sh, which is sensitive to formatting/structure changes in lightspeed-stack.yaml. Accept this as an interim solution if needed for RHIDP-14063 delivery, but strongly consider a follow-up to switch the patching to a YAML-aware mechanism (yq or templating) to reduce drift and breakage risk.

Files changed (10) +171 / -80

Refactor (1) +0 / -26

config.yamlRemove inference provider definitions from Llama Stack profile config +0/-26

Remove inference provider definitions from Llama Stack profile config

• Deletes the 'providers.inference' entries (vLLM, Ollama, OpenAI, Vertex AI, sentence-transformers) from the Llama Stack profile YAML. Leaves inference provider selection to be driven by Lightspeed Core’s stack config instead.

llama-stack-configs/config.yaml

Documentation (2) +122 / -32

CONTRIBUTING.mdDocument '.local.yaml' override and new validation/provider expectations +4/-3

Document '.local.yaml' override and new validation/provider expectations

• Adds contributor guidance for using 'lightspeed-stack.local.yaml' to configure providers without touching tracked config. Updates validation instructions to reference providers configured in Lightspeed Core’s stack YAML.

docs/CONTRIBUTING.md

PROVIDERS.mdRewrite provider docs to configure inference in 'lightspeed-stack.yaml' +118/-29

Rewrite provider docs to configure inference in 'lightspeed-stack.yaml'

• Updates provider setup instructions (vLLM, Ollama via vLLM, OpenAI, Vertex AI) to use the new 'inference.providers' block in Lightspeed Core’s stack config. Documents 'api_key_env' behavior, adds guidance on 'allowed_models', and includes a full example stack YAML.

docs/PROVIDERS.md

Other (7) +49 / -22

sync-gitops.ymlSync GitOps workflow now publishes Lightspeed stack ConfigMap +3/-1

Sync GitOps workflow now publishes Lightspeed stack ConfigMap

• Extends the workflow trigger paths and copy step to include the generated 'lightspeed-stack-config.yaml'. Updates the PR body text to reflect the additional synced manifest.

.github/workflows/sync-gitops.yml

MakefileAuto-select '.local' Lightspeed stack config for local compose +6/-0

Auto-select '.local' Lightspeed stack config for local compose

• Introduces 'LIGHTSPEED_STACK_CONFIG' selection logic that prefers 'lightspeed-stack.local.yaml' when present. Exports the variable so compose can mount the chosen config without editing tracked files.

Makefile

compose.yamlMount Lightspeed stack config via 'LIGHTSPEED_STACK_CONFIG' override +1/-1

Mount Lightspeed stack config via 'LIGHTSPEED_STACK_CONFIG' override

• Changes the Lightspeed Core config volume mount to use the 'LIGHTSPEED_STACK_CONFIG' environment variable with a default fallback to the tracked stack YAML. Enables seamless local overrides via the Makefile-driven '.local.yaml' workflow.

compose/compose.yaml

default-values.envBump LCORE image and remove legacy ENABLE_* provider toggles +4/-15

Bump LCORE image and remove legacy ENABLE_* provider toggles

• Updates 'LIGHTSPEED_CORE_IMAGE' to a newer build that supports the 'inference' section in 'lightspeed-stack.yaml'. Removes the old ENABLE_* provider toggles and clarifies validation configuration expectations relative to the stack YAML.

env/default-values.env

images.yamlUpdate tracked Lightspeed Core image version +1/-1

Update tracked Lightspeed Core image version

• Bumps the 'lightspeed-core.image' reference to match the newer stack image used by default env values.

images.yaml

lightspeed-stack.yamlAdd 'inference.providers' to Lightspeed Core stack config +5/-1

Add 'inference.providers' to Lightspeed Core stack config

• Switches Llama Stack client config wiring to 'llama_stack.config.profile' and adds a baseline 'inference.providers' list (starting with 'sentence_transformers'). This becomes the new home for inference provider registration.

lightspeed-core-configs/lightspeed-stack.yaml

generate-gitops-manifests.shGenerate Lightspeed stack ConfigMap and patch providers/allowed_models for GitOps +29/-3

Generate Lightspeed stack ConfigMap and patch providers/allowed_models for GitOps

• Stops rewriting the llama-stack ConfigMap content and instead adds generation of a new 'lightspeed-stack-config.yaml' ConfigMap. Injects vLLM and OpenAI providers (including OpenAI 'allowed_models') into the generated Lightspeed stack YAML for the rolling demo GitOps flow.

scripts/generate-gitops-manifests.sh

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Context used

⚠️ Tickets: not configured — ticket URL found in PR but could not be fetched — check ticket provider credentials

1. Embeddings provider dropped ✗ Dismissed 🐞 Bug ≡ Correctness

Description

scripts/generate-gitops-manifests.sh removes the sentence_transformers provider from the generated
GitOps lightspeed-stack.yaml because the awk rule uses next and never prints the matched line.
This risks breaking RAG because the vector store configuration expects `embedding_model:
sentence-transformers//...`, which requires that provider to be registered.

Code

scripts/generate-gitops-manifests.sh[R68-87]

+    | awk '/^    - type: sentence_transformers$/ {
+        print "    - type: vllm"
+        print "      api_key_env: VLLM_API_KEY"
+        print "      extra:"
+        print "        base_url: ${env.VLLM_URL}"
+        print "        max_tokens: ${env.VLLM_MAX_TOKENS:=4096}"
+        print "        network:"
+        print "          tls:"
+        print "            verify: ${env.VLLM_TLS_VERIFY:=true}"
+        print "    - type: openai"
+        print "      api_key_env: OPENAI_API_KEY"
+        print "      extra:"
        print "        allowed_models:"
        print "          - gpt-4o-mini"
        print "          - gpt-5.1"
        print "          - gpt-4.1-mini"
        print "          - gpt-4.1-nano"
+        print
        next
      }

Evidence

The generator matches the only provider line (- type: sentence_transformers) and uses next, so
that provider is omitted from the generated output; meanwhile, RAG documentation explicitly
references sentence-transformers//... as the embedding model, implying the provider must exist.

scripts/generate-gitops-manifests.sh[56-90]
lightspeed-core-configs/lightspeed-stack.yaml[24-31]
docs/CONTRIBUTING.md[71-79]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The GitOps manifest generator replaces the `- type: sentence_transformers` line instead of preserving it, so the generated ConfigMap loses the embeddings provider.

## Issue Context
`lightspeed-core-configs/lightspeed-stack.yaml` defines `sentence_transformers` as the only inference provider, and docs reference `sentence-transformers//...` as the embedding_model used by RAG vector stores.

## Fix Focus Areas
- scripts/generate-gitops-manifests.sh[56-90]

### Suggested change
Update the awk transformation to *print the matched `sentence_transformers` line* and then append the extra providers (vllm/openai), e.g.:
- `print` the current line first, then emit additional provider blocks
- remove the `next` (or keep `next` but explicitly `print` the matched line before it)

This ensures the generated GitOps config contains `sentence_transformers` plus the patched providers.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. VLLM_URL default omitted ✓ Resolved 🐞 Bug ☼ Reliability

Description

The GitOps manifest generator emits base_url: ${env.VLLM_URL} without the repo’s standard :=
default syntax, making the generated vLLM provider configuration brittle when VLLM_URL is unset.
This diverges from both the stack config’s existing env-expansion pattern and the documented vLLM
example.

Code

scripts/generate-gitops-manifests.sh[72]

+        print "        base_url: ${env.VLLM_URL}"

Evidence

The generator prints base_url: ${env.VLLM_URL}; elsewhere the stack config uses
${env.SERVICE_HOST:=127.0.0.1} style defaulting, and the provider docs recommend
${env.VLLM_URL:=} specifically for vLLM.

scripts/generate-gitops-manifests.sh[67-77]
lightspeed-core-configs/lightspeed-stack.yaml[16-23]
docs/PROVIDERS.md[14-29]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The generated GitOps `lightspeed-stack.yaml` sets vLLM `base_url` to `${env.VLLM_URL}` (no default), which can leave an unresolved/empty placeholder if the env var is not provided.

## Issue Context
This repo’s config conventions use `${env.X:=...}` to provide safe defaults, and the provider docs show `${env.VLLM_URL:=}` for vLLM.

## Fix Focus Areas
- scripts/generate-gitops-manifests.sh[67-77]

### Suggested change
In the awk print block, change:
- `base_url: ${env.VLLM_URL}`
To:
- `base_url: ${env.VLLM_URL:=}`

(Optionally review other inserted env placeholders to ensure they follow the same defaulting convention.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[michael-valdron]

lightspeed stack deploys good, and confirmed its working with OpenAI.

I'm having connection difficulties with vLLM/Ollama though, I'm not sure if I missed something and entered something wrong but both methods are giving me similar 404 connection errors:
Ollama

[lightspeed-core] | ERROR    2026-07-08 20:02:45,583 llama_stack.providers.utils.inference.openai_mixin:525 providers::utils:               
[lightspeed-core] |          VLLMInferenceAdapter.list_provider_model_ids() failed with: Connection error.

vLLM

[lightspeed-core] | ERROR    2026-07-08 20:08:22,352 llama_stack.providers.utils.inference.openai_mixin:525 providers::utils:               
[lightspeed-core] |          VLLMInferenceAdapter.list_provider_model_ids() failed with: Error code: 404 - {'detail': 'Not Found'}

I did use the inference YAML in the docs and tried ollama host http://localhost:11434/v1 and http://host.containers.internal:11434/v1.

fwiw the vLLM problem is present before your changes so it is most likely unrelated and maybe a problem with the team cluster vLLM endpoints.

[michael-valdron]

@Jdubrick Can you confirm vLLM and ollama run on your end?

[Jdubrick]

@Jdubrick Can you confirm vLLM and ollama run on your end?

@michael-valdron vLLM and ollama are both working for me. What does your config look like?

Here is my POSTMAN result for /v1/models:

{
    "models": [
        {
            "identifier": "sentence-transformers//rag-content/embeddings_model",
            "metadata": {
                "embedding_dimension": 768
            },
            "api_model_type": "embedding",
            "provider_id": "sentence-transformers",
            "type": "model",
            "provider_resource_id": "/rag-content/embeddings_model",
            "model_type": "embedding"
        },
        {
            "identifier": "sentence-transformers/nomic-ai/nomic-embed-text-v1.5",
            "metadata": {
                "embedding_dimension": 768
            },
            "api_model_type": "embedding",
            "provider_id": "sentence-transformers",
            "type": "model",
            "provider_resource_id": "nomic-ai/nomic-embed-text-v1.5",
            "model_type": "embedding"
        },
        {
            "identifier": "vllm/all-minilm:l6-v2",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "all-minilm:l6-v2",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/deepseek-r1:7b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "deepseek-r1:7b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/deepseek-r1:8b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "deepseek-r1:8b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/gemma3:4b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "gemma3:4b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/llama3.1:8b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "llama3.1:8b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/llama3.2:3b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "llama3.2:3b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/mistral:7b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "mistral:7b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/nomic-embed-text:latest",
            "metadata": {},
            "api_model_type": "embedding",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "nomic-embed-text:latest",
            "model_type": "embedding"
        },
        {
            "identifier": "vllm/qwen3:1.7b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "qwen3:1.7b",
            "model_type": "llm"
        }
    ]
}

and this is the snippet from my config file for the providers:

inference:
  providers:
    - type: sentence_transformers
    - type: vllm
      api_key_env: VLLM_API_KEY
      extra:
        base_url: ${env.VLLM_URL:=}
        max_tokens: ${env.VLLM_MAX_TOKENS:=4096}
        network:
          tls:
            verify: ${env.VLLM_TLS_VERIFY:=true}
    - type: vllm
      extra:
        base_url: ${env.OLLAMA_URL:=http://localhost:11434/v1}

Edit: Investigating a potential collision internally in LCORE when there are > 1 vLLM providers, I don't think they uniquely name them under the hood

[michael-valdron]

Edit: Investigating a potential collision internally in LCORE when there are > 1 vLLM providers, I don't think they uniquely name them under the hood

@Jdubrick Is that where you have a model server with > 1 models and have to specify the model name or > 1 vLLM model server specified? The latter I didn't think was possible with the compose we have but perhaps with a different deployment setup it is.

[michael-valdron]

${env.OLLAMA_URL:=http://localhost:11434/v1}

@Jdubrick For the compose, do you have OLLAMA_URL set to http://localhost:11434/v1 or http://host.containers.internal:11434/v1 since its accessing from a container network?

[michael-valdron]

Edit: Investigating a potential collision internally in LCORE when there are > 1 vLLM providers, I don't think they uniquely name them under the hood

@Jdubrick Is that where you have a model server with > 1 models and have to specify the model name or > 1 vLLM model server specified? The latter I didn't think was possible with the compose we have but perhaps with a different deployment setup it is.

I guess excluding ollama + vLLM since ollama also uses a vLLM inference entry.

[michael-valdron]

@michael-valdron vLLM and ollama are both working for me. What does your config look like?

@Jdubrick My setup looks like this:

values.env:

# Note: You only need to set the variables you normally would with '-e' flags.
# You do not need to set them all if they will go unused.

# Service Images
LIGHTSPEED_CORE_IMAGE=quay.io/lightspeed-core/lightspeed-stack:dev-20260706-ca481ca
RAG_CONTENT_IMAGE=quay.io/redhat-ai-dev/rag-content:release-1.10-lls-0.5.0-8c231a3b5177f12fff9db042dfa4091d8f2f26b3

# vLLM Inference Settings
VLLM_URL=<super_secret_qwen3_model_server_url>
VLLM_API_KEY=<super_secret_qwen3_key>
# vLLM Optional Variables
VLLM_MAX_TOKENS=
VLLM_TLS_VERIFY=

# OpenAI Inference Settings
OPENAI_API_KEY=

# Vertex AI Inference Settings
VERTEX_AI_PROJECT=
VERTEX_AI_LOCATION=
GOOGLE_APPLICATION_CREDENTIALS=

# Ollama Inference Settings
OLLAMA_URL=http://host.containers.internal:11434/v1

# Question Validation Safety Shield Settings
## Ensure VALIDATION_PROVIDER is one of the providers enabled in lightspeed-stack.yaml (or lightspeed-stack.local.yaml)
## E.g. VALIDATION_PROVIDER=vllm
ENABLE_VALIDATION=
VALIDATION_PROVIDER=
VALIDATION_MODEL_NAME=

# Other
LLAMA_STACK_LOGGING=

lightspeed-stack.local.yaml:

inference:
  providers:
    - type: sentence_transformers
    - type: vllm
      api_key_env: VLLM_API_KEY
      extra:
        base_url: ${env.VLLM_URL:=}
        max_tokens: ${env.VLLM_MAX_TOKENS:=4096}
        network:
          tls:
            verify: ${env.VLLM_TLS_VERIFY:=true}
    - type: vllm
      extra:
        base_url: ${env.OLLAMA_URL:=http://localhost:11434/v1}

POSTMAN Output:

{
    "models": [
        {
            "identifier": "sentence-transformers//rag-content/embeddings_model",
            "metadata": {
                "embedding_dimension": 768
            },
            "api_model_type": "embedding",
            "provider_id": "sentence-transformers",
            "type": "model",
            "provider_resource_id": "/rag-content/embeddings_model",
            "model_type": "embedding"
        },
        {
            "identifier": "sentence-transformers/nomic-ai/nomic-embed-text-v1.5",
            "metadata": {
                "embedding_dimension": 768
            },
            "api_model_type": "embedding",
            "provider_id": "sentence-transformers",
            "type": "model",
            "provider_resource_id": "nomic-ai/nomic-embed-text-v1.5",
            "model_type": "embedding"
        }
    ]
}

[Jdubrick]

@michael-valdron

For the compose, do you have OLLAMA_URL set to http://localhost:11434/v1 or http://host.containers.internal:11434/v1 since its accessing from a container network?

Yeah, host.containers.xyz is what is set in my env when running LCORE as a container like this compose file

Is that where you have a model server with > 1 models and have to specify the model name or > 1 vLLM model server specified? The latter I didn't think was possible with the compose we have but perhaps with a different deployment setup it is.

When there is more than 1 vLLM server specified like in this change (Ollama is routed through vllm, so they collide) lightspeed-core/lightspeed-stack#2106 is open to add a way to specify an id so they don't anymore and I'll be updating this PR with it after it's merged

You're on linux, do you need to run any host network or anything?

[michael-valdron]

You're on linux, do you need to run any host network or anything?

@Jdubrick I swear I got an error last time I tried the host network but tried again just now and its now working!

{
    "models": [
        {
            "identifier": "sentence-transformers//rag-content/embeddings_model",
            "metadata": {
                "embedding_dimension": 768
            },
            "api_model_type": "embedding",
            "provider_id": "sentence-transformers",
            "type": "model",
            "provider_resource_id": "/rag-content/embeddings_model",
            "model_type": "embedding"
        },
        {
            "identifier": "sentence-transformers/nomic-ai/nomic-embed-text-v1.5",
            "metadata": {
                "embedding_dimension": 768
            },
            "api_model_type": "embedding",
            "provider_id": "sentence-transformers",
            "type": "model",
            "provider_resource_id": "nomic-ai/nomic-embed-text-v1.5",
            "model_type": "embedding"
        },
        {
            "identifier": "vllm/all-minilm:l6-v2",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "all-minilm:l6-v2",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/gemma3:4b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "gemma3:4b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/gemma3:latest",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "gemma3:latest",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/granite-code:8b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "granite-code:8b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/granite3.3:8b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "granite3.3:8b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/ibm/granite3.3:2b-base",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "ibm/granite3.3:2b-base",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/llama3.1:8b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "llama3.1:8b",
            "model_type": "llm"
        },
        {
            "identifier": "vllm/nomic-embed-text:latest",
            "metadata": {},
            "api_model_type": "embedding",
            "provider_id": "vllm",
            "type": "model",
            "provider_resource_id": "nomic-embed-text:latest",
            "model_type": "embedding"
        }
    ]
}

I believe I do see the collision you mean as I now get no errors with both ollama and vLLM but I know vLLM is not working as I don't see the qwen3 model, only the ones from my local ollama.

[michael-valdron]

lgtm

POSTMAN output:

{
    "models": [
        {
            "identifier": "ollama-provider/all-minilm:l6-v2",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "ollama-provider",
            "type": "model",
            "provider_resource_id": "all-minilm:l6-v2",
            "model_type": "llm"
        },
        {
            "identifier": "ollama-provider/gemma3:4b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "ollama-provider",
            "type": "model",
            "provider_resource_id": "gemma3:4b",
            "model_type": "llm"
        },
        {
            "identifier": "ollama-provider/gemma3:latest",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "ollama-provider",
            "type": "model",
            "provider_resource_id": "gemma3:latest",
            "model_type": "llm"
        },
        {
            "identifier": "ollama-provider/granite-code:8b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "ollama-provider",
            "type": "model",
            "provider_resource_id": "granite-code:8b",
            "model_type": "llm"
        },
        {
            "identifier": "ollama-provider/granite3.3:8b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "ollama-provider",
            "type": "model",
            "provider_resource_id": "granite3.3:8b",
            "model_type": "llm"
        },
        {
            "identifier": "ollama-provider/ibm/granite3.3:2b-base",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "ollama-provider",
            "type": "model",
            "provider_resource_id": "ibm/granite3.3:2b-base",
            "model_type": "llm"
        },
        {
            "identifier": "ollama-provider/llama3.1:8b",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "ollama-provider",
            "type": "model",
            "provider_resource_id": "llama3.1:8b",
            "model_type": "llm"
        },
        {
            "identifier": "ollama-provider/nomic-embed-text:latest",
            "metadata": {},
            "api_model_type": "embedding",
            "provider_id": "ollama-provider",
            "type": "model",
            "provider_resource_id": "nomic-embed-text:latest",
            "model_type": "embedding"
        },
        {
            "identifier": "sentence-transformers//rag-content/embeddings_model",
            "metadata": {
                "embedding_dimension": 768
            },
            "api_model_type": "embedding",
            "provider_id": "sentence-transformers",
            "type": "model",
            "provider_resource_id": "/rag-content/embeddings_model",
            "model_type": "embedding"
        },
        {
            "identifier": "sentence-transformers/nomic-ai/nomic-embed-text-v1.5",
            "metadata": {
                "embedding_dimension": 768
            },
            "api_model_type": "embedding",
            "provider_id": "sentence-transformers",
            "type": "model",
            "provider_resource_id": "nomic-ai/nomic-embed-text-v1.5",
            "model_type": "embedding"
        },
        {
            "identifier": "vllm-provider/redhataillama-31-8b-instruct",
            "metadata": {},
            "api_model_type": "llm",
            "provider_id": "vllm-provider",
            "type": "model",
            "provider_resource_id": "redhataillama-31-8b-instruct",
            "model_type": "llm"
        }
    ]
}