Add BYOK knowledge source support#264
Open
rajin-kichannagari wants to merge 11 commits into
Open
Conversation
rajin-kichannagari
requested review from
Jdubrick,
JslYoon,
gabemontero,
johnmcollier,
maysunfaisal,
michael-valdron,
thepetk and
yangcao77
as code owners
July 14, 2026 18:52
PR Summary by QodoAdd BYOK RAG knowledge source support to rolling demo
AI Description
Diagram
High-Level Assessment
Files changed (9)
|
Code Review by Qodo
Context used✅ Compliance rules (platform):
4 rules 1.
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
Comment on lines
+3
to
+6
| Sample BYOK (Bring Your Own Knowledge) vector database for the AI Rolling Demo. | ||
|
|
||
| Contains internal RHDH team docs (Feature Exploration Process, Engineering Workflow, Onboarding), built into a FAISS vector DB using [rag-content](https://github.com/lightspeed-core/rag-content) tooling. | ||
|
|
There was a problem hiding this comment.
3. Internal content checked in 🐞 Bug ⛨ Security
The PR adds internal RHDH team documentation and packages it into a BYOK vector DB/container, exposing internal processes/URLs to anyone who can access this repo or the resulting image. This is a confidentiality/security leak and can violate content handling policies.
Agent Prompt
### Issue description
Internal RHDH team docs are being committed under `byok/docs/` and explicitly used to build/distribute a prebuilt FAISS vector DB (`byok/vector_db`) that is copied into an image. This broadens distribution of internal-only information (internal URLs, processes, tooling links) and should not ship in this repository by default.
### Issue Context
This repo appears to be a demo/deployment artifact; committing internal team onboarding/workflow content (and embedding it into a distributable vector store) creates a persistent disclosure path.
### Fix Focus Areas
- byok/README.md[1-6]
- byok/Containerfile[1-2]
- byok/docs/RHDH Onboarding.md[1-11]
- byok/docs/RHDH AI Feature Exploration Process.md[1-5]
### Recommended fix
1. Remove the internal documents from the repo (and remove any derived vector DB built from them).
2. Replace them with sanitized/public sample documents (or lorem-ipsum style content) that are explicitly safe to distribute.
3. If BYOK needs to be demonstrated with internal content, move it to a private repo/artifact store and keep this repo referencing only a safe sample (or require users to build the vector DB locally from their own private sources).
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Jdubrick
requested changes
Jul 14, 2026
Jdubrick
left a comment
Jdubrick
left a comment
Contributor
There was a problem hiding this comment.
I think we might need to see if we can get unblocked on the plugin builds before merging changes
Comment on lines
1
to
+47
| kind: ConfigMap | ||
| apiVersion: v1 | ||
| metadata: | ||
| name: lightspeed-stack-config | ||
| namespace: {{ .Release.Namespace }} | ||
| data: | ||
| lightspeed-stack.yaml: | | ||
| name: lightspeed-core-stack | ||
| service: | ||
| host: ${env.SERVICE_HOST:=127.0.0.1} | ||
| host: 0.0.0.0 | ||
| port: 8080 | ||
| auth_enabled: false | ||
| workers: 1 | ||
| color_log: true | ||
| access_log: true | ||
| llama_stack: | ||
| use_as_library_client: true | ||
| config: | ||
| profile: /app-root/config.yaml | ||
| inference: | ||
| providers: | ||
| - type: vllm | ||
| id: vllm | ||
| api_key_env: VLLM_API_KEY | ||
| extra: | ||
| base_url: ${env.VLLM_URL:=} | ||
| max_tokens: ${env.VLLM_MAX_TOKENS:=4096} | ||
| network: | ||
| tls: | ||
| verify: ${env.VLLM_TLS_VERIFY:=true} | ||
| - type: openai | ||
| id: openai | ||
| api_key_env: OPENAI_API_KEY | ||
| extra: | ||
| allowed_models: | ||
| - gpt-4o-mini | ||
| - gpt-5.1 | ||
| - gpt-4.1-mini | ||
| - gpt-4.1-nano | ||
| - type: sentence_transformers | ||
| library_client_config_path: /app-root/config.yaml | ||
| user_data_collection: | ||
| feedback_enabled: true | ||
| feedback_storage: '/tmp/data/feedback' | ||
| authentication: | ||
| module: 'noop' | ||
| conversation_cache: | ||
| type: 'sqlite' | ||
| sqlite: | ||
| db_path: '/tmp/cache.db' | ||
| customization: | ||
| profile_path: '/app-root/rhdh-profile.py' | ||
| mcp_servers: | ||
| - name: mcp-integration-tools | ||
| provider_id: 'model-context-protocol' | ||
| url: 'http://localhost:7007/api/mcp-actions/v1' | ||
| provider_id: "model-context-protocol" | ||
| url: "http://localhost:7007/api/mcp-actions/v1" | ||
| authorization_headers: | ||
| Authorization: 'client' | ||
| Authorization: "client" | ||
| byok_rag: | ||
| - rag_id: custom-org-docs | ||
| rag_type: inline::faiss | ||
| embedding_model: sentence-transformers//rag-content/embeddings_model | ||
| embedding_dimension: 768 | ||
| vector_db_id: vs_727b6321-1ff4-47bf-a76b-1cc12426c954 | ||
| db_path: /rag-content/vector_db/custom_docs/faiss_store.db | ||
| score_multiplier: 1.0 | ||
| rag: | ||
| tool: | ||
| - rhdh-docs | ||
| - custom-org-docs |
Contributor
There was a problem hiding this comment.
Changes to the config should be defined in https://github.com/redhat-ai-dev/lightspeed-configs/blob/main/lightspeed-core-configs/lightspeed-stack.yaml because we have CI that runs on changes to the file. If they diverge we're going to end up overwriting this content
Add a custom BYOK RAG source using real RHDH team docs (Feature Exploration, Engineering Workflow, Onboarding) alongside the existing product docs RAG. Includes a pre-built FAISS vector DB, init container, custom lightspeed-stack-config ConfigMap with byok_rag entries, and CI overrides. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
imagePullPolicy was IfNotPresent, causing the node to reuse the cached image with an empty vector store. Switch to Always. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The lightspeed-rag-init runs last and overwrites /rag-content/vector_db/, clobbering BYOK data. Fix by staging BYOK data first, then merging it after the standard rag-content copy completes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The args override was under a duplicate global.lightspeed that wasn't being read by the subchart. Move it to the existing global.lightspeed block so Helm correctly merges it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The backstage plugin sets vector_store_ids to only product docs, which filters out BYOK stores from inline RAG. This patch overrides the filtering to always include all rag.inline stores.
Both BYOK custom-org-docs and the standard product docs vector store are now queried via inline RAG, ensuring both sources appear in referenced_documents.
Use rag.tool with rhdh-docs and custom-org-docs so file_search queries both product docs and BYOK sources. Remove the vector-search-patch ConfigMap mount since the plugin fix handles this properly now.
rajin-kichannagari
force-pushed
the
byok-knowledge-source
branch
from
July 14, 2026 19:29
50b013a to
c786a24
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds BYOK (Bring Your Own Knowledge) RAG support to the rolling demo. Includes a sample BYOK vector DB built from 3 internal RHDH team docs, configured with
rag.toolfor multi-source retrieval alongside product docs.quay.io/rh-ee-rkichann/byok-sample:latest)rag.toolfor bothrhdh-docsandcustom-org-docsvector_store_ids)How to test changes / Special notes to the reviewer
make install-no-rhoairag.toolis probabilistic — explicit prompts triggerfile_searchreliably, generic questions may not (LCOREtool_choicebehavior)rag.tool