feat(lightspeed): replace RAG init container with OKP deployment - #500
feat(lightspeed): replace RAG init container with OKP deployment#500maysunfaisal wants to merge 6 commits into
Conversation
8780222 to
73fafce
Compare
|
Corresponding rhdh-operator PR redhat-developer/rhdh-operator#3370 |
There was a problem hiding this comment.
@maysunfaisal Note that the charts/backstage is being deprecated on main in favor of a brand-new standalone charts/rhdh chart - see #438
I'm just back from vacation but I'm hoping we can get #438 merged as soon as possible. So it'll be great if your changes here could be applied against this new rhdh chart instead. Thanks for your understanding.
|
/hold |
rm3l
left a comment
rm3l
left a comment
There was a problem hiding this comment.
@maysunfaisal FYI, #438 has just been merged. So it'll be great if your changes here could be applied against this new rhdh chart instead. Thanks.
@rm3l Thanks Armel, let me take a look on Tuesday |
73fafce to
25dde1b
Compare
25dde1b to
f66eed3
Compare
f66eed3 to
03f9ea9
Compare
Jdubrick
left a comment
Jdubrick
left a comment
There was a problem hiding this comment.
Generally looks good. I think there is some stale RAG references in the readmes though
|
@rm3l @gazarenkov can i also get a review on this PR alongside redhat-developer/rhdh-operator#3370 (same idea) |
|
I see that the Intelligent Assistant rebrand PR #515 merged my EOD, I wlll rebase my PR with main on Wed. |
| 3. **OKP image pull secret** — the OKP image is hosted on `registry.redhat.io`, which | ||
| requires authentication. Create a pull secret from your Red Hat registry credentials | ||
| or Podman auth: |
There was a problem hiding this comment.
Just curious, is there a public upstream version of that OKP image?
There was a problem hiding this comment.
No, there isn't a public upstream OKP image currently. The OKP image (registry.redhat.io/offline-knowledge-portal/rhokp-rhel9) is only available from the Red Hat registry, which requires authentication. On OpenShift, the cluster-wide pull secret handles this. On vanilla Kubernetes, users need to pass --set intelligentAssistant.okp.imagePullSecrets[0]= with their Red Hat registry credentials. The imagePullSecrets field is documented in the chart values and in docs/intelligent-assistant.md. If you are curious why RHOKP does not have a public upstream, you can read it here (basically because of license and RH Knowledge Base in the image).
There was a problem hiding this comment.
ack, thanks for the explanation. This kinda conflicts with the decision in https://redhat.atlassian.net/browse/RHDHPLAN-1110, but okay. Or maybe if they could push the same image to registry.access.redhat.com, unless this image really requires terms acceptance?
That said, I see that they have created https://redhat.atlassian.net/browse/RHOKP-1792
There was a problem hiding this comment.
They have created https://redhat.atlassian.net/browse/RHOKP-1792 but in the Slack message I linked in the comments, they do not currently support public unauth image because of terms like you said. They have Knowledge Articles in the RHOKP image and they require auth for users to access that. The PMs from RHOKP and RHDH (Ben) might be meeting soon-ish; but i dont think we will have it available in time for 2.1 FF.
I checked out podman pull for OKP image from https://catalog.redhat.com/en/software/containers/offline-knowledge-portal/rhokp-rhel9/680143c03b895f32bbd104c2?image=6a96e70fbff9e5e7a59ba25c&architecture=arm64#unauthenticated
- registry.redhat.io — fails with unauthorized, needs auth
- registry.access.redhat.com — fails with This repo requires terms acceptance and is only available on registry.redhat.io
There was a problem hiding this comment.
But reading https://redhat.atlassian.net/browse/RHDHPLAN-1110, registry.redhat.io/offline-knowledge-portal/rhokp-rhel9 is not an RHDH image though... it is a separate RH product image.
d704d2b to
e0f0d47
Compare
ba42515 to
9e7a344
Compare
| {{- printf "http://%s" $ia.okp.ingress.host -}} | ||
| {{- end -}} | ||
| {{- else if .Values.openshift.clusterRouterBase -}} | ||
| {{- printf "http://%s-%s.%s" $fullname .Release.Namespace .Values.openshift.clusterRouterBase -}} |
There was a problem hiding this comment.
| {{- printf "http://%s-%s.%s" $fullname .Release.Namespace .Values.openshift.clusterRouterBase -}} | |
| {{- printf "https://%s-%s.%s" $fullname .Release.Namespace .Values.openshift.clusterRouterBase -}} |
HTTPS, I guess?
There was a problem hiding this comment.
Keeping http:// for now. OKP_SERVICE_URL serves a dual purpose in LCORE — it's used for the backend Solr/OKP connection AND returned as citation URLs in chatbot responses that users click to view referenced documentation pages.
Ideally, LCORE's backend connection would use the cluster-internal Service URL (http://svc:8080) which never leaves the cluster network, while citation links would use the external Route URL (https://...). However, LCORE currently uses a single env var for both purposes, so splitting them requires an upstream LCORE change.
Since we must use the Route URL, I tried https:// but LCORE fails with SSL: CERTIFICATE_VERIFY_FAILED — the cluster's self-signed router CA isn't in LCORE's trust store (cluster has self signed certs via openshift installer). Setting REQUESTS_CA_BUNDLE to the SA CA bundle fixes the Route connection but replaces the entire system CA bundle, breaking LCORE's ability to reach external HTTPS endpoints (e.g., tiktoken encoding downloads).
The OKP Route still terminates TLS for end users via insecureEdgeTerminationPolicy: Allow, so external traffic is protected regardless. (Had to do Agentic debugging for this)
There was a problem hiding this comment.
The OKP Route still terminates TLS for end users via insecureEdgeTerminationPolicy: Allow, so external traffic is protected regardless.
Are you sure about this? Yes, it terminates TLS, but this will be useless if you accept plain public-facing HTTP URLs. Because you are defaulting to Allow in the values file, the response will be sent unencrypted if the request is HTTP, which might be a security concern OOTB IMO. cc @kim-tsao
But if a user sets it to Redirect for example, HTTP traffic will automatically be redirected to HTTPS (3xx status code response with the actual HTTPS URL that they need to follow). See the description of the field in https://docs.okd.io/4.20/rest_api/network_apis/route-route-openshift-io-v1.html
This means that the LCORE backend would still face the same certificate issues, I think.
Note that the Route might also be annotated to enforce HSTS, which means that HTTP will not even be allowed and clients will just be informed by the server to change their HTTP requests to HTTPS (no Redirect at all).
Also, IIUC, if this is being used as public-facing citation URLs, it also means that the {{- else -}} {{- printf "http://%s.%s.svc.cluster.local:8080" $fullname .Release.Namespace -}} case below should be an error, no? Otherwise, it would be returning a cluster-internal service url that will not be routable from the outside.
It would indeed be better to have different ways to configure public-facing citation URLs and internal backend connection URLs. Just curious, has this been reported to the LCORE folks? If not, it should be tracked so we can fix this later.
There was a problem hiding this comment.
Updated,
Switched OKP_SERVICE_URL to https:// on OpenShift and insecureEdgeTerminationPolicy to Redirect.
To handle self-signed router certs, the LCORE container gets a command override that combines the system CA bundle with the cluster's service account CA and exports SSL_CERT_FILE + REQUESTS_CA_BUNDLE before calling the original entrypoint.
On vanilla K8s, OKP defaults to HTTP; users can opt into HTTPS via okp.ingress.tls.enabled=true with a TLS secret.
Tested on OpenShift (self-signed certs) and Kind (HTTP + HTTPS Ingress).
OpenShift:
LCORE container spec:
- resources:
limits:
cpu: '1'
memory: 2Gi
requests:
cpu: 100m
memory: 512Mi
terminationMessagePath: /dev/termination-log
name: lightspeed-core
command:
- /bin/sh
- '-c'
env:
- name: OKP_SERVICE_URL
value: 'https://rhdh-redhat-developer-hub-ia-okp-rhdh-test.apps.mfaisal-sep042026.rhdh-pai.devfile-ci.com'
- name: KV_STORE_PATH
value: /tmp/kvstore.db
- name: SQL_STORE_PATH
value: /tmp/sql_store.db
- name: SQLITE_STORE_DIR
value: /tmp/llama-stack-files
- name: HF_HOME
value: /tmp/hf_cache
- name: OTEL_SDK_DISABLED
value: 'true'
securityContext:
capabilities:
drop:
- ALL
runAsUser: 1000740000
runAsNonRoot: true
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
seccompProfile:
type: RuntimeDefault
ports:
- name: http-lightspeed
containerPort: 8080
protocol: TCP
imagePullPolicy: IfNotPresent
volumeMounts:
- name: lightspeed-data
mountPath: /tmp
- name: lightspeed-rag
mountPath: /rag-content
- name: lightspeed-config-stack
readOnly: true
mountPath: /app-root/lightspeed-stack.yaml
subPath: lightspeed-stack.yaml
- name: lightspeed-config-profile
readOnly: true
mountPath: /app-root/rhdh-profile.py
subPath: rhdh-profile.py
- name: kube-api-access-cc9hg
readOnly: true
mountPath: /var/run/secrets/kubernetes.io/serviceaccount
terminationMessagePolicy: File
envFrom:
- secretRef:
name: lightspeed-secret
image: 'quay.io/lightspeed-core/lightspeed-stack:dev-20260824-cbd182b'
args:
- |
if [ -f /var/run/secrets/kubernetes.io/serviceaccount/ca.crt ]; then
cat /etc/pki/tls/certs/ca-bundle.crt /var/run/secrets/kubernetes.io/serviceaccount/ca.crt > /tmp/combined-ca-bundle.crt
export SSL_CERT_FILE=/tmp/combined-ca-bundle.crt
export REQUESTS_CA_BUNDLE=/tmp/combined-ca-bundle.crt
fi
exec /app-root/entrypoint.sh --synthesized-config-output /tmp/.generated/run.yaml
Kind:
Ingress Spec:
spec:
ingressClassName: nginx
rules:
- host: okp.127.0.0.1.nip.io
http:
paths:
- backend:
service:
name: rhdh-redhat-developer-hub-ia-okp
port:
number: 8080
path: /
pathType: Prefix
tls:
- hosts:
- okp.127.0.0.1.nip.io
secretName: okp-tls
status:
loadBalancer:
ingress:
- hostname: localhost
LCORE container spec:
- args:
- |
if [ -f /var/run/secrets/kubernetes.io/serviceaccount/ca.crt ]; then
cat /etc/pki/tls/certs/ca-bundle.crt /var/run/secrets/kubernetes.io/serviceaccount/ca.crt > /tmp/combined-ca-bundle.crt
export SSL_CERT_FILE=/tmp/combined-ca-bundle.crt
export REQUESTS_CA_BUNDLE=/tmp/combined-ca-bundle.crt
fi
exec /app-root/entrypoint.sh --synthesized-config-output /tmp/.generated/run.yaml
command:
- /bin/sh
- -c
env:
- name: OKP_SERVICE_URL
value: https://okp.127.0.0.1.nip.io
- name: KV_STORE_PATH
value: /tmp/kvstore.db
- name: SQL_STORE_PATH
value: /tmp/sql_store.db
- name: SQLITE_STORE_DIR
value: /tmp/llama-stack-files
- name: HF_HOME
value: /tmp/hf_cache
- name: OTEL_SDK_DISABLED
value: "true"
envFrom:
- secretRef:
name: lightspeed-secret
image: quay.io/lightspeed-core/lightspeed-stack:dev-20260824-cbd182b
imagePullPolicy: IfNotPresent
If Ingress is without TLS then LCORE starts with default entrypoint
There was a problem hiding this comment.
Also opened a request to enable internal and external URLs on LCORE JIRA https://redhat.atlassian.net/browse/LCORE-3965
| 3. **OKP image pull secret** — the OKP image is hosted on `registry.redhat.io`, which | ||
| requires authentication. Create a pull secret from your Red Hat registry credentials | ||
| or Podman auth: |
There was a problem hiding this comment.
ack, thanks for the explanation. This kinda conflicts with the decision in https://redhat.atlassian.net/browse/RHDHPLAN-1110, but okay. Or maybe if they could push the same image to registry.access.redhat.com, unless this image really requires terms acceptance?
That said, I see that they have created https://redhat.atlassian.net/browse/RHOKP-1792
There was a problem hiding this comment.
let's rename this file since there is no more lightspeed => with-intelligent-assistant-servide-host-values.yaml.
| {{- if $ia.okp.imagePullSecrets }} | ||
| imagePullSecrets: | ||
| {{- range $ia.okp.imagePullSecrets }} | ||
| - name: {{ . | quote }} | ||
| {{- end }} | ||
| {{- end }} |
There was a problem hiding this comment.
The chart exposes a global.imagePullSecrets (shared with the other subcharts like postgresql). This should be merged with the okp-specific pull secrets as well. See https://github.com/maysunfaisal/rhdh-chart/blob/c7b124ebb2c338ed66491b5220d953f198a43305/charts/rhdh/templates/_helpers.tpl#L87-L112 and https://github.com/maysunfaisal/rhdh-chart/blob/c7b124ebb2c338ed66491b5220d953f198a43305/charts/rhdh/templates/deployment.yaml#L49
| metadata: | ||
| labels: | ||
| {{- include "rhdh.intelligentAssistant.okp.selectorLabels" . | nindent 8 }} | ||
| spec: |
There was a problem hiding this comment.
I would suggest allowing the configuration of PodSpec fields like affinity, topologySpreadConstraints, nodeSelector, tolerations, so users can control the pod scheduling and placement. I recall from past customer cases that some users had different requirements related to this.
| readinessProbe: | ||
| httpGet: | ||
| path: / | ||
| port: httpd | ||
| initialDelaySeconds: 10 | ||
| periodSeconds: 10 | ||
| livenessProbe: | ||
| tcpSocket: | ||
| port: solr | ||
| initialDelaySeconds: 30 | ||
| periodSeconds: 30 |
There was a problem hiding this comment.
would be great for all these probes to be configurable as well from the values file, at least the periods and delays. Containers might be slower in some environments.
| {{- printf "http://%s" $ia.okp.ingress.host -}} | ||
| {{- end -}} | ||
| {{- else if .Values.openshift.clusterRouterBase -}} | ||
| {{- printf "http://%s-%s.%s" $fullname .Release.Namespace .Values.openshift.clusterRouterBase -}} |
There was a problem hiding this comment.
The OKP Route still terminates TLS for end users via insecureEdgeTerminationPolicy: Allow, so external traffic is protected regardless.
Are you sure about this? Yes, it terminates TLS, but this will be useless if you accept plain public-facing HTTP URLs. Because you are defaulting to Allow in the values file, the response will be sent unencrypted if the request is HTTP, which might be a security concern OOTB IMO. cc @kim-tsao
But if a user sets it to Redirect for example, HTTP traffic will automatically be redirected to HTTPS (3xx status code response with the actual HTTPS URL that they need to follow). See the description of the field in https://docs.okd.io/4.20/rest_api/network_apis/route-route-openshift-io-v1.html
This means that the LCORE backend would still face the same certificate issues, I think.
Note that the Route might also be annotated to enforce HSTS, which means that HTTP will not even be allowed and clients will just be informed by the server to change their HTTP requests to HTTPS (no Redirect at all).
Also, IIUC, if this is being used as public-facing citation URLs, it also means that the {{- else -}} {{- printf "http://%s.%s.svc.cluster.local:8080" $fullname .Release.Namespace -}} case below should be an error, no? Otherwise, it would be returning a cluster-internal service url that will not be routable from the outside.
It would indeed be better to have different ways to configure public-facing citation URLs and internal backend connection URLs. Just curious, has this been reported to the LCORE folks? If not, it should be tracked so we can fix this later.
There was a problem hiding this comment.
Another note: I think the RHDH must-gather should also be updated to collect these new OKP resources, so we can better troubleshoot potential customer issues related to OKP. I've quickly tested it and it is not handling this correctly because it currently expects only one RHDH Deployment, not additional Deployments.
Maybe this is already tracked and I missed it, but I haven't seen anything related to this under https://redhat.atlassian.net/browse/RHDHPLAN-1189
There was a problem hiding this comment.
I do not have anything targeted for RHDH must-gather. Let me see that repo, I have not checked it out previously. Thx.
…DP-16102] Replace the RAG init container / FAISS vector store with an OKP (Offline Knowledge Portal) Deployment, Service, and Route. Add platform-aware config selection: OpenShift (or K8s with okp.ingress.host set) uses the full OKP config; vanilla K8s uses the lightspeed-stack-no-okp.yaml variant so LCORE does not crash without an OKP backend. Add OKP Ingress support, imagePullSecrets, vector_store config, and align/pin the OKP and LCORE (OGX) images. Swap Lightspeed plugins to Intelligent Assistant. Add systemArgs/systemEnv for LCORE sidecar (HF_HOME, OTEL_SDK_DISABLED, --synthesized-config-output). Sync vendored Lightspeed configs from lightspeed-configs@main. Add docs/lightspeed.md with deployment scenarios, vanilla K8s prerequisites, and OKP configuration reference. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…[RHIDP-16102] - Add securityContext to OKP deployment (allowPrivilegeEscalation: false, runAsNonRoot: true, capabilities.drop: [ALL], seccompProfile: RuntimeDefault) - Fix CI test failure: use lightspeed-stack-no-okp.yaml for upgrade test ConfigMap to avoid LCORE crash against missing OKP services - Remove OTEL_SDK_DISABLED from secret.example.yaml (managed by chart) - Remove stale RAG init container references from README and values.yaml Co-authored-by: Cursor <cursoragent@cursor.com>
…on [RHIDP-16102] - Replace Capabilities.APIVersions.Has with openshift.route.enabled for ArgoCD/helm-template compatibility - Move OKP securityContext from hardcoded template to values.yaml - Add readiness and liveness probes to OKP deployment - Remove systemArgs/systemEnv (hardcoded in deployment template instead) - Remove unused chunkFilterQuery field - Rename lightspeed-okp component labels to intelligent-assistant-okp - Rename docs/lightspeed.md to docs/intelligent-assistant.md - Update OKP image tag to 1.2.12-1788274041 - Regenerate schema and README Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…6102] - Bump chart version from 2.1.0 to 2.2.0 and regenerate README - Fix okp.active helper to require okp.route.enabled on OpenShift (prevents OKP stack from activating in CI Kind environments) - Disable OKP in CI values files (default, custom-config, external-db) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…DP-16102] - Shorten OKP resource name to avoid Route hostname >63 char overflow - Add comment explaining why readOnlyRootFilesystem is not set for OKP - Remove unused Solr admin port (8983) from OKP Service Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…g fields [RHIDP-16102] Address rm3l third round review: switch OKP_SERVICE_URL to HTTPS with combined CA bundle (system + cluster self-signed), merge global and OKP imagePullSecrets, add scheduling fields (affinity, nodeSelector, tolerations, topologySpreadConstraints), make probes configurable, rename CI values file. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
c7b124e to
3e0a4cd
Compare
|
3 participants
Description of the change
Replace the Lightspeed RAG init container / FAISS vector store with an OKP (Offline Knowledge Portal) backend and swap the Lightspeed dynamic plugins to the Intelligent Assistant plugins.
OKP deployment
Deployment,Service,Route, andIngresstemplates (Solr8983+ httpd8080) undertemplates/lightspeed/._helpers.tpl): OpenShift — or K8s withokp.ingress.hostset — uses the full OKP config; vanilla K8s useslightspeed-stack-no-okp.yamlso LCORE does not crash without an OKP backend.okp.*values (image, replicas, Solr memory, ingress, imagePullSecrets); OKP image pinned toregistry.redhat.io/offline-knowledge-portal/rhokp-rhel9:1.2.10-1786628394.Plugin swap (Intelligent Assistant)
values.yamlFE + BE plugins →oci://quay.io/rhdh/red-hat-developer-hub-backstage-plugin-intelligent-assistant(-backend):{{ "{{inherit}}" }},enabled: true.Config & images
lightspeed-stack.yaml(OKP) +lightspeed-stack-no-okp.yaml(no-OKP), addvector_storeconfig, updaterhdh-profile.py/secret.yaml.quay.io/lightspeed-core/lightspeed-stack:dev-20260811-0da4b12); drop the RAG init container frombackstage-deployment.yaml.lightspeed-configs@main(hack/sync-lightspeed-configs.sh).Which issue(s) does this PR fix or relate to
https://redhat.atlassian.net/browse/RHIDP-16102
How to test changes / Special notes to the reviewer
On OpenShift:
lightspeedenabled and provide your inference key (e.g.OPENAI_API_KEY) via the Lightspeed secret.lightspeed-coresidecar (no RAG init container); OKPDeployment/Service/Route1/1 Running.OKP_SERVICE_URLinjected into the sidecar; LCOREGET /v1/modelsreturns 200.On vanilla K8s:
okp.ingress.host: OKP is skipped,lightspeed-stack-no-okp.yamlis used, LCORE starts without crashing.okp.ingress.hostset: OKP + Ingress are deployed and used.Checklist
Chart.yamlaccording to Semantic Versioning.values.yamland added to the corresponding README.md (regenerated via thehelm-docspre-commit hook).pre-commithook.ct lintcommand.orchestrator-infrachart not touched.