Skip to content

fix(backend): replace full CRD list fetch with targeted GET in /hub route (ACM-38058)#6530

Open
fxiang1 wants to merge 1 commit into
stolostron:mainfrom
fxiang1:fix/acm-38058-hub-crd-memory-spike
Open

fix(backend): replace full CRD list fetch with targeted GET in /hub route (ACM-38058)#6530
fxiang1 wants to merge 1 commit into
stolostron:mainfrom
fxiang1:fix/acm-38058-hub-crd-memory-spike

Conversation

@fxiang1

@fxiang1 fxiang1 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Root Cause Analysis

The /hub route handler in backend/src/routes/hub.ts fetched the entire CRD list (~51MB) from the API server every 30 seconds per connected browser tab, just to check if a single CRD (multiclusterglobalhubs.operator.open-cluster-management.io) exists:

GET /apis/apiextensions.k8s.io/v1/customresourcedefinitions

This caused repeated massive memory allocations/deallocations in the console-chart-console-v2 pod, leading to memory spikes.

Fix

Replace the full CRD list fetch with a targeted GET for the specific CRD by name:

GET /apis/apiextensions.k8s.io/v1/customresourcedefinitions/multiclusterglobalhubs.operator.open-cluster-management.io

This reduces the response from ~51MB to ~10KB per call. The targeted GET returns:

  • 200 with kind: CustomResourceDefinition when the CRD exists → isGlobalHub: true
  • 404 with kind: Status when it does not → isGlobalHub: false

Also removed the now-unused ResourceList type import.

Regression Test

Added a test case that mocks a 404 response for the targeted CRD GET and verifies the handler correctly returns isGlobalHub: false. Updated existing test mocks to match the new targeted endpoint.

How to Test

  1. Run backend tests: npm run test:backend — all 338 tests pass (including 3 hub tests)
  2. Run checks: npm run check:backend — prettier, lint, and tsc all pass
  3. Start in plugin mode (npm run plugins) and verify the clusters page loads correctly
  4. Inspect the /hub network response — should contain isGlobalHub with correct value
    If the Multiclusterglobalhub operator is installed isGlobalHub: true otherwise isGlobalHub: false

Linked Issue

Resolves ACM-38058

Summary by CodeRabbit

  • Bug Fixes

    • Improved detection of global hub availability for more reliable hub status reporting.
    • Hub status now correctly reports a non-global hub when the required resource is unavailable.
  • Tests

    • Added coverage for scenarios where the global hub resource does not exist.
    • Updated endpoint validation for direct resource checks.

…oute (ACM-38058)

The /hub route fetched the entire CRD list (~51MB) every 30 seconds per
connected browser tab just to check if one specific CRD exists. This
caused repeated massive memory allocations in the console-v2 pod.

Replace the full list fetch with a targeted GET for the specific CRD by
name, reducing the response from ~51MB to ~10KB per call. The targeted
GET returns 200 when the CRD exists and 404 when it does not, which is
detected by checking the response kind.

Add regression test for the 404 case (CRD not found → isGlobalHub: false).

Signed-off-by: fxiang1 <fxiang@redhat.com>
@openshift-ci

openshift-ci Bot commented Jul 21, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fxiang1

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

The hub route now queries the multiclusterglobalhubs CRD directly and identifies global hub status from the returned resource kind. Tests update the CRD mock and verify that a 404 response produces isGlobalHub: false.

Changes

Hub CRD detection

Layer / File(s) Summary
Direct CRD lookup and fallback validation
backend/src/routes/hub.ts, backend/test/routes/hub.test.ts
The route replaces CRD list retrieval with a named CRD request, while tests mock the direct endpoint and verify the missing-CRD response and default authentication shape.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed Concise and specific title matches the main change: targeted CRD GET in /hub for ACM-38058.
Description check ✅ Passed Includes root cause, fix, regression test, validation steps, and linked issue; the repo template headings and checklist are not followed exactly.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

@fxiang1
fxiang1 requested a review from KevinFCormier July 21, 2026 20:01
@KevinFCormier

Copy link
Copy Markdown
Contributor

@fxiang1 IIRC, I think this was coded this way as a lazy way of avoiding adjusting the ServiceAccount permissions. I think we have permission to list/watch CRDs, but not get CRDs. Can you check the installer projects?

Were you able to observe memory growth because of this pattern?

@KevinFCormier

Copy link
Copy Markdown
Contributor

/hold for permission updates

@KevinFCormier

Copy link
Copy Markdown
Contributor

/cc @zlayne

@openshift-ci
openshift-ci Bot requested a review from zlayne July 21, 2026 20:42
@KevinFCormier

Copy link
Copy Markdown
Contributor

/uncc @KevinFCormier

@openshift-ci
openshift-ci Bot removed the request for review from KevinFCormier July 21, 2026 20:42
@fxiang1

fxiang1 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@fxiang1 IIRC, I think this was coded this way as a lazy way of avoiding adjusting the ServiceAccount permissions. I think we have permission to list/watch CRDs, but not get CRDs. Can you check the installer projects?

Were you able to observe memory growth because of this pattern?

@KevinFCormier Thanks! you're right we don't have get permission on CRDs, only list/watch.

I don't think this is causing the memory leaks but I do see a temporary memory spike. I was testing with 10000 CRDs though but then I realized it's probably not realistic to have that many. So this change won't buy us much.

@zlayne

zlayne commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

These changes look good to me, assuming we are okay with adding GET permissions for CRDs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants