global-agent.bootstrap() breaks TLS verification for any plugin using a custom https.Agent

[juanpsm]

Related context

• RHDHBUGS-3304 (backlog): proposes dropping undici/global-agent in favor of Node 22.21+'s built-in corporate proxy support. This report provides a concrete bug that motivates that migration.
• RHDHBUGS-914 (closed/done): same exact symptom in CI e2e tests for the Kubernetes plugin (unable to verify the first certificate), but was attributed to a missing NODE_TLS_REJECT_UNAUTHORIZED=0/skipTLSVerify in the test config. The actual root cause may be this one instead.
• 🐛 Bug Report: Kubernetes plugin certificate attachment and skipTlsverify not working backstage/backstage#28486 (upstream, closed as stale, never diagnosed): same symptom, reporter mentions going through a corporate proxy.

Description

configureCorporateProxyAgent() is called unconditionally on backend startup (packages/backend/dist/index.cjs.js), even when no corporate proxy is configured at all (no HTTP_PROXY/HTTPS_PROXY/GLOBAL_AGENT_HTTP_PROXY set). That function calls global-agent's bootstrap(), which by default (forceGlobalAgent: true) unconditionally replaces the agent of any outgoing http/https request with its own (see global-agent's bindHttpMethod.js: options.agent = agent with no condition when forceGlobalAgent is true).

This silently breaks any plugin that builds its own https.Agent with a custom CA or client certs (for example, the Kubernetes plugin against a cluster with an internal CA, or a self-hosted GitLab/Git plugin with its own CA): the plugin's Agent gets discarded, the request goes out through global-agent's own Agent instead, without the custom CA, and TLS validation fails.

Steps to reproduce

1. Configure the Kubernetes plugin (@backstage/plugin-kubernetes-backend) against a cluster with an internal CA, via caFile + serviceAccountToken (not the automatic in-cluster mode).
2. With no proxy configured, start the backend normally.
3. The Kubernetes tab of any entity shows: Error communicating with Kubernetes: FETCH_ERROR, message: request to https://... failed, reason: unable to verify the first certificate.

Diagnosis (isolated on a real environment, RHDH 2.0 / rhdh-hub-rhel9:2.0-80)

By instrumenting the actual running code (injecting console.log into the Kubernetes plugin's fetcher, inside the same process):

• The https.Agent built by the plugin carries the correct CA (verified by sha256, matches the mounted file byte for byte).
• A raw https.request using that exact agent, from inside the real process: fails with UNABLE_TO_VERIFY_LEAF_SIGNATURE.
• A brand new https.Agent created at that same moment, with the same CA, in the same process: also fails.
• The same CA passed directly in the request options (no agent): 200 OK.
• The exact same code, run as a standalone script (without loading packages/backend/dist/index.cjs.js, so without bootstrapping global-agent): always works.

Setting GLOBAL_AGENT_FORCE_GLOBAL_AGENT=false (which makes global-agent only replace the agent when the caller didn't specify one) makes the problem disappear completely.

Expected behavior

configureCorporateProxyAgent() shouldn't bootstrap global-agent with forceGlobalAgent: true by default, or this should at least be clearly documented, since it currently breaks any plugin with a custom Agent completely silently (no log warns that the agent was replaced).

Workaround

Env var GLOBAL_AGENT_FORCE_GLOBAL_AGENT=false.

Version

RHDH 2.0 (image quay.io/rhdh/rhdh-hub-rhel9:2.0-80)

[github-actions]

This issue is stale because it has been open 30 days with no activity. This project switched to Jira at https://issues.redhat.com/projects/RHIDP/ for issue tracking. If this issues is still relevant, please reopen it there.