Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/fresh-logins-settle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@hyperdx/app': patch
---

Fix repeated login redirects after a session expires by clearing cached
authentication state with a full navigation to the login page.
8 changes: 8 additions & 0 deletions .changeset/keycloak-oidc-login.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
'@hyperdx/api': minor
'@hyperdx/app': minor
'@hyperdx/common-utils': patch
---

Add optional Keycloak OIDC login with invitation-based provisioning, explicit
identity mapping, existing Passport sessions, and an opt-in required SSO mode.
9 changes: 9 additions & 0 deletions .changeset/quiet-teams-expire.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@hyperdx/api': patch
'@hyperdx/app': patch
'@hyperdx/common-utils': patch
---

Add optional session idle and absolute limits, and a team administrator
allowlist for member, source, connection, and integration management across the
UI and APIs.
37 changes: 37 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,43 @@ ClickHouse).
<br/>
<img alt="Search logs and traces all in one place" src="./.github/images/search_splash.png" title="Search logs and traces all in one place">

## Authentication changes in this fork

This fork adds Keycloak OIDC login to HyperDX's existing Passport and MongoDB
sessions. It also adds configurable session limits and an optional team settings
allowlist for Members, Data, and Integrations, enforced by the API and MCP as

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 minor — Upstream README gains a fork-specific section

The new "Authentication changes in this fork" section says to "build images from this branch" and that "the upstream images below do not include this fork's additions". That is wrong once this is merged into hyperdxio/hyperdx. Fix: drop the fork framing and keep a short pointer to docs/keycloak.md.

Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond

well as the UI. Existing authentication and team access remain unchanged until
these options are configured.

Set the following on the API server (keep secrets out of frontend variables):

```dotenv
FRONTEND_URL=https://hyperdx.example.com
KEYCLOAK_ISSUER=https://identity.example.com/realms/hyperdx
KEYCLOAK_CLIENT_ID=hyperdx
KEYCLOAK_CLIENT_SECRET=<client secret>
KEYCLOAK_REQUIRED=false
SESSION_MAX_AGE_SECONDS=86400
SESSION_IDLE_TIMEOUT_SECONDS=1800
```

Configure the Keycloak client with the exact callback
`https://hyperdx.example.com/api/login/keycloak/callback` and post-logout URI
`https://hyperdx.example.com/login`. Users need an explicitly linked HyperDX
account or a valid team invitation; matching email addresses never link accounts
automatically. Verify login before setting `KEYCLOAK_REQUIRED=true` to disable
password login and existing password sessions.

The sample session policy is a 24-hour maximum with a 30-minute idle timeout.
Configure Keycloak's session limits separately. Keycloak and HyperDX sessions
are independent; back-channel logout is not implemented.

See the [Keycloak setup guide](./docs/keycloak.md) for client settings, account
migration, proxy configuration, settings permissions, and
[regression test commands](./docs/keycloak.md#regression-tests). Build images
from this branch to use these changes; the upstream images below do not include
this fork's additions.

## Spinning Up HyperDX

HyperDX can be deployed as part of ClickStack, which includes ClickHouse,
Expand Down
4 changes: 4 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@ services:
# tokens; unset means they are stored in plain text.
TOKEN_ENCRYPTION_KEY: ${TOKEN_ENCRYPTION_KEY:-}
EXPRESS_SESSION_SECRET: ${EXPRESS_SESSION_SECRET:-}
KEYCLOAK_ISSUER: ${KEYCLOAK_ISSUER:-}
KEYCLOAK_CLIENT_ID: ${KEYCLOAK_CLIENT_ID:-}
KEYCLOAK_CLIENT_SECRET: ${KEYCLOAK_CLIENT_SECRET:-}
KEYCLOAK_REQUIRED: ${KEYCLOAK_REQUIRED:-false}
# Uncomment the next two lines to enable PromQL (Prometheus-compatible metrics)
# ENABLE_PROMQL: 'true'
# NEXT_PUBLIC_ENABLE_PROMQL: 'true'
Expand Down
181 changes: 181 additions & 0 deletions docs/keycloak.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# Keycloak authentication

Keycloak authenticates users; HyperDX retains its users, teams, personal API
keys, and authorization rules. Realm roles and groups do not grant HyperDX
access. Password login remains available until required mode is enabled.

## Configuration

Set these variables on the **API server**, never as `NEXT_PUBLIC_*` variables:

```dotenv
FRONTEND_URL=https://hyperdx.example.com
EXPRESS_SESSION_SECRET=<random secret shared by every API replica>
KEYCLOAK_ISSUER=https://identity.example.com/realms/hyperdx
KEYCLOAK_CLIENT_ID=hyperdx
KEYCLOAK_CLIENT_SECRET=<client secret from Keycloak>
KEYCLOAK_REQUIRED=false
```

The issuer and client ID enable Keycloak. The secret is required for a
confidential client; omit it only for a public client. Partial configuration
fails startup. HTTPS is required in production; outside production, HTTP is
allowed only on loopback hosts. Local/no-auth app mode cannot enable Keycloak.

The root Docker Compose file forwards the four Keycloak variables from `.env`.
Set `FRONTEND_URL` to the browser-facing origin in your Compose override when
using an HTTPS reverse proxy. For `yarn dev`, use the API environment above
(examples are in `packages/api/.env.development`). No frontend secret or
frontend environment flag is needed.

Create an OpenID Connect client in Keycloak:

- Enable **Standard flow** (Authorization Code). Enable client authentication
for a confidential client. Disable implicit flow and direct access grants.
- Require PKCE **S256**. Make the `email` scope available and include `email`
and `email_verified` in ID tokens. Require email verification for invitees.
- Set the exact valid redirect URI to
`https://hyperdx.example.com/api/login/keycloak/callback`.
- Set the exact valid post-logout redirect URI to
`https://hyperdx.example.com/login`. Avoid wildcards.

Endpoints come from OIDC discovery. The callback is derived from `FRONTEND_URL`
and uses the existing frontend `/api` proxy. Keep the API private behind the
trusted proxy: Express currently trusts one proxy hop. Preserve HTTPS forwarding
headers so the existing Secure, HttpOnly, SameSite=Lax session cookie works.
All replicas need the same MongoDB, session secret, and Keycloak settings.

## Provisioning and migration

1. Bootstrap an empty installation with the existing password registration flow
**before** enabling required mode. Keycloak login never creates a team.
2. Explicitly map existing users as below. An email match never links accounts.
3. Invite new users through HyperDX's existing team invitation UI. The invitee
opens that link and chooses **Sign in with Keycloak**. Their verified email
must match the invitation; the invitation must be less than 30 days old.
Only that invited team is assigned. Existing accounts must be mapped instead.
4. Verify login, access, and logout, then set `KEYCLOAK_REQUIRED=true` on every
API replica. Password login, registration, and password invitation acceptance
return 403. Existing password sessions stop working on protected routes.

For an existing user, an operator with database access must verify the intended
person's exact OIDC issuer and subject independently. Do not infer them from an
email or username. After starting this version (which creates the partial unique
identity index), use `mongosh` against the intended HyperDX database:

```javascript
db.users.updateOne(
{
_id: ObjectId('<verified HyperDX user ID>'),
team: ObjectId('<verified existing HyperDX team ID>'),
keycloak: { $exists: false },
},
{ $set: { keycloak: {
issuer: 'https://identity.example.com/realms/hyperdx',
subject: '<verified OIDC sub>',
} } },
);
```

Require `modifiedCount: 1`. A zero match or duplicate-key error requires
investigation; do not remove the filter or overwrite an existing identity.
The compound unique index prevents one identity from belonging to two users.
Mapping preserves the user's team, password, and API key. Changing email at
Keycloak does not change the identity mapping or HyperDX's stored email.

## Sessions and logout

Login validates state, nonce, PKCE, issuer, audience, signature, and expiry using
[`openid-client`](https://github.com/panva/openid-client). Passport rotates the
session ID. Only issuer, subject, and expiry are stored alongside the existing
Passport user ID; no access, refresh, or ID tokens are persisted.

By default, HyperDX's Keycloak sessions end at ID token expiry, capped at 30
days, and password sessions retain their rolling 30-day behavior. To apply a
24-hour absolute limit and 30-minute idle timeout to both login methods, set on
the API:

```dotenv
SESSION_MAX_AGE_SECONDS=86400
SESSION_IDLE_TIMEOUT_SECONDS=1800
```

Both values must be positive whole seconds, at most 30 days, with idle no
greater than maximum. Enabling the maximum requires existing users to sign in
again. The server enforces the absolute deadline even if the browser replays a
cookie; MongoDB session expiry enforces inactivity. Dashboard polling counts as
activity.

With an explicit maximum, HyperDX creates an application session after
validating the ID token; token expiry does not end that session. Keep Keycloak
tokens short (for example 5 minutes). Set Keycloak's SSO Session Max to 24 hours
and SSO Session Idle to 30 minutes separately; those realm settings affect other
clients too. The two session timers are independent: HyperDX traffic does not
renew Keycloak's idle timer, and an existing SSO session may let users sign back
in without a credential prompt. No refresh tokens are stored.

The logout menu submits a same-origin POST, destroys the local session, then
uses the discovered end-session endpoint. Keycloak may ask for confirmation
because no ID token hint is retained. See
[Keycloak logout](https://www.keycloak.org/securing-apps/oidc-layers#_logout).

**Back-channel logout and token refresh are not implemented.** Logging out at
Keycloak, disabling a realm account, or changing realm roles does not immediately
invalidate an existing HyperDX session. It remains usable until its deadline.
Deleting the HyperDX user or removing its mapping revokes browser access on the
next protected request. Personal API keys retain existing behavior and must be
revoked separately; required mode governs interactive login.

## Team settings access

Operators can set `settingsAdminIds` on the existing MongoDB team document to an
array of that team's HyperDX user ObjectIds. Only those users see the Members,
Data, and Integrations tabs or manage members, invitations, connections,
sources, and webhooks through the internal API, personal API keys, or MCP.
Permission changes take effect on the next request. New invitees do not inherit
access.

An absent field preserves upstream access for all members; an empty array grants
nobody. Verify team membership and keep at least one administrator when setting
the array. Read access needed for searches, dashboards, and alert configuration
is preserved. This is a settings permission, not a ClickHouse query sandbox.

Keep auth query strings, request bodies, and redirect headers out of reverse
proxy/APM logs. Application request logs redact the new auth URLs and never log
protocol error objects. To roll back, disable required mode first; removing all
Keycloak variables also disables its routes and invalidates its browser sessions.

## Regression tests

Use Node.js 22.16 or later and the repository's pinned Yarn version. After
`yarn install` and `yarn build:common-utils`, run the focused unit/UI suites
from the repository root:

```bash
yarn workspace @hyperdx/api ci:unit --runInBand keycloakConfig sessionPolicyConfig logger
yarn workspace @hyperdx/app ci:unit --runInBand KeycloakAuth TeamSettingsAccess
```

The authentication integration suites use real Passport sessions, MongoDB and
the existing `oidc-provider` test fixture. They need no running Keycloak or
ClickHouse server. Start a disposable MongoDB and run from the repository root:

```bash
docker run -d --rm --name hdx-auth-test-mongo \
-p 127.0.0.1:27080:27017 mongo:7.0.45
cd packages/api
NODE_ENV=test DOTENV_CONFIG_PATH=/dev/null \
MONGO_URI=mongodb://127.0.0.1:27080/hdx_auth_test \
yarn jest --config jest.int.config.js --runInBand --forceExit \
keycloak.int.test.ts sessionPolicy.int.test.ts teamSettingsAccess.int.test.ts
docker stop hdx-auth-test-mongo
```

Wait for MongoDB to accept connections before running Jest. Use only a
disposable database: the fixtures clear collections and drop it during cleanup.
The suites cover callback validation and replay, safe identity mapping and
invitation provisioning, session rotation/expiry/logout, password and required
modes, and settings permissions through browser sessions, API keys and MCP. UI
tests cover login choices and restricted tab deep links and permission changes.
These checks do not replace verifying the deployed Keycloak client, HTTPS proxy,
and logout flow in a browser.
5 changes: 5 additions & 0 deletions packages/api/.env.development
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
HYPERDX_LOG_LEVEL=debug
# Development only: keep sessions valid across nodemon restarts.
EXPRESS_SESSION_SECRET="local-development-only"
# Optional Keycloak login; see docs/keycloak.md. Use the public realm issuer.
# KEYCLOAK_ISSUER=http://localhost:8080/realms/hyperdx
# KEYCLOAK_CLIENT_ID=hyperdx
# KEYCLOAK_CLIENT_SECRET=<server-side client secret>
# KEYCLOAK_REQUIRED=false
FRONTEND_URL="http://localhost:${HYPERDX_APP_PORT}"
HDX_NODE_ADVANCED_NETWORK_CAPTURE=1
HDX_NODE_BETA_MODE=1
Expand Down
2 changes: 1 addition & 1 deletion packages/api/jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,5 @@ module.exports = {
'@/(.*)$': '<rootDir>/$1',
},
// Keep in sync with jest.int.config.js.
transformIgnorePatterns: ['/node_modules/(?!(ai|@ai-sdk|@workflow/serde)/)'],
transformIgnorePatterns: ['/node_modules/(?!(ai|@ai-sdk|@workflow/serde|openid-client|oauth4webapi|jose)/)'],
};
2 changes: 1 addition & 1 deletion packages/api/jest.int.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ module.exports = {
moduleFileExtensions: ['js', 'mjs', 'json', 'ts', 'tsx', 'node'],
// Superset of the unit config's list (AI SDK packages) plus the proxy deps.
transformIgnorePatterns: [
'/node_modules/(?!(ai|@ai-sdk|@workflow/serde|http-proxy-middleware|httpxy|is-plain-obj)/)',
'/node_modules/(?!(ai|@ai-sdk|@workflow/serde|http-proxy-middleware|httpxy|is-plain-obj|openid-client|oauth4webapi|jose|oidc-provider)/)',
],
testMatch: ['**/__tests__/**/*.int.test.ts?(x)'],
// Override the unit config's ignore list: it excludes `.int.test.ts`, which
Expand Down
3 changes: 3 additions & 0 deletions packages/api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
"mongoose": "^6.13.10",
"ms": "^2.1.3",
"on-headers": "^1.1.0",
"openid-client": "^6.8.8",
"passport": "^0.6.0",
"passport-local": "^1.0.0",
"pino": "^10.0.0",
Expand All @@ -76,6 +77,7 @@
"@types/lodash": "^4.14.198",
"@types/minimist": "^1.2.2",
"@types/ms": "^0.7.31",
"@types/oidc-provider": "^9.13.0",
"@types/passport-local": "^1.0.34",
"@types/sqlstring": "^2.3.0",
"@types/supertest": "^2.0.12",
Expand All @@ -85,6 +87,7 @@
"jest": "^30.2.0",
"migrate-mongo": "^11.0.0",
"nodemon": "^2.0.20",
"oidc-provider": "^9.12.0",
"pino-pretty": "^13.1.1",
"rimraf": "^4.4.1",
"supertest": "^6.3.1",
Expand Down
73 changes: 73 additions & 0 deletions packages/api/src/__tests__/keycloakConfig.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
describe('Keycloak configuration', () => {
const originalEnv = { ...process.env };

beforeEach(() => {
jest.resetModules();
for (const key of Object.keys(process.env)) {
if (key.startsWith('KEYCLOAK_')) delete process.env[key];
}
process.env.NODE_ENV = 'production';
process.env.FRONTEND_URL = 'https://hyperdx.example.com';
delete process.env.IS_LOCAL_APP_MODE;
});

afterEach(() => {
process.env = { ...originalEnv };
});

const readConfig = () =>
jest.requireActual<typeof import('@/config')>('@/config');

it('leaves Keycloak disabled without configuration', () => {
expect(readConfig().KEYCLOAK).toBeUndefined();
expect(readConfig().KEYCLOAK_REQUIRED).toBe(false);
});

it('derives the callback from the public frontend URL', () => {
process.env.KEYCLOAK_ISSUER = 'https://identity.example.com/realms/hyperdx';
process.env.KEYCLOAK_CLIENT_ID = 'hyperdx';
expect(readConfig().KEYCLOAK).toMatchObject({
issuer: 'https://identity.example.com/realms/hyperdx',
clientId: 'hyperdx',
callbackUrl: 'https://hyperdx.example.com/api/login/keycloak/callback',
});
});

it.each([
{ KEYCLOAK_REQUIRED: 'true' },
{ KEYCLOAK_REQUIRED: 'tru' },
{ KEYCLOAK_ISSUER: 'https://id.example.com/realms/test' },
{ KEYCLOAK_CLIENT_SECRET: 'secret-sentinel' },
{
KEYCLOAK_ISSUER: 'http://id.example.com/realms/test',
KEYCLOAK_CLIENT_ID: 'hdx',
},
{
KEYCLOAK_ISSUER: 'https://id.example.com/realms/test?x=1',
KEYCLOAK_CLIENT_ID: 'hdx',
},
{
KEYCLOAK_ISSUER: 'https://id.example.com/realms/test',
KEYCLOAK_CLIENT_ID: 'hdx',
FRONTEND_URL: 'http://hdx.example.com',
},
{
KEYCLOAK_ISSUER: 'https://id.example.com/realms/test',
KEYCLOAK_CLIENT_ID: 'hdx',
FRONTEND_URL: '//hdx.example.com',
},
{
KEYCLOAK_ISSUER: 'https://id.example.com/realms/test',
KEYCLOAK_CLIENT_ID: 'hdx',
IS_LOCAL_APP_MODE: 'DANGEROUSLY_is_local_app_mode💀',
},
])('rejects incomplete or unsafe settings (%j)', settings => {
Object.assign(process.env, settings);
expect(readConfig).toThrow(/Keycloak/);
try {
readConfig();
} catch (error) {
expect(String(error)).not.toContain('secret-sentinel');
}
});
});
Loading
Loading