StreamsHub AuthZ

[tomncooper]

Currently, each of the main StreamsHub components has a different way of configuring and managing authorization (AuthZ). We should investigate how we might provide a more unified authorization approach.

We should summarise the current approach to AuthZ in our stack and identify pain points. We should then research what options are available for removing those pain points and making a more unified (and hopefully easier to manage) user experience.

The outcome should be a proposal covering the recommended approach.

[tomncooper]

I have done a bit more thinking, prototyping and consulting with component teams on the unified authorization problem. Below is a summary of where my current thinking is:

The Problem

The fundamental problem we need to solve is that Kafka, Kroxylicious, Apicurio Registry and StreamsHub Console each have their own way of expressing and evaluating access control decisions, which means users have to learn and configure multiple different systems to achieve a common security setup across the stack. I think we should converge on a single authorization API and a shared policy decision point (PDP), so that the same policy language and configuration format works everywhere.

The Kroxylicious Authorizer API as the Common Foundation

The Kroxylicious project already defines a general-purpose authorizer API that was deliberately designed to be reusable beyond Kroxylicious itself. It provides a small set of types that model authorization decisions in a way that is independent of any particular domain (e.g. Kafka, Apicurio, etc): a ResourceType interface that each consumer implements as an enum of operations per resource kind, an Action that pairs an operation with a named resource, a Subject representing the authenticated caller, and an Authorizer interface whose authorize() method accepts a subject and a list of actions and returns a partitioned result of what was allowed and what was denied.

The power of this design is that each domain defines its own resource types and operations while sharing the same evaluation machinery. Each resource kind is modelled as a separate enum that implements ResourceType, where the enum values are the operations available on that kind of resource. For example, Kroxylicious already defines a TopicResource enum with operations like READ, WRITE, CREATE, and DELETE, so the action "can this subject read the topic orders.eu?" is expressed as new Action(TopicResource.READ, "orders.eu"). Apicurio Registry would define its own enums in exactly the same way, perhaps an Artifact enum with operations like READ, WRITE, and ADMIN, so the action "can this subject write the artifact payments-schema?" becomes new Action(Artifact.WRITE, "payments-schema"). Both domains express their authorization questions through the same Authorizer.authorize() interface, and the same PDP evaluates them using the same configuration, even though the resource types and operations are completely different. In authorization terminology, the Authorizer interface is the contract between a policy enforcement point (PEP) and a PDP: each component's PEP constructs the authorization request using its own resource types, and the shared PDP evaluates it against the configured policy.

This API is currently tied to the broader kroxylicious-api module, which brings in Kafka client libraries, Jackson, and compression codecs as transitive dependencies. That makes it impractical for non-Kroxylicious projects to depend on. I have opened an upstream proposal (kroxylicious/design#119) to extract the identity types (Principal, Subject) into a new lightweight module with zero runtime dependencies, and to re-point the authorizer API at that module instead. Once that work lands, the authorizer API becomes genuinely standalone and suitable for adoption by Apicurio Registry and StreamsHub Console without dragging in unwanted dependencies.

I think we should standardise on this API as the common authorization interface across all StreamsHub components. Components that already have their own authorization layers could be refactored to implement or delegate to the Kroxylicious Authorizer interface, so that the PDP and configuration can be shared.

A Shared Authorizer Implementation (Policy Decision Point)

Once we have a shared authorization API, we could build a shared Authorizer implementation to act as the common PDP for our components. This would give us a single configuration format across all of them, with shared tooling and processes.

I think a grants-based approach would work well here, where access is expressed as a set of allow and deny rules that match on who is making the request, what operation they're performing, and which resource they're targeting. The configuration would be loaded from pluggable sources (a JSON file to start with, potentially Keycloak's authorization services or other backends later), so we can evolve how policy is managed without changing how it is evaluated.

The Challenge of Resource Enumeration

One of the interesting complications is that our components interact with resources in fundamentally different ways, and this affects what the shared authorizer needs to provide beyond a simple "is this allowed?" check.

Kroxylicious sees resource names as they flow through Kafka protocol messages. It never needs to ask "what resources exist?", it just needs to decide whether a given action on a named resource should be permitted. A straightforward authorize() call is all it needs.

Apicurio Registry and StreamsHub Console are different. They own resource enumeration: when a user asks for a list of artifacts or topics, these components actively query a backend (a SQL database for Apicurio, Kafka's AdminClient for Console) to discover what exists. In a deployment with tens of thousands of artifacts or topics, we can't just fetch everything and then filter by authorization after the fact. The authorizer needs to provide a way for these components to push authorization constraints into their query layer, so that the database query or API call only returns resources the user is actually allowed to see. This means the shared authorizer would need to support a scope resolution capability alongside the basic authorize check, giving components structured patterns (like name prefixes or exact name sets) that they can translate into query predicates or use for efficient client-side filtering.

Identity as a Shared Vocabulary

Each component also has a different runtime identity context. Kroxylicious works at the protocol level, Apicurio and Console sit behind Quarkus security, and different deployments might use different identity providers entirely. The shared authorizer needs a common set of principal types (user, role, group) that mean the same thing regardless of which component is evaluating a grant or which identity provider authenticated the user. This acts as the policy information point (PIP): each component maps its own identity context into these shared types, so the authorization policy doesn't need to know whether the "admin" role came from a Keycloak token or a different OIDC provider.

What This Would Mean for StreamsHub Users

From a user's perspective, the end result would be a single policy administration point (initially a configuration file, potentially Keycloak or another source later) that expresses authorization policy in a consistent language across all StreamsHub components. Instead of configuring Kafka ACLs in one format, Apicurio role mappings in another, and Console RBAC in a third, a user would write grants using the same principal types, resource matchers, and operation names regardless of which component enforces them. Each component's resource types would remain distinct (Kafka topics are not the same as registry artifacts), but the policy language, the PDP that evaluates it, and the identity vocabulary that describes subjects would all be shared.

Next Steps

I plan to develop this idea more into a prototype and proposal once kroxylicious/design#119 is merged and implemented.