Integration

[lukaszlacinski]

PR #48 — Integration Summary

Repo: ESGF/stac-transaction-api
Branch: integration → main
Author: @lukaszlacinski
Status: Open — awaiting review from @rhysrevans3, @sturoscy-personal
Commits: 13

---

Overview

This PR integrates esgf-core-utils into the Globus authorization layer,
introduces a structured Pydantic auth model, adds access control policy
caching, and fixes several issues in the PATCH request handling path.

---

New File: src/authorizer/globus_auth.py

Introduces a clean Pydantic-based authorization model hierarchy to replace
ad-hoc dict handling of Globus entitlements.

Models:

Class	Purpose
Node	Holds a node ID and its set of permitted roles
Project	Holds a project ID and its set of permitted roles
Nodes	Collection of Node objects; handles node-level asset authorization
Projects	Collection of Project objects; handles project-level authorization
GlobusAuth	Top-level auth context combining requester data, nodes, and projects

Roles (Literal type): CREATE, UPDATE, DELETE, REPLICATE, REVOKE

Key behavior:

• GlobusAuth.add(entitlements) parses raw Globus entitlement strings via
  regex and populates per-project and per-node role sets.
• GlobusAuth.authorize(collection_id, item, role, ...) checks both project
  and node permissions, raising AuthorizationException (from
  esgf-core-utils) on failure.
• Nodes.authorize() recursively checks asset HREFs including alternate
  entries.
• Role merging: if a project or node already exists, new roles are unioned
  into the existing set rather than replaced.

Exceptions sourced from esgf_core_utils.models.exceptions:

• MissingPermissionException — raised by Projects / Nodes on missing
  permission
• AuthorizationException — raised by GlobusAuth, wrapping the above with
  request/event context

---

Modified: src/authorizer/globus_authorizer.py

Access Control Policy Caching

Added a TTL-based in-memory cache for the access control policy (the list of
entitlement strings mapping Globus groups to roles):

• _CachedPolicy dataclass holds the policy list and an expiry timestamp.
• _policy_lock (threading Lock) guards cache reads and writes.
• get_access_control_policy() returns the cached policy if still valid;
  otherwise reloads from settings.client.policy_path (supports both
  file:// URIs and HTTP URLs via urllib3).
• Stale-on-failure fallback: if a reload fails and a prior cached policy
  exists, it is returned with a warning log rather than raising.

_authorizer_context() helper

New function that builds a GlobusAuth object from token introspection data
and cached policy entitlements:

1. Extracts the user's Globus group IDs from the auth["groups"] list.
2. Filters the cached policy to entitlements whose group ID matches the user's
   memberships.
3. Constructs and returns a GlobusAuth with those entitlements applied.

Middleware dispatch changes

Both the cache-hit and cache-miss paths in GlobusAuthorizerMiddleware.dispatch()
now call _authorizer_context(auth) and assign the resulting GlobusAuth
object to request.state.authorizer, replacing the previous behavior of
assigning the raw auth dict directly.

---

Modified: src/client.py

• PATCH handler now extracts institute_id from item_id (parsed from the
  URL path) rather than from the request body, making it robust when the body
  omits that field.
• Removed assertion that collection_id matches item.collection, relaxing
  a validation constraint that was failing legitimate cross-collection
  references.
• Fixed validate_patch() function signature.
• Fixed patch operation serialization (Dump patch operations correctly).
• Validation error causes are now included in HTTP error responses.

---

Modified: src/settings/

File	Change
globus.py	Imports projects/nodes from esgf-core-utils instead of esgf-playground-utils
example.env	Updated to reflect new env var names / policy path settings
access_control_policy.json / .txt	Updated institution list for CORDEX-CMIP6

---

Modified: src/utils.py

Removed unused module imports (cleanup).

---

Notable Migration

This PR completes the shift from esgf-playground-utils to esgf-core-utils
for project and node definitions used in Globus authorization — consistent
with the refactor tracked in earlier work on stac-transaction-api.

[rhysrevans3]

This looks really good @lukaszlacinski I've made a new release of esgf-core-utils:1.1.0 with your changes to unify the authorizers. Happy for this to be merged once those changes have been made 👍

[sturoscy-personal]

Fixing merge conflicts.