Skip to content

Add the Multi-Tenant Catalogs Endpoint Extension, for nested catalog support - #366

Merged
vincentsarago merged 81 commits into
stac-utils:mainfrom
jonhealy1:stac-fastapi-catalogs-extension
Jun 9, 2026
Merged

Add the Multi-Tenant Catalogs Endpoint Extension, for nested catalog support#366
vincentsarago merged 81 commits into
stac-utils:mainfrom
jonhealy1:stac-fastapi-catalogs-extension

Conversation

@jonhealy1

@jonhealy1 jonhealy1 commented Mar 25, 2026

Copy link
Copy Markdown
Collaborator

Related Issue(s):

Description:

Extension spec: https://github.com/StacLabs/multi-tenant-catalogs
STAC-FastAPI catalogs extension: https://github.com/StacLabs/stac-fastapi-catalogs-extension

PR Checklist:

  • pre-commit hooks pass locally
  • Tests pass (run make test)
  • Documentation has been updated to reflect changes, if applicable, and docs build successfully (run make docs)
  • Changes are added to the CHANGELOG.

@jonhealy1

Copy link
Copy Markdown
Collaborator Author

It's going to be best to fix the extension so it supports python 3.11

@jonhealy1 jonhealy1 changed the title route extension, create, get catalogs Add the Multi-Tenant Virtual Catalogs Extension, for nested catalog support Mar 25, 2026
@jonhealy1 jonhealy1 changed the title Add the Multi-Tenant Virtual Catalogs Extension, for nested catalog support Add the Multi-Tenant Catalogs Endpoint Extension, for nested catalog support Mar 25, 2026
Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_database_logic.py Outdated
@jonhealy1

jonhealy1 commented Apr 21, 2026

Copy link
Copy Markdown
Collaborator Author

This is really close to being reviewable, just need some time to do some qa, documentation.

@hrodmn hrodmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am partway through reviewing but I found a possible bug in the catalog transactions endpoints for linking two existing catalogs.

Assuming the local Docker network/API is running at http://localhost:8082:

APP_HOST="http://localhost:8082"
SUFFIX="$(date +%Y%m%d%H%M%S)"
PARENT_1="catalog-link-repro-$SUFFIX-parent-1"
PARENT_2="catalog-link-repro-$SUFFIX-parent-2"
CHILD="catalog-link-repro-$SUFFIX-child"

# create a new parent catalog
curl --fail-with-body --show-error --silent \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{
    "id": "'"$PARENT_1"'",
    "type": "Catalog",
    "title": "Parent 1",
    "description": "First parent catalog",
    "stac_version": "1.0.0",
    "links": []
  }' \
  "$APP_HOST/catalogs"

# create another new parent catalog
curl --fail-with-body --show-error --silent \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{
    "id": "'"$PARENT_2"'",
    "type": "Catalog",
    "title": "Parent 2",
    "description": "Second parent catalog",
    "stac_version": "1.0.0",
    "links": []
  }' \
  "$APP_HOST/catalogs"

# Create the child under the first parent. This succeeds.
curl --fail-with-body --show-error --silent \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{
    "id": "'"$CHILD"'",
    "type": "Catalog",
    "title": "Shared child",
    "description": "Child catalog to link under two parents",
    "stac_version": "1.0.0",
    "links": []
  }' \
  "$APP_HOST/catalogs/$PARENT_1/catalogs"

# Link the existing child under the second parent. This should succeed, but currently returns 409.
curl --fail-with-body --show-error --silent \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{"id": "'"$CHILD"'"}' \
  "$APP_HOST/catalogs/$PARENT_2/catalogs"

Observed output for the final request:

POST http://localhost:8082/catalogs/<parent-2>/catalogs
curl: (22) The requested URL returned error: 409
{"code":"ConflictError","description":""}

Unit test reproduction

A failing test for this behavior can be added to tests/extensions/test_catalogs.py:

@pytest.mark.asyncio
async def test_link_existing_sub_catalog_to_second_parent(app_client):
    """Test linking an existing catalog as a sub-catalog of another parent."""
    await create_catalog(
        app_client,
        "catalog-poly-parent-1",
        description="First parent catalog for sub-catalog poly-hierarchy",
    )
    await create_catalog(
        app_client,
        "catalog-poly-parent-2",
        description="Second parent catalog for sub-catalog poly-hierarchy",
    )

    await create_sub_catalog(
        app_client,
        "catalog-poly-parent-1",
        "catalog-poly-child",
        description="Sub-catalog linked to multiple parents",
    )

    resp = await app_client.post(
        "/catalogs/catalog-poly-parent-2/catalogs", json={"id": "catalog-poly-child"}
    )
    assert resp.status_code == 200, resp.text

Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_client.py Outdated
Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_client.py Outdated
Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_database_logic.py Outdated
Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_database_logic.py Outdated
Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_database_logic.py Outdated
Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_client.py
Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_client.py Outdated
@jonhealy1

Copy link
Copy Markdown
Collaborator Author

I am partway through reviewing but I found a possible bug in the catalog transactions endpoints for linking two existing catalogs.

Assuming the local Docker network/API is running at http://localhost:8082:

APP_HOST="http://localhost:8082"
SUFFIX="$(date +%Y%m%d%H%M%S)"
PARENT_1="catalog-link-repro-$SUFFIX-parent-1"
PARENT_2="catalog-link-repro-$SUFFIX-parent-2"
CHILD="catalog-link-repro-$SUFFIX-child"

# create a new parent catalog
curl --fail-with-body --show-error --silent \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{
    "id": "'"$PARENT_1"'",
    "type": "Catalog",
    "title": "Parent 1",
    "description": "First parent catalog",
    "stac_version": "1.0.0",
    "links": []
  }' \
  "$APP_HOST/catalogs"

# create another new parent catalog
curl --fail-with-body --show-error --silent \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{
    "id": "'"$PARENT_2"'",
    "type": "Catalog",
    "title": "Parent 2",
    "description": "Second parent catalog",
    "stac_version": "1.0.0",
    "links": []
  }' \
  "$APP_HOST/catalogs"

# Create the child under the first parent. This succeeds.
curl --fail-with-body --show-error --silent \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{
    "id": "'"$CHILD"'",
    "type": "Catalog",
    "title": "Shared child",
    "description": "Child catalog to link under two parents",
    "stac_version": "1.0.0",
    "links": []
  }' \
  "$APP_HOST/catalogs/$PARENT_1/catalogs"

# Link the existing child under the second parent. This should succeed, but currently returns 409.
curl --fail-with-body --show-error --silent \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{"id": "'"$CHILD"'"}' \
  "$APP_HOST/catalogs/$PARENT_2/catalogs"

Observed output for the final request:

POST http://localhost:8082/catalogs/<parent-2>/catalogs
curl: (22) The requested URL returned error: 409
{"code":"ConflictError","description":""}

Unit test reproduction

A failing test for this behavior can be added to tests/extensions/test_catalogs.py:

@pytest.mark.asyncio
async def test_link_existing_sub_catalog_to_second_parent(app_client):
    """Test linking an existing catalog as a sub-catalog of another parent."""
    await create_catalog(
        app_client,
        "catalog-poly-parent-1",
        description="First parent catalog for sub-catalog poly-hierarchy",
    )
    await create_catalog(
        app_client,
        "catalog-poly-parent-2",
        description="Second parent catalog for sub-catalog poly-hierarchy",
    )

    await create_sub_catalog(
        app_client,
        "catalog-poly-parent-1",
        "catalog-poly-child",
        description="Sub-catalog linked to multiple parents",
    )

    resp = await app_client.post(
        "/catalogs/catalog-poly-parent-2/catalogs", json={"id": "catalog-poly-child"}
    )
    assert resp.status_code == 200, resp.text

@hrodmn Good catch! This should be fixed now

@jonhealy1
jonhealy1 requested review from hrodmn and vincentsarago June 4, 2026 06:10

@hrodmn hrodmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonhealy1 thanks for fixing the catalog linking bug. I reviewed the rest of the code today and my only remaining change request is to check that the parent catalog exists when adding a collection. There is similar logic in create_sub_catalog that should be copy-pastable. Please add a test for that case too!

Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_client.py
@jonhealy1

Copy link
Copy Markdown
Collaborator Author

@hrodmn Thanks for reviewing!

@jonhealy1
jonhealy1 requested a review from hrodmn June 5, 2026 03:32

@hrodmn hrodmn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this contribution @jonhealy1! I think many will find this feature useful.

@bkanuka

bkanuka commented Jun 8, 2026

Copy link
Copy Markdown

Congrats @jonhealy1 !! Lots of solid dev work as well as balancing requirements and expectations of everyone involved. Glad I could see the journey!

@bkanuka

bkanuka commented Jun 8, 2026

Copy link
Copy Markdown

ha I think I just caught another bug 😆 there's some double slashes in generated links. They likely need a rstrip on the base_url. I'll leave a comment

Comment thread stac_fastapi/pgstac/extensions/catalogs/catalogs_client.py Outdated
@vincentsarago

Copy link
Copy Markdown
Member

thanks everyone 🙏

@vincentsarago
vincentsarago merged commit ac432cf into stac-utils:main Jun 9, 2026
6 checks passed
@jonhealy1

jonhealy1 commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for all the help! @hrodmn @bkanuka @bitner @vincentsarago :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants