feat(athena): Part 4 — vendor the dbt-athena 1.11.0 macro package

[aoelvp94]

Part of #16252. Related: #13822. Independent PR; the sequence is at the bottom.

Problem

Every supported adapter has its Jinja package under crates/dbt-loader/src/dbt_macro_assets/. Athena has none, so parse/ls stop at "missing dbt-athena package" once the profile loads.

Solution

Vendors dbt/include/athena/ from the dbt-athena 1.11.0 wheel as dbt-athena/ (41 files, 2,212 SQL lines; dbt-redshift is 38 files / 1,754 for comparison). Sourcing note in the README: the macros ship in the dbt-athena distribution, not dbt-athena-community, which is an empty shim depending on it.

No loader code change: rust-embed picks the folder up and internal_package_names derives dbt-{adapter_type}. Athena inherits no other adapter's macros, so it needs no entry in the match that gives Redshift dbt-postgres.

Six macros differ from upstream dbt-athena, and one file is left out. Each is marked with a comment at the site:

• athena__list_schemas and athena__get_columns_in_relation delegated to adapter.list_schemas / adapter.get_columns_in_relation. In dbt-core those are Python methods; in Fusion AdapterImpl::list_schemas runs the Jinja macro of the same name, so the delegation recursed until the Jinja stack overflowed. Both are now SQL over information_schema, which is what the Python did.

• athena__create_schema called render_hive() on the relation. Athena CREATE SCHEMA is Hive DDL, so it now renders the backticked bare name directly.

• athena__check_schema_exists added: the default implementation queries a column Athena's information_schema.schemata does not have.

• athena__get_catalog and athena__get_catalog_relations delegated to adapter.get_catalog / adapter.get_catalog_by_relations, Python methods that read the Glue catalog through boto3. Under Fusion compile --write-catalog (and so docs generate) reported "unknown method: get_catalog_by_relations" per schema and produced an empty catalog.json. Both now query information_schema.tables joined to information_schema.columns, returning the Postgres-shaped rows the metadata adapter already parses. Known limit: a schema-wide information_schema.columns scan fails with ICEBERG_MISSING_METADATA if any table in the schema has unreadable Iceberg metadata, which loses that schema's catalog for the run; the Glue GetTables API does not have that problem and a Glue-backed catalog is the faithful follow-up.

• resolve_table_type copied catalog_relation.table_format verbatim. Fusion's TableFormat renders default where dbt-athena expects hive, so an unconfigured model resolved to 'default' and table.sql took the else (Iceberg) branch of {% if table_type == 'hive' %}. It now maps the format instead of copying it.

sample_profiles.yml is not vendored: is_metadata_file in load_packages.rs accepts only dbt_project.yml, packages.yml, profile_template.yml and __init__.py in an internal package, and construct_internal_packages asserts everything else lives under macros/ or tests/, so a debug build tripped during dbt parse while the file was present. The profile shape it documented is what the Part 9 wizard generates.

Everything else is byte-identical to 1.11.0, including python_submissions.sql, which Fusion cannot use but which keeps the diff against upstream dbt-athena trivial to re-vendor.

Permission requested: .agents/adapters.md says vendored macros are not to be touched without explicit approval. The six edits above are the minimum for the package to load, for docs to work, and for an unconfigured model to land as Hive under Fusion. If you would rather keep the package byte-identical, the same six can be implemented as Rust arms in AdapterImpl and the macros left as shipped. Say which.

Verification

• With Parts 1, 2, 3: dbt parse on a 750-model project using dbt_utils, elementary and a root-project override of iceberg_merge: 0 errors, 1 deprecation warning unrelated to Athena. Every macro in the package is parsed; no Jinja construct is rejected. Root-project macro overrides still shadow package macros as in dbt-core.
• With the later execution parts as well: view, table and incremental materializations run unmodified end to end, and compile --write-catalog produces a populated catalog.json.

Feedback wanted: permission for the six macro edits, or the Rust-arm alternative.

Sequence (independent PRs, each compiles alone against main)

• Part 0: feat(athena): Part 0 — map Backend::Athena to its driver entrypoint and send the driver's output-location option #16365 — ADBC driver wiring
• Part 1: feat(athena): AthenaDbConfig profile schema #16274 — AthenaDbConfig profile schema (@fpiped)
• Part 2: feat(athena): Part 2 — fill the parse-path todo!() arms and the config alias #16367 — dbt-adapter core wiring
• Part 3: feat(athena): Part 3 — accept partitioned_by, table_properties and force_batch as model configs #16368 — model config keys
• Part 4: this PR — dbt-athena macro package
• Part 5: feat(athena): Part 5 — AthenaMetadataAdapter and athena_get_relation #16374 — AthenaMetadataAdapter and get_relation
• Parts 6+7: branch ready, not yet opened — Glue / Lake Formation housekeeping shims and the table / incremental helpers (athena/parts-6-7-execution); see also feat(athena): Part 6 — AthenaAdapter methods behind the dbt-athena macros #16376
• Part 8: feat(athena): Part 8 — run dbt unit tests on Athena #16370 — dbt unit tests on Athena
• Part 9: feat(athena): Part 9 — dbt init profile wizard for Athena #16375 — dbt init profile wizard
• Full branch for context: https://github.com/aoelvp94/dbt/tree/spike/athena-profile-parse

Checklist

• I have read the contributing guide and understand what's expected of me.
• I have run this code in development, and it appears to resolve the stated issue.
• This PR includes tests, or tests are not required or relevant for this PR. (loader macro tests for the modified macros are a pending follow-up on this branch)
• This PR has no interface changes. (adds the dbt-athena macro namespace)

[fpiped]

Tested with the rest of the stack (and Part 6) against a real workgroup. Two things:

1. crates/dbt-loader/src/dbt_macro_assets/dbt-athena/sample_profiles.yml: the loader accepts only dbt_project.yml, packages.yml, profile_template.yml and __init__.py in an internal package (load_packages.rs, is_metadata_file), so dbt parse panics in debug builds while this file is present. The sample profile is covered by the Part 9 wizard.
2. resolve_table_type copies catalog_relation.table_format verbatim. Fusion's TableFormat renders default, not hive, so an unconfigured model resolves to 'default' and table.sql takes the else (Iceberg) branch of {% if table_type == 'hive' %}. {%- set table_type = 'iceberg' if catalog_relation.table_format == 'iceberg' else 'hive' -%} fixes it.

[fpiped]

Match objects can't be indexed in dbt v2 (#16451), so these render empty and bucketed Iceberg tables fail. .group() works today:

Suggested change

	{%- set ns.bucket_column = bucket_match[1] -%}
	{%- set bucket_num = adapter.murmur3_hash(col, bucket_match[2] | int) -%}
	{%- set ns.bucket_column = bucket_match.group(1) -%}
	{%- set bucket_num = adapter.murmur3_hash(col, bucket_match.group(2) | int) -%}

[fpiped]

athena__create_schema skips adapter.add_lf_tags_to_database(relation), which dbt-athena calls after the DDL. #16376 now implements it, so the call can come back:

  {%- call statement('create_schema') -%}
    create schema if not exists `{{ relation.schema }}`
  {% endcall %}
  {{ adapter.add_lf_tags_to_database(relation) }}

Checked on a real work group: the profile's lf_tags_database lands on the schema the run creates.

[fpiped]

Fixes from the end-to-end runs, as a PR to this branch: aoelvp94#1 (S3 Tables macros from dbt-adapters 62a5495cf, bucket_match.group(), the add_lf_tags_to_database call, catalog-qualified information_schema.columns).