diff --git a/CHANGELOG.md b/CHANGELOG.md index 7019334..7a9603c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,8 @@ +# dbt_fivetran_utils v0.4.12 + +## Bug Fix +- Adds DuckDB support to the `percentile` macro by dispatching to DuckDB's aggregate `percentile_cont(...) within group (...)` form. + # dbt_fivetran_utils v0.4.11 [PR #154](https://github.com/fivetran/dbt_fivetran_utils/pull/154) includes the following updates: diff --git a/README.md b/README.md index e7a280c..8486904 100644 --- a/README.md +++ b/README.md @@ -239,18 +239,18 @@ This macro allows for cross database use of obtaining the max boolean value of a ---- ### percentile ([source](macros/percentile.sql)) -This macro is used to return the designated percentile of a field with cross db functionality. The percentile function stems from percentile_cont across db's. For Snowflake and Redshift this macro uses the window function opposed to the aggregate for percentile. For Postgres, this macro uses the aggregate, as it does not support a percentile window function. Thus, you will need to add a target-dependent `group by` in the query you are calling this macro in. +This macro is used to return the designated percentile of a field with cross db functionality. The percentile function stems from percentile_cont across db's. For Snowflake and Redshift this macro uses the window function opposed to the aggregate for percentile. For Postgres and DuckDB, this macro uses the aggregate, as they do not support a percentile window function. Thus, you will need to add a target-dependent `group by` in the query you are calling this macro in. **Usage:** ```sql select id, {{ fivetran_utils.percentile(percentile_field='time_to_close', partition_field='id', percent='0.5') }} from your_cte -{% if target.type == 'postgres' %} group by id {% endif %} +{% if target.type in ('postgres', 'duckdb') %} group by id {% endif %} ``` **Args:** * `percentile_field` (required): Name of the field of which you are determining the desired percentile. -* `partition_field` (required): Name of the field you want to partition by to determine the designated percentile. You will need to group by this for Postgres. +* `partition_field` (required): Name of the field you want to partition by to determine the designated percentile. You will need to group by this for Postgres and DuckDB. * `percent` (required): The percent necessary for `percentile_cont` to determine the percentile. If you want to find the median, you will input `0.5` for the percent. ---- diff --git a/dbt_project.yml b/dbt_project.yml index d4470d5..ef467a2 100644 --- a/dbt_project.yml +++ b/dbt_project.yml @@ -1,4 +1,4 @@ name: 'fivetran_utils' -version: '0.4.11' +version: '0.4.12' config-version: 2 require-dbt-version: [">=1.3.0", "<3.0.0"] diff --git a/integration_tests/dbt_project.yml b/integration_tests/dbt_project.yml index 5e0ee49..fcb8b75 100644 --- a/integration_tests/dbt_project.yml +++ b/integration_tests/dbt_project.yml @@ -1,5 +1,5 @@ name: 'fivetran_utils_integration_tests' -version: '0.4.11' +version: '0.4.12' config-version: 2 profile: 'integration_tests' @@ -15,4 +15,4 @@ dispatch: flags: send_anonymous_usage_stats: False - use_colors: True \ No newline at end of file + use_colors: True diff --git a/macros/percentile.sql b/macros/percentile.sql index 5bb29a9..a2febaa 100644 --- a/macros/percentile.sql +++ b/macros/percentile.sql @@ -44,6 +44,15 @@ {% endmacro %} +{% macro duckdb__percentile(percentile_field, partition_field, percent) %} + + percentile_cont( + {{ percent }} ) + within group ( order by {{ percentile_field }} ) + /* have to group by partition field */ + +{% endmacro %} + {% macro spark__percentile(percentile_field, partition_field, percent) %} percentile( @@ -52,4 +61,4 @@ over (partition by {{ partition_field }} ) -{% endmacro %} \ No newline at end of file +{% endmacro %}