Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
# dbt_fivetran_utils v0.4.12

## Bug Fix
- Adds DuckDB support to the `percentile` macro by dispatching to DuckDB's aggregate `percentile_cont(...) within group (...)` form.

# dbt_fivetran_utils v0.4.11

[PR #154](https://github.com/fivetran/dbt_fivetran_utils/pull/154) includes the following updates:
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,18 +239,18 @@ This macro allows for cross database use of obtaining the max boolean value of a

----
### percentile ([source](macros/percentile.sql))
This macro is used to return the designated percentile of a field with cross db functionality. The percentile function stems from percentile_cont across db's. For Snowflake and Redshift this macro uses the window function opposed to the aggregate for percentile. For Postgres, this macro uses the aggregate, as it does not support a percentile window function. Thus, you will need to add a target-dependent `group by` in the query you are calling this macro in.
This macro is used to return the designated percentile of a field with cross db functionality. The percentile function stems from percentile_cont across db's. For Snowflake and Redshift this macro uses the window function opposed to the aggregate for percentile. For Postgres and DuckDB, this macro uses the aggregate, as they do not support a percentile window function. Thus, you will need to add a target-dependent `group by` in the query you are calling this macro in.

**Usage:**
```sql
select id,
{{ fivetran_utils.percentile(percentile_field='time_to_close', partition_field='id', percent='0.5') }}
from your_cte
{% if target.type == 'postgres' %} group by id {% endif %}
{% if target.type in ('postgres', 'duckdb') %} group by id {% endif %}
```
**Args:**
* `percentile_field` (required): Name of the field of which you are determining the desired percentile.
* `partition_field` (required): Name of the field you want to partition by to determine the designated percentile. You will need to group by this for Postgres.
* `partition_field` (required): Name of the field you want to partition by to determine the designated percentile. You will need to group by this for Postgres and DuckDB.
* `percent` (required): The percent necessary for `percentile_cont` to determine the percentile. If you want to find the median, you will input `0.5` for the percent.

----
Expand Down
2 changes: 1 addition & 1 deletion dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: 'fivetran_utils'
version: '0.4.11'
version: '0.4.12'
config-version: 2
require-dbt-version: [">=1.3.0", "<3.0.0"]
4 changes: 2 additions & 2 deletions integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 'fivetran_utils_integration_tests'
version: '0.4.11'
version: '0.4.12'
config-version: 2
profile: 'integration_tests'

Expand All @@ -15,4 +15,4 @@ dispatch:

flags:
send_anonymous_usage_stats: False
use_colors: True
use_colors: True
11 changes: 10 additions & 1 deletion macros/percentile.sql
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,15 @@

{% endmacro %}

{% macro duckdb__percentile(percentile_field, partition_field, percent) %}

percentile_cont(
{{ percent }} )
within group ( order by {{ percentile_field }} )
/* have to group by partition field */

{% endmacro %}

{% macro spark__percentile(percentile_field, partition_field, percent) %}

percentile(
Expand All @@ -52,4 +61,4 @@
over (partition by {{ partition_field }}
)

{% endmacro %}
{% endmacro %}