Skip to content

Commit d53214e

Browse files
lennartkats-dbjuliacrawf-db
lennartkats-db
and
juliacrawf-db
authored
Knowledge base: metric views (#156)
Adds metric view examples to knowledge base: one based on a SQL job, one based on databricks-dbt. --------- Co-authored-by: Julia Crawford (Databricks) <julia.crawford@databricks.com>
1 parent 6e03d62 commit d53214e

11 files changed

Lines changed: 469 additions & 0 deletions

File tree

‎knowledge_base/metric_view/README.md‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Unity Catalog Metric View
2+
3+
This project demonstrates how to create a [Unity Catalog Metric View](https://docs.databricks.com/metric-views/) using Declarative Automation Bundles. Once registered, the metric view becomes available to analysts and BI tools across your workspace, queryable via the `MEASURE()` SQL function.
4+
5+
## `bookings_kpis` metric view
6+
7+
This project defines `bookings_kpis`, a metric view over the public sample dataset `samples.wanderbricks.bookings`.
8+
9+
A SQL task in the job runs `CREATE OR REPLACE VIEW … WITH METRICS LANGUAGE YAML` from [`src/bookings_kpis.metric_view.sql`](src/bookings_kpis.metric_view.sql):
10+
11+
```sql
12+
CREATE OR REPLACE VIEW bookings_kpis
13+
WITH METRICS
14+
LANGUAGE YAML
15+
AS $$
16+
version: 1.0
17+
source: samples.wanderbricks.bookings
18+
filter: status = 'confirmed'
19+
dimensions:
20+
- name: check_in_month
21+
expr: date_trunc('MONTH', check_in)
22+
measures:
23+
- name: total_bookings
24+
expr: COUNT(1)
25+
- name: total_revenue
26+
expr: SUM(total_amount)
27+
$$;
28+
```
29+
30+
`{{catalog}}` and `{{schema}}` in the SQL file are substituted from job parameters at run time.
31+
32+
Once registered, query the metric view from any SQL editor:
33+
34+
```sql
35+
SELECT
36+
check_in_month,
37+
MEASURE(total_bookings) AS bookings,
38+
MEASURE(total_revenue) AS revenue
39+
FROM <catalog>.<your_schema>.bookings_kpis
40+
GROUP BY check_in_month
41+
ORDER BY check_in_month;
42+
```
43+
44+
## Get started
45+
46+
### Prerequisites
47+
48+
* Databricks workspace with Unity Catalog enabled
49+
* A SQL warehouse on a runtime that supports Unity Catalog metric views
50+
* Databricks CLI installed and configured
51+
52+
### Setup
53+
54+
1. In `databricks.yml`, replace `<your-warehouse-id>` (in both `targets.dev` and `targets.prod`) with one of your warehouse IDs (`databricks warehouses list`).
55+
2. If you don't have write access to `main`, change `catalog:` under `variables` to a catalog you can write to.
56+
57+
### Deployment
58+
59+
Deploy to dev:
60+
61+
```bash
62+
databricks bundle deploy --target dev
63+
```
64+
65+
```bash
66+
databricks bundle run bookings_kpis_metric_view --target dev
67+
```
68+
69+
Deploy to production:
70+
71+
```bash
72+
databricks bundle deploy --target prod
73+
```
74+
75+
```bash
76+
databricks bundle run bookings_kpis_metric_view --target prod
77+
```
78+
79+
The metric view will be created at `<catalog>.<your_username>.bookings_kpis` (dev) or `<catalog>.prod.bookings_kpis` (prod).
80+
81+
### Notes
82+
83+
- The job has a daily `periodic` trigger so the view definition is re-applied in production. [Development-mode](https://docs.databricks.com/dev-tools/bundles/deployment-modes.html) pauses the trigger automatically, so it only fires after `bundle deploy --target prod`.
84+
- Set `source:` in the YAML body to any UC table you read from. The sample `samples.wanderbricks.bookings` is convenient for getting started. For production, use a table from your own pipeline.
85+
86+
## Learn more
87+
88+
- [Unity Catalog Metric Views](https://docs.databricks.com/metric-views/) — Official documentation
89+
- [Metric View YAML Reference](https://docs.databricks.com/metric-views/yaml-ref)
90+
- [Declarative Automation Bundles](https://docs.databricks.com/dev-tools/bundles/index.html)

‎knowledge_base/metric_view/databricks.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# This example bundle defines a job that creates a Unity Catalog Metric View.
2+
#
3+
# Metric Views are a Unity Catalog feature that let you declare reusable
4+
# dimensions and measures over a base table, queryable via the MEASURE() SQL
5+
# function.
6+
#
7+
# Docs:
8+
# https://docs.databricks.com/metric-views/
9+
# https://docs.databricks.com/metric-views/yaml-ref
10+
11+
bundle:
12+
name: metric_view
13+
14+
include:
15+
- resources/*.yml
16+
17+
variables:
18+
catalog:
19+
description: The Unity Catalog where the metric view will be created.
20+
schema:
21+
description: The schema where the metric view will be created.
22+
warehouse_id:
23+
description: SQL warehouse ID used to run the CREATE VIEW statement. To list available warehouses, use `databricks warehouses list`.
24+
25+
targets:
26+
dev:
27+
mode: development
28+
default: true
29+
variables:
30+
catalog: main
31+
schema: ${workspace.current_user.short_name}
32+
warehouse_id: <your-warehouse-id>
33+
prod:
34+
mode: production
35+
variables:
36+
catalog: main
37+
schema: prod
38+
warehouse_id: <your-warehouse-id>

‎knowledge_base/metric_view/resources/bookings_kpis.job.yml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
resources:
2+
jobs:
3+
bookings_kpis_metric_view:
4+
name: bookings_kpis_metric_view
5+
description: Creates/refreshes the `bookings_kpis` Unity Catalog metric view.
6+
7+
trigger:
8+
# Re-apply the view definition daily. Dev deployment mode pauses this trigger;
9+
# in prod the job runs once per day. See https://docs.databricks.com/api/workspace/jobs/create#trigger
10+
periodic:
11+
interval: 1
12+
unit: DAYS
13+
14+
parameters:
15+
- name: catalog
16+
default: ${var.catalog}
17+
- name: schema
18+
default: ${var.schema}
19+
20+
tasks:
21+
- task_key: create_metric_view
22+
sql_task:
23+
warehouse_id: ${var.warehouse_id}
24+
file:
25+
path: ../src/bookings_kpis.metric_view.sql

‎knowledge_base/metric_view/src/bookings_kpis.metric_view.sql‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
-- Create (or replace) a Unity Catalog Metric View over samples.wanderbricks.bookings.
2+
-- See https://docs.databricks.com/metric-views/yaml-ref for the YAML syntax.
3+
--
4+
-- Once deployed and run, query the metric view from any SQL editor with:
5+
-- SELECT MEASURE(total_bookings), MEASURE(total_revenue)
6+
-- FROM <catalog>.<your_schema>.bookings_kpis
7+
-- WHERE check_in_month >= '2024-01-01'
8+
-- GROUP BY check_in_month;
9+
10+
CREATE SCHEMA IF NOT EXISTS IDENTIFIER({{catalog}} || '.' || {{schema}});
11+
USE CATALOG IDENTIFIER({{catalog}});
12+
USE IDENTIFIER({{schema}});
13+
14+
CREATE OR REPLACE VIEW bookings_kpis
15+
WITH METRICS
16+
LANGUAGE YAML
17+
AS $$
18+
version: 1.0
19+
comment: Booking KPIs (count, revenue, AOV, guests) over samples.wanderbricks.bookings.
20+
source: samples.wanderbricks.bookings
21+
22+
filter: status = 'confirmed'
23+
24+
dimensions:
25+
- name: check_in_date
26+
expr: check_in
27+
- name: check_in_month
28+
expr: date_trunc('MONTH', check_in)
29+
- name: guests_count
30+
expr: guests_count
31+
32+
measures:
33+
- name: total_bookings
34+
expr: COUNT(1)
35+
comment: Number of confirmed bookings.
36+
- name: total_revenue
37+
expr: SUM(total_amount)
38+
comment: Total revenue across confirmed bookings.
39+
- name: avg_booking_value
40+
expr: AVG(total_amount)
41+
comment: Average revenue per confirmed booking.
42+
- name: total_guests
43+
expr: SUM(guests_count)
44+
comment: Total guests across confirmed bookings.
45+
$$;

‎knowledge_base/metric_view_dbt/README.md‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# Unity Catalog Metric View using dbt
2+
3+
This project demonstrates how to materialize a [Unity Catalog Metric View](https://docs.databricks.com/metric-views/) via dbt-databricks using Declarative Automation Bundles. The metric view is defined as a dbt model with the [`metric_view` materialization](https://github.com/databricks/dbt-databricks/pull/1285) added in dbt-databricks 1.12.0.
4+
5+
For the SQL-job variant, see [`../metric_view`](../metric_view).
6+
7+
## `bookings_kpis` metric view
8+
9+
This project defines `bookings_kpis`, a metric view over the public sample dataset `samples.wanderbricks.bookings`.
10+
11+
The model lives in [`src/models/bookings_kpis.sql`](src/models/bookings_kpis.sql):
12+
13+
```sql
14+
{{ config(materialized='metric_view') }}
15+
16+
version: 1.0
17+
source: samples.wanderbricks.bookings
18+
filter: status = 'confirmed'
19+
dimensions:
20+
- name: check_in_month
21+
expr: date_trunc('MONTH', check_in)
22+
measures:
23+
- name: total_bookings
24+
expr: COUNT(1)
25+
- name: total_revenue
26+
expr: SUM(total_amount)
27+
```
28+
29+
The `{{ config(materialized='metric_view') }}` line is the only jinja; everything below it is the metric view YAML body. dbt-databricks issues the equivalent `CREATE OR REPLACE VIEW … WITH METRICS LANGUAGE YAML` against the warehouse at `dbt run` time.
30+
31+
Once materialized, query the metric view from any SQL editor:
32+
33+
```sql
34+
SELECT
35+
check_in_month,
36+
MEASURE(total_bookings) AS bookings,
37+
MEASURE(total_revenue) AS revenue
38+
FROM <catalog>.<your_schema>.bookings_kpis
39+
GROUP BY check_in_month
40+
ORDER BY check_in_month;
41+
```
42+
43+
## Get started
44+
45+
### Prerequisites
46+
47+
* Databricks workspace with Unity Catalog enabled
48+
* A SQL warehouse on a runtime that supports Unity Catalog metric views
49+
* Databricks CLI installed and configured
50+
51+
### Setup
52+
53+
1. In `dbt_profiles/profiles.yml`, set `http_path` (in both `dev` and `prod`) to one of your warehouses (`databricks warehouses list`) and update `catalog` to one you can write to (the default `main` is often not writable).
54+
55+
### Deployment
56+
57+
Deploy to dev:
58+
59+
```bash
60+
databricks bundle deploy --target dev
61+
```
62+
63+
```bash
64+
databricks bundle run metric_view_dbt_job --target dev
65+
```
66+
67+
Deploy to production:
68+
69+
```bash
70+
databricks bundle deploy --target prod
71+
```
72+
73+
```bash
74+
databricks bundle run metric_view_dbt_job --target prod
75+
```
76+
77+
The metric view will be created at `<catalog>.<your_username>.bookings_kpis` (dev) or `<catalog>.prod.bookings_kpis` (prod).
78+
79+
### Notes
80+
81+
- The job has a daily `periodic` trigger so `dbt run` re-applies the view definition in production. [Development-mode](https://docs.databricks.com/dev-tools/bundles/deployment-modes.html) pauses the trigger automatically, so it only fires after `bundle deploy --target prod`.
82+
- Requires `dbt-databricks >= 1.12.0` (the version that introduced the `metric_view` materialization). The job pins this in its task environment.
83+
- The model file is `.sql` even though its body is YAML — dbt model files must use `.sql`. dbt-databricks wraps the body in `CREATE OR REPLACE VIEW … LANGUAGE YAML AS $$ … $$` at run time.
84+
- Set `source:` in the YAML body to any UC table you read from. For production, replace `samples.wanderbricks.bookings` with a table from your own pipeline.
85+
86+
## Learn more
87+
88+
- [Unity Catalog Metric Views](https://docs.databricks.com/metric-views/) — Official documentation
89+
- [Metric View YAML Reference](https://docs.databricks.com/metric-views/yaml-ref)
90+
- [`metric_view` materialization in dbt-databricks](https://github.com/databricks/dbt-databricks/pull/1285)
91+
- [Declarative Automation Bundles](https://docs.databricks.com/dev-tools/bundles/index.html)

‎knowledge_base/metric_view_dbt/databricks.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# This example bundle materializes a Unity Catalog Metric View using dbt-databricks.
2+
#
3+
# Metric Views are a Unity Catalog feature that let you declare reusable
4+
# dimensions and measures over a base table, queryable via the MEASURE() SQL
5+
# function.
6+
#
7+
# See also the SQL-job variant at ../metric_view.
8+
#
9+
# Docs:
10+
# https://docs.databricks.com/metric-views/
11+
# https://docs.databricks.com/metric-views/yaml-ref
12+
# https://docs.getdbt.com/reference/resource-configs/databricks-configs
13+
14+
bundle:
15+
name: metric_view_dbt
16+
17+
include:
18+
- resources/*.yml
19+
20+
# Deployment targets.
21+
# The schema and catalog for dbt are configured in dbt_profiles/profiles.yml.
22+
targets:
23+
dev:
24+
mode: development
25+
default: true
26+
prod:
27+
mode: production

‎knowledge_base/metric_view_dbt/dbt_profiles/profiles.yml‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# dbt profiles used by the deployed dbt job in resources/metric_view_dbt.job.yml.
2+
#
3+
# For local development with the dbt CLI you should create your own profile in
4+
# ~/.dbt/profiles.yml using `dbt init`; this file is only read by the job.
5+
metric_view_dbt:
6+
target: dev # default target
7+
outputs:
8+
9+
# The 'dev' target uses the workspace's current_user.short_name as the
10+
# schema (passed in as the `dev_schema` var from the job; see the job YAML).
11+
dev:
12+
type: databricks
13+
method: http
14+
catalog: main
15+
schema: "{{ var('dev_schema') }}"
16+
17+
http_path: /sql/1.0/warehouses/abcdef1234567890
18+
19+
# DBT_HOST / DBT_ACCESS_TOKEN are injected by Databricks at run time.
20+
host: "{{ env_var('DBT_HOST') }}"
21+
token: "{{ env_var('DBT_ACCESS_TOKEN') }}"
22+
23+
prod:
24+
type: databricks
25+
method: http
26+
catalog: main
27+
schema: prod
28+
29+
http_path: /sql/1.0/warehouses/abcdef1234567890
30+
31+
host: "{{ env_var('DBT_HOST') }}"
32+
token: "{{ env_var('DBT_ACCESS_TOKEN') }}"

‎knowledge_base/metric_view_dbt/dbt_project.yml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: 'metric_view_dbt'
2+
version: '1.0.0'
3+
config-version: 2
4+
5+
# Which 'profile' (in dbt_profiles/profiles.yml) dbt uses for this project.
6+
profile: 'metric_view_dbt'
7+
8+
# Everything dbt needs lives under src/ so that non-dbt bundle resources
9+
# (such as the job in resources/) can sit alongside without confusing dbt.
10+
model-paths: ["src/models"]
11+
analysis-paths: ["src/analyses"]
12+
test-paths: ["src/tests"]
13+
seed-paths: ["src/seeds"]
14+
macro-paths: ["src/macros"]
15+
snapshot-paths: ["src/snapshots"]
16+
17+
clean-targets:
18+
- "target"
19+
- "dbt_packages"
20+
21+
# Default materialization is 'view'; the single metric_view model overrides
22+
# this with `{{ config(materialized='metric_view') }}`.
23+
models:
24+
metric_view_dbt:
25+
+materialized: view

0 commit comments

Comments
 (0)