dbt-monthly-channel-revenue: consecutive_growth_months ground truth is off by one against the ticket's worked example

[genesis-gh-jlangseth]

The frozen ground truth for consecutive_growth_months in dbt-monthly-channel-revenue is one
higher than the value the task's own instruction.md defines and works out by example, so a build
that follows the ticket fails test_all_rows_consecutive_growth. The root cause is a window frame
in the reference solve.sh. This PR corrects the reference, the fixture, and the one
strategic_recommendation label that follows from it.

Everything below is reproducible from files in this repository alone — no database, no Snowflake
account. All line references are at a3278ad102829a6084dde086244a0ef665a8011c (master).

---

dbt-monthly-channel-revenue — the frozen ground truth for consecutive_growth_months is one higher than the ticket's own worked example

What the ticket says

tasks/dbt-monthly-channel-revenue/instruction.md:151-155:

### Consecutive Growth Months (Column 20)
- Count how many months IN A ROW this channel has had positive growth (revenue_mom_pct > 0)
- Reset to 0 when revenue_mom_pct <= 0 or NULL
- First month = 0 (no prior month to compare)
- Example: If a channel has mom_pct of [NULL, 10, 25, -5, 15, 30], consecutive would be [0, 1, 2, 0, 1, 2]

The worked example on the last line is unambiguous: the first month of a growth run scores 1.

What the ground truth stores

tasks/dbt-monthly-channel-revenue/tests/test_outputs.py, positions 7 (consecutive_growth_months)
of the tuples documented at :2013. The smallest instance is two adjacent lines:

2015:    ('2024-01-01', 'WEB', 1743.59, 1, 35.95, None,  None,              0, 73, 58, 'Scale Up', 'Emerging'),
2018:    ('2024-02-01', 'WEB', 2596.48, 1, 59.19, 48.92, 'Moderate Growth', 2, 90, 72, 'Scale Up', 'Market Leader'),

WEB's January revenue_mom_pct is None, so January is 0 — correct. February is the channel's
first positive month, and the ticket's example prints 1 for that position. The fixture stores 2.

Reproduction (no database needed)

Recomputing the column from the ground-truth table's own revenue_mom_pct (position 5) under
the rule at :151-155, on the file as shipped:

import ast, collections, re, pathlib
src = pathlib.Path("tasks/dbt-monthly-channel-revenue/tests/test_outputs.py").read_text()
rows = ast.literal_eval(re.search(r"ALL_ROWS_GROUND_TRUTH = (\[.*?\n\])", src, re.S).group(1))

print(dict(sorted(collections.Counter(r[7] for r in rows).items())))

by_ch = collections.defaultdict(list)
for r in rows:
    by_ch[r[1]].append(r)

mismatch, deltas = 0, collections.Counter()
for rs in by_ch.values():
    rs.sort(key=lambda r: r[0])
    streak = 0
    for r in rs:
        streak = streak + 1 if (r[5] is not None and r[5] > 0) else 0
        if r[7] != streak:
            mismatch += 1
            deltas[r[7] - streak] += 1
print(mismatch, len(rows), dict(deltas))

{0: 23, 2: 18, 3: 6, 4: 3, 5: 1}
28 51 {1: 28}

Three things fall out of that:

• The value 1 never appears in the fixture. Eighteen rows carry 2, and under the ticket's
  rule a streak must pass through 1 to reach 2. That alone is not satisfiable by any build that
  implements :151-155.
• 28 of the 51 rows disagree with the ticket's rule — exactly the 28 rows with positive growth —
  and every single one disagrees by exactly +1.
• There are 18 run-starts in the fixture (first positive month after a NULL or a non-positive
  month) and every one of them is stored as 2.

The full WEB channel makes the shift plain:

month	revenue_mom_pct	ground truth	ticket rule
2024-01	NULL	0	0
2024-02	48.92	2	1
2024-03	43.33	3	2
2024-04	50.65	4	3
2024-05	-36.01	0	0
2024-06	9.04	2	1
2024-07	20.08	3	2
2024-08	6.61	4	3
2024-09	10.90	5	4
2024-10	-11.39	0	0
2024-11	115.26	2	1
2024-12	-8.27	0	0

Where the +1 comes from

The mechanism is in the task's own reference. solution/solve.sh:597-601 builds the reset group
with a running sum whose frame includes the current row:

        sum(case when has_positive_growth = 0 then 1 else 0 end) over (
            partition by channel
            order by month_start
            rows between unbounded preceding and current row
        ) as growth_group

so the streak-breaking month increments the counter at its own row and therefore opens the next
island, consuming row_number() = 1. :634-640 then masks that row back to 0:

        case
            when has_positive_growth = 0 then 0
            else row_number() over (
                partition by channel, growth_group
                order by month_start
            )
        end as consecutive_growth_months

which leaves the first genuinely positive month of every run holding 2.

Your own file already records the ticket's values

tests/test_outputs.py:377-381:

CONSECUTIVE_GROWTH_CHECKS = [
    ('2024-01-01', 'WEB', 0),  # First month, no prior to compare
    ('2024-02-01', 'WEB', 1),  # Positive growth (48.92%)
    ('2024-03-01', 'WEB', 2),  # Positive growth (43.32%)
]

Those are the ticket's numbers, for the same two cells ALL_ROWS_GROUND_TRUTH stores as 2 and
3. That constant is defined and never consumed by any test, so it grades nothing; it is cited as
authored intent, not as a failing assertion.

Why the behavioural tests don't catch it

The three streak tests — test_consecutive_growth_first_month_zero (:897),
test_consecutive_growth_resets_on_decline (:911), test_consecutive_growth_increases (:926)
— are satisfied by both readings. The first month is 0 either way, resets land on 0 either way, and
the value increases within a run either way. Only the frozen table discriminates, and it encodes
the shift.

The change

Three hunks, all in this one commit. They must land together.

1. solution/solve.sh:636-640 — subtract the reset row from the island position:

             else row_number() over (
                 partition by channel, growth_group
                 order by month_start
-            )
+            ) - 1
         end as consecutive_growth_months

Simulating the reference's window logic against the ground truth's own revenue_mom_pct
sequences: as shipped it matches the ticket rule on 23 of 51 rows; with - 1 it matches on
51 of 51. (Excluding the reset row from the group-boundary frame instead — rows between unbounded preceding and 1 preceding — gives the same answer, but returns NULL on a channel's
first row, so I went with the arithmetic form.)

2. tests/test_outputs.py:2015-2065 — subtract 1 from the 28 non-zero
consecutive_growth_months values in ALL_ROWS_GROUND_TRUTH. After the change the value counts
are {0: 23, 1: 18, 2: 6, 3: 3, 4: 1} and the recomputation above reports 0 51 {}.

3. tests/test_outputs.py:2062 — 'Invest Heavily' → 'Scale Up'.

This one is a consequence, not a separate opinion, and it's worth showing the work. I re-derived
strategic_recommendation for all 51 rows from the rules at instruction.md:252-261: against the
values as stored, the table reproduces itself on 51 of 51 rows, so the derivation is faithful.
Applying the same rules with the corrected streaks, exactly one row changes:

2024-12-01 MOBILE  consec 3 -> 2  |  'Invest Heavily' -> 'Scale Up'

At consec = 2 the 'Invest Heavily' rule (momentum >= 80 AND efficiency >= 70 AND consecutive_growth_months >= 3) no longer fires, and the next rule in evaluation order
(momentum >= 70 AND market_share_pct >= 20, with momentum 88 and share 33.03) does. The other two
'Invest Heavily' rows go 4 → 3 and are unaffected — and both still satisfy
test_invest_heavily_criteria (:1340), which I checked. The three 'Maintain' rows go
3 → 2, 2 → 1, 2 → 1, all still >= 1, so that rule's rows are unaffected too.

I also bumped EXPECTED_STRATEGIC_RECOMMENDATION_COUNTS (:257) from
'Scale Up': 13 / 'Invest Heavily': 3 to 14 / 2 to keep it consistent with the table. Like
CONSECUTIVE_GROWTH_CHECKS, that constant is defined once and consumed nowhere, so it grades
nothing either way — it would simply have gone stale. Happy to drop that hunk to keep the diff to
the graded surface.

Blast radius

Two of the 95 tests fail for a ticket-faithful submission, from this single root cause:
test_all_rows_consecutive_growth (:2195) directly, and
test_all_rows_strategic_recommendation (:2161) through the propagation above.

Validation

Run in fresh containers from ghcr.io/snowflake-labs/data-eng-bench-base:1.0.0 with
DB_TYPE=duckdb: bash /solution/solve.sh followed by the task's own tests/test.sh, for each
pairing of reference and suite.

solution/solve.sh	tests/test_outputs.py	result
master	master	95 passed — the shipped reference passes the shipped suite (both carry the shift)
this PR	this PR	95 passed
master	this PR	93 passed, 2 failed: test_all_rows_consecutive_growth — Row 3 (2024-02-01, WEB): expected 1, got 2; test_all_rows_strategic_recommendation — Row 47 (2024-12-01, MOBILE): expected 'Scale Up', got 'Invest Heavily'
this PR	master	93 passed, 2 failed: the same two tests, mirrored

The two failing tests in the cross pairings are exactly the two named under Blast radius, and the
failing cells are exactly the two the reproduction above predicts. No other test moves in any
pairing.

---

Found while running this benchmark at volume and root-causing persistent failures: an agent
solution that implemented :151-155 as written returned 1 for ('2024-02-01', 'WEB') and
failed test_all_rows_consecutive_growth on exactly that cell.

[snowflake-security-bot]

Snowflake Security Review

Security grade: A — Passed ✅

This PR was classified as LOW risk by the automated pre-screen.

[snowflake-security-bot]

Snowflake Security Review

Security grade: A — Passed ✅

This PR was classified as LOW risk by the automated pre-screen.