Skip to content

feat: support template functions in custom step types#2020

Open
yottahmd wants to merge 4 commits intomainfrom
custom-step-template-functions
Open

feat: support template functions in custom step types#2020
yottahmd wants to merge 4 commits intomainfrom
custom-step-template-functions

Conversation

@yottahmd
Copy link
Copy Markdown
Collaborator

@yottahmd yottahmd commented Apr 20, 2026

Summary

  • share Dagu's hermetic template function map between template steps and custom step type rendering
  • keep the custom-step json helper while exposing the same pipeline-friendly functions and slim-sprig subset
  • add coverage for custom step template functions, blocked functions, JSON encoding, and typed harness prompt injection

Documentation

  • updated dagucloud/docs main directly in commits c2207df and 6ad74a6 with custom step type, harness, and template function availability docs

Testing

  • go test ./internal/core/spec ./internal/runtime/builtin/template ./internal/runtime/builtin/harness -count=1
  • pnpm build in docs

Summary by CodeRabbit

  • Refactor
    • Centralized template function management for consistent pipeline behavior and uniform enforcement of blocked functions across renderers.
  • New Features
    • Custom step templates now use the shared hermetic function set with improved pipeline-friendly semantics; JSON helper output remains unchanged.
  • Tests
    • Added unit and integration tests covering template helpers, blocked-function failures, and custom-step template behavior.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 20, 2026

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0b65127c-0636-4941-a32a-96b49d7994a6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Centralized hermetic Go text-template function map into internal/cmn/templatefuncs exposing FuncMap() and BlockedFuncNames(). Runtime and spec code now use this shared map; tests added/updated to validate function behavior, erroring on blocked names, and JSON helper preservation.

Changes

Cohort / File(s) Summary
Template Function Centralization
internal/cmn/templatefuncs/functions.go
New file providing FuncMap() template.FuncMap (builds hermetic slim-sprig map, prunes a blocked-name list, and registers Dagu-specific overrides for split,join,count,add,empty,upper,lower,trim,default) and BlockedFuncNames().
Template function tests
internal/cmn/templatefuncs/functions_test.go
New tests: TestFuncMapJoinRejectsUnsupportedInput (ensures join errors on unsupported types) and TestFuncMapCountNilIsZero (ensures count nil renders 0).
Runtime Template Integration
internal/runtime/builtin/template/template.go
Replaced local func-map construction and blocked-list with calls to templatefuncs.FuncMap() and templatefuncs.BlockedFuncNames(); removed redundant overrides and unused imports.
Runtime template tests
internal/runtime/builtin/template/template_test.go
Updated TestFuncMap_JoinGenericSlices to accept join returning (string, error) and assert no error before comparing result.
Spec Template Usage
internal/core/spec/step_types.go
renderCustomStepTemplateString now reuses templatefuncs.FuncMap() and then registers/overrides the json helper into that map before parsing templates.
Spec tests (custom steps)
internal/core/spec/step_types_test.go
Added four tests covering hermetic template pipelines, JSON helper quoting/escaping preservation, blocked-function rejection (e.g., now), and typed $input usage for harness custom steps.

Sequence Diagram(s)

sequenceDiagram
    participant StepLoader as Step Loader
    participant TemplateFuncs as TemplateFuncs (package)
    participant TemplateBuilder as Go Template
    participant Runtime as Runtime Executor

    StepLoader->>TemplateFuncs: call FuncMap()
    TemplateFuncs-->>StepLoader: return hermetic FuncMap (with overrides)
    StepLoader->>TemplateBuilder: Parse template with FuncMap (+ json helper)
    TemplateBuilder-->>StepLoader: compiled template
    StepLoader->>Runtime: provide compiled template + input
    Runtime->>TemplateBuilder: Execute(template, data)
    TemplateBuilder-->>Runtime: rendered string/args
    Runtime->>Runtime: execute task with rendered command/args
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the main change: introducing support for template functions in custom step types by centralizing hermetic template function maps.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch custom-step-template-functions

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@yottahmd yottahmd force-pushed the custom-step-template-functions branch from d5f2720 to c214ad2 Compare April 20, 2026 15:37
@yottahmd
Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 20, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
internal/cmn/templatefuncs/functions.go (2)

39-63: join silently stringifies non-slice inputs (e.g. maps).

The default branch calls fmt.Sprint(v) for anything that isn't a slice/array, which means passing a map or scalar returns its fmt representation rather than an error or empty string. That's easy to miss in templates and produces output like map[a:1 b:2]. Consider returning "" (or an error) for unsupported kinds so misuse is visible rather than silently rendering Go's default formatting.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/cmn/templatefuncs/functions.go` around lines 39 - 63, The join
template func silently stringifies non-slice/array inputs (e.g. maps) by falling
through to fmt.Sprint; update the join function so that after the reflect-based
slice/array check (using rv := reflect.ValueOf(v) and rv.Kind()), unsupported
kinds (anything not slice/array) return an empty string (or propagate an error)
instead of fmt.Sprint(v) so maps and scalars don't silently render their Go fmt
representation.

64-74: Consider handling nil input in count consistently with empty/default.

empty and default explicitly short-circuit on v == nil, but count(nil) falls through to reflect.ValueOf(nil).Kind() == Invalid and returns an error. In templates where a pipeline value may be nil (e.g. missing map key), this will surface as a template execution error rather than 0. Consider treating nil as length 0 for symmetry.

Proposed tweak
 m["count"] = func(v any) (int, error) {
+    if v == nil {
+        return 0, nil
+    }
     rv := reflect.ValueOf(v)
     switch rv.Kind() { //nolint:exhaustive // unsupported kinds return an error below
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/cmn/templatefuncs/functions.go` around lines 64 - 74, The count
template func currently errors for nil because reflect.ValueOf(nil).Kind() is
Invalid; update the anonymous function stored in m["count"] (the count func) to
short-circuit when v == nil (or when reflect.ValueOf(v).IsValid() is false) and
return 0, nil so nil is treated as length 0, otherwise proceed to the existing
switch on rv.Kind() and keep the default unsupported-type error.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@internal/cmn/templatefuncs/functions.go`:
- Around line 126-146: The blockedFuncs list includes entries that do not exist
in slim-sprig v3 (notably "randString" and the crypto-related names
"genPrivateKey", "derivePassword", "buildCustomCert", "genCA",
"genSelfSignedCert", "genSignedCert"); either remove these nonexistent names
from the blockedFuncs slice in functions.go or add a brief comment explaining
they are intentionally retained for compatibility with older/forked sprig
versions, and verify the final list against slim-sprig's current
genericMap/nonhermeticFunctions to ensure you're only blocking real,
non-hermetic functions.

---

Nitpick comments:
In `@internal/cmn/templatefuncs/functions.go`:
- Around line 39-63: The join template func silently stringifies non-slice/array
inputs (e.g. maps) by falling through to fmt.Sprint; update the join function so
that after the reflect-based slice/array check (using rv := reflect.ValueOf(v)
and rv.Kind()), unsupported kinds (anything not slice/array) return an empty
string (or propagate an error) instead of fmt.Sprint(v) so maps and scalars
don't silently render their Go fmt representation.
- Around line 64-74: The count template func currently errors for nil because
reflect.ValueOf(nil).Kind() is Invalid; update the anonymous function stored in
m["count"] (the count func) to short-circuit when v == nil (or when
reflect.ValueOf(v).IsValid() is false) and return 0, nil so nil is treated as
length 0, otherwise proceed to the existing switch on rv.Kind() and keep the
default unsupported-type error.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9b27e34d-f4bb-4938-a300-d0df05f001f5

📥 Commits

Reviewing files that changed from the base of the PR and between d5f2720 and c214ad2.

📒 Files selected for processing (4)
  • internal/cmn/templatefuncs/functions.go
  • internal/core/spec/step_types.go
  • internal/core/spec/step_types_test.go
  • internal/runtime/builtin/template/template.go
✅ Files skipped from review due to trivial changes (2)
  • internal/core/spec/step_types.go
  • internal/core/spec/step_types_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • internal/runtime/builtin/template/template.go

Comment thread internal/cmn/templatefuncs/functions.go
@yottahmd yottahmd force-pushed the custom-step-template-functions branch from c214ad2 to 47ac28b Compare April 20, 2026 15:51
@yottahmd
Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 20, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
internal/runtime/builtin/template/template_test.go (1)

606-608: Consider two-value type assertion for clearer test failure.

The single-value assertion on line 606 will panic with a raw interface conversion message if the join signature ever drifts again (e.g., from a future refactor). Using the two-value form produces a proper test failure with context.

♻️ Optional tweak
-			fn := funcMap["join"].(func(string, any) (string, error))
+			fn, ok := funcMap["join"].(func(string, any) (string, error))
+			require.True(t, ok, "join has unexpected signature: %T", funcMap["join"])
 			result, err := fn(tt.sep, tt.input)
 			require.NoError(t, err)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/runtime/builtin/template/template_test.go` around lines 606 - 608,
The test currently uses a single-value type assertion for fn :=
funcMap["join"].(func(string, any) (string, error)) which will panic on
mismatch; change this to the two-value form (fn, ok :=
funcMap["join"].(func(string, any) (string, error))) and add an assertion (e.g.,
require.True(t, ok, "funcMap[\"join\"] has unexpected type") or require.Truef to
include the actual type) before calling fn so the test fails cleanly with
context instead of panicking.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@internal/runtime/builtin/template/template_test.go`:
- Around line 606-608: The test currently uses a single-value type assertion for
fn := funcMap["join"].(func(string, any) (string, error)) which will panic on
mismatch; change this to the two-value form (fn, ok :=
funcMap["join"].(func(string, any) (string, error))) and add an assertion (e.g.,
require.True(t, ok, "funcMap[\"join\"] has unexpected type") or require.Truef to
include the actual type) before calling fn so the test fails cleanly with
context instead of panicking.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0c4acc65-1f48-4985-b22c-d03d03aaa37d

📥 Commits

Reviewing files that changed from the base of the PR and between c214ad2 and 47ac28b.

📒 Files selected for processing (6)
  • internal/cmn/templatefuncs/functions.go
  • internal/cmn/templatefuncs/functions_test.go
  • internal/core/spec/step_types.go
  • internal/core/spec/step_types_test.go
  • internal/runtime/builtin/template/template.go
  • internal/runtime/builtin/template/template_test.go
✅ Files skipped from review due to trivial changes (1)
  • internal/cmn/templatefuncs/functions.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/core/spec/step_types.go
  • internal/core/spec/step_types_test.go

@yottahmd yottahmd force-pushed the custom-step-template-functions branch from 47ac28b to b45d6db Compare April 20, 2026 15:59
@yottahmd
Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 20, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@yottahmd yottahmd force-pushed the custom-step-template-functions branch from 625d938 to 16ee167 Compare April 20, 2026 17:06
@yottahmd
Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant