Child Workflows for Large Module Decomposition

[bdsaglam]

Problem

Large DSPy modules with many LLM calls create workflows with long histories. Temporal recommends decomposing into child workflows for:

• Better isolation
• Improved observability
• History size management
• Independent retry/timeout policies

Example Use Case

class ResearchPipeline(dspy.Module):
    def __init__(self):
        self.planner = PlannerModule()      # Could be child workflow
        self.researcher = ResearchModule()   # Could be child workflow  
        self.writer = WriterModule()         # Could be child workflow

Each sub-module could run as a child workflow with its own history.

Proposed Solution

Allow marking sub-modules as child workflows:

temporal_pipeline = TemporalModule(
    pipeline,
    child_workflows=["planner", "researcher", "writer"]
)

Or automatic detection based on module complexity.

Implementation Notes

• Use workflow.execute_child_workflow() for marked modules
• Each child workflow has its own activities
• Parent workflow orchestrates child workflows
• Consider continue_as_new for very long-running modules

Benefits

• Cleaner Temporal UI (hierarchical view)
• Independent failure handling per sub-module
• Better scalability for complex agents

Priority

Medium

References

• Temporal child workflows documentation
• continue_as_new for history management