You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The runtime plugin system supports column generators, seed readers, and processors. The current scaffold emits only a column-generator package, so DDPlugins is not yet a complete reference tap authoring workflow.
CLI contract
Extend ddp new to:
uv run ddp new my-plugin --type column-generator
uv run ddp new my-plugin --type seed-reader
uv run ddp new my-plugin --type processor
Backwards compatibility:
uv run ddp new my-plugin
must continue to create a column-generator plugin and the help text must say that this is the default.
Generated column-generator template
Config class subclasses data_designer.config.base.SingleColumnConfig.
Discriminator field is column_type: Literal["<slug>"] = "<slug>".
Implementation class subclasses ColumnGeneratorFullColumn[ConfigClass].
Plugin object uses PluginType.COLUMN_GENERATOR.
Test imports plugin and calls assert_valid_plugin(plugin).
Generated seed-reader template
Config class subclasses data_designer.config.seed_source.FileSystemSeedSource.
Discriminator field is seed_type: Literal["<slug>"] = "<slug>".
Implementation class subclasses data_designer.engine.resources.seed_reader.FileSystemSeedReader[ConfigClass].
Generated implementation should be a minimal placeholder that imports cleanly and clearly marks get_dataset_uri / DuckDB connection behavior as TODOs rather than pretending to be production-ready.
Plugin object uses PluginType.SEED_READER.
Test imports plugin and calls assert_valid_plugin(plugin).
Generated processor template
Config class subclasses data_designer.config.base.ProcessorConfig.
Discriminator field is processor_type: Literal["<slug>"] = "<slug>".
Implementation class subclasses data_designer.engine.processing.processors.base.Processor[ConfigClass].
Generated implementation overrides process_after_batch and returns data unchanged with a TODO.
Plugin object uses PluginType.PROCESSOR.
Test imports plugin and calls assert_valid_plugin(plugin) if DataDesigner validates processor plugins through that helper; otherwise the test should cover public import and plugin object construction while documenting the DataDesigner validation gap.
Generate README and docs text that says “plugin type” or “entry point,” not “column type,” unless the generated plugin is specifically a column generator.
Add scaffold golden tests for file tree and important file contents for all three plugin types.
Add CLI tests for valid types, invalid type errors, and default behavior.
Acceptance criteria
ddp new my-plugin --type column-generator creates a valid column-generator plugin.
ddp new my-plugin --type seed-reader creates a valid seed-reader plugin.
ddp new my-plugin --type processor creates a valid processor plugin or a documented processor-validation-limited scaffold if DataDesigner lacks full validation.
uv run pytest devtools/ddp/tests/test_cli.py devtools/ddp/tests/test_scaffold.py -q covers all variants.
Generated docs and plugin cards do not label seed readers or processors as column types.
Parent epic: #15
Depends on: #21
Why
The runtime plugin system supports column generators, seed readers, and processors. The current scaffold emits only a column-generator package, so DDPlugins is not yet a complete reference tap authoring workflow.
CLI contract
Extend
ddp newto:Backwards compatibility:
must continue to create a
column-generatorplugin and the help text must say that this is the default.Generated column-generator template
data_designer.config.base.SingleColumnConfig.column_type: Literal["<slug>"] = "<slug>".ColumnGeneratorFullColumn[ConfigClass].PluginType.COLUMN_GENERATOR.pluginand callsassert_valid_plugin(plugin).Generated seed-reader template
data_designer.config.seed_source.FileSystemSeedSource.seed_type: Literal["<slug>"] = "<slug>".data_designer.engine.resources.seed_reader.FileSystemSeedReader[ConfigClass].get_dataset_uri/ DuckDB connection behavior as TODOs rather than pretending to be production-ready.PluginType.SEED_READER.pluginand callsassert_valid_plugin(plugin).Generated processor template
data_designer.config.base.ProcessorConfig.processor_type: Literal["<slug>"] = "<slug>".data_designer.engine.processing.processors.base.Processor[ConfigClass].process_after_batchand returns data unchanged with a TODO.PluginType.PROCESSOR.pluginand callsassert_valid_plugin(plugin)if DataDesigner validates processor plugins through that helper; otherwise the test should cover public import and plugin object construction while documenting the DataDesigner validation gap.Shared generated behavior
data-designerdependency.Acceptance criteria
ddp new my-plugin --type column-generatorcreates a valid column-generator plugin.ddp new my-plugin --type seed-readercreates a valid seed-reader plugin.ddp new my-plugin --type processorcreates a valid processor plugin or a documented processor-validation-limited scaffold if DataDesigner lacks full validation.uv run pytest devtools/ddp/tests/test_cli.py devtools/ddp/tests/test_scaffold.py -qcovers all variants.Dependencies
DD-CORE-PROCESSOR-PLUGIN-VALIDATION.