Skip to content

Extend ddp new to scaffold all DataDesigner plugin types #22

@eric-tramel

Description

@eric-tramel

Parent epic: #15

Depends on: #21

Why

The runtime plugin system supports column generators, seed readers, and processors. The current scaffold emits only a column-generator package, so DDPlugins is not yet a complete reference tap authoring workflow.

CLI contract

Extend ddp new to:

uv run ddp new my-plugin --type column-generator
uv run ddp new my-plugin --type seed-reader
uv run ddp new my-plugin --type processor

Backwards compatibility:

uv run ddp new my-plugin

must continue to create a column-generator plugin and the help text must say that this is the default.

Generated column-generator template

  • Config class subclasses data_designer.config.base.SingleColumnConfig.
  • Discriminator field is column_type: Literal["<slug>"] = "<slug>".
  • Implementation class subclasses ColumnGeneratorFullColumn[ConfigClass].
  • Plugin object uses PluginType.COLUMN_GENERATOR.
  • Test imports plugin and calls assert_valid_plugin(plugin).

Generated seed-reader template

  • Config class subclasses data_designer.config.seed_source.FileSystemSeedSource.
  • Discriminator field is seed_type: Literal["<slug>"] = "<slug>".
  • Implementation class subclasses data_designer.engine.resources.seed_reader.FileSystemSeedReader[ConfigClass].
  • Generated implementation should be a minimal placeholder that imports cleanly and clearly marks get_dataset_uri / DuckDB connection behavior as TODOs rather than pretending to be production-ready.
  • Plugin object uses PluginType.SEED_READER.
  • Test imports plugin and calls assert_valid_plugin(plugin).

Generated processor template

  • Config class subclasses data_designer.config.base.ProcessorConfig.
  • Discriminator field is processor_type: Literal["<slug>"] = "<slug>".
  • Implementation class subclasses data_designer.engine.processing.processors.base.Processor[ConfigClass].
  • Generated implementation overrides process_after_batch and returns data unchanged with a TODO.
  • Plugin object uses PluginType.PROCESSOR.
  • Test imports plugin and calls assert_valid_plugin(plugin) if DataDesigner validates processor plugins through that helper; otherwise the test should cover public import and plugin object construction while documenting the DataDesigner validation gap.

Shared generated behavior

  • Use tap config from Add repo-level tap metadata config for ddp tooling #21 for package prefix, author, repository URL, docs URL examples, and data-designer dependency.
  • Use absolute imports only.
  • Use Python 3.10+ annotations.
  • Generate README and docs text that says “plugin type” or “entry point,” not “column type,” unless the generated plugin is specifically a column generator.
  • Add scaffold golden tests for file tree and important file contents for all three plugin types.
  • Add CLI tests for valid types, invalid type errors, and default behavior.

Acceptance criteria

  • ddp new my-plugin --type column-generator creates a valid column-generator plugin.
  • ddp new my-plugin --type seed-reader creates a valid seed-reader plugin.
  • ddp new my-plugin --type processor creates a valid processor plugin or a documented processor-validation-limited scaffold if DataDesigner lacks full validation.
  • uv run pytest devtools/ddp/tests/test_cli.py devtools/ddp/tests/test_scaffold.py -q covers all variants.
  • Generated docs and plugin cards do not label seed readers or processors as column types.

Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestplugin tapPlugin catalog and tap ecosystem work

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions