Python: a composite reports the data tables its children produce - #8684
Merged
Conversation
RecipeDescriptor.from_recipe returned only the tables a recipe owns, so a composite whose tables all come from its recipe_list advertised none. Java's Recipe#createRecipeDescriptor aggregates child tables (aggregateDataTableDescriptors), so Java composites are unaffected and only Python-authored ones lose them -- including from the marketplace listing, which is what consumers resolve a run's data tables against. The rows are still collected, but nothing downstream can describe the table they belong to. Union a recipe's own tables with its children's, deduped by name and parent-first. Sub-descriptors are built the same way, so the union is recursive. RpcRecipe gains an optional data_tables argument. It references a recipe on another peer by name, and descriptors are built during marketplace registration when no peer is necessarily connected, so the delegate's tables cannot be introspected here -- the composite author declares them. This also stops data_tables from falling into **options and being shipped to the delegate as a recipe option.
jkschneider
approved these changes
Aug 27, 2026
github-project-automation
Bot
moved this from In Progress
to Ready to Review
in OpenRewrite
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RecipeDescriptor.from_recipereturned only the tables a recipe owns, so a Python composite whose tables all come from itsrecipe_listadvertised none — the rows are still collected at run time, but nothing downstream can describe the table they belong to. Java'sRecipe#createRecipeDescriptoralready aggregates child tables viaaggregateDataTableDescriptors, so Java composites are unaffected and this is a Python/Java parity gap; it needs a Python-authored composite whose table is owned by a child, which is why no shipped recipe hits it.This unions a recipe's own tables with its children's, deduped by name and parent-first — sub-descriptors are built the same way, so the union is recursive.
RpcRecipegains an optionaldata_tablesargument: it references a recipe on another peer by name and descriptors are built during marketplace registration when no peer is necessarily connected, so the delegate's tables can't be introspected and the composite author declares them. That also stopsdata_tablesfrom falling into**optionsand being shipped to the delegate as a recipe option.Verified with
tests/test_composite_data_tables.py(5 of its 7 cases fail before the change, all pass after) and by running the customer-shaped recipes through_collect_marketplace_rows, where the composite's payload goes from[]to both tables with full column metadata. Full suite: 1882 passed; the 4 failures are pre-existing PEP 695 (type X = int) parse failures that also fail on a clean tree under Python 3.11.tests/rpcwas not run — it needs the generated Java classpath.This specifically addresses the bug reported in https://github.com/moderneinc/customer-requests/issues/3051