Repository navigation
fix(dispatch): send split-EDI outputs instead of the original file - #40
Merged
Merged
Conversation
The splitter wrote split files into a TemporaryDirectory owned by EDISplitterStep.execute(), which deleted them before returning, so the production pipeline discarded the split outputs and sent the original multi-invoice file — reported as a success. The working split-aware flow lived only in dead code (DispatchOrchestrator._process_split_pipeline, zero callers). FileProcessor now drives splitting itself with a caller-owned temp dir tracked on the processing context, converts and sends each split file through the normal send path, and threads the UPC dictionary into the splitter so category filtering works (also fixing the filter-without-split case, which sent the unfiltered original). EDISplitterStep.execute() is kept as a compat shim and no longer deletes its outputs. Pipeline imports in file_processor stay function-local (TYPE_CHECKING for annotations) so the new split flow does not reintroduce a module-level dispatch.results -> dispatch.services -> dispatch.pipeline cycle. Adds end-to-end tests through FileProcessor.process_file with a real EDISplitterStep and a snapshot backend asserting the split files (and their filtered variants) are what actually reach the backends. Generated with Codebuff 🤖 Co-Authored-By: Codebuff <noreply@codebuff.com>
dtg01100
force-pushed
the
fix/split-edi-pipeline
branch
from
August 9, 2026 09:26
4641e7a to
b3ba869
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The production pipeline sent the original unsplit file when
split_ediwas enabled.EDISplitterStep.execute()wrote split outputs into aTemporaryDirectoryit owned, deleted them before returning, andFileProcessor._run_splittingdiscarded the return value — so recipients got the wrong granularity and it was recorded as success. The correct split-aware flow existed only in dead code.Changes
FileProcessornow drives splitting with a caller-owned temp dir (tracked on the processing context, cleaned up by the existing artifact cleanup) and converts + sends each split file through the normal send path.EDISplitterStep.execute()is a compat shim that no longer deletes its outputs.file_processor.pystay function-local to avoid reintroducing a module-level import cycle.Tests
End-to-end tests run a real
EDISplitterStepthroughFileProcessor.process_filewith a snapshot backend asserting the delivered bytes are the split (and category-filtered) outputs — the original never reaches a backend. Verified with mutation testing: breaking UPC threading or temp-dir tracking makes every new e2e test fail.2,506 passed, 0 failures across the dispatch unit + integration suites; ruff clean; no new mypy errors.