Skip to content

fix(edi): reject /document files the KDE gate would refuse before ISA13 - #3820

Merged
PetrefiedThunder merged 5 commits into
claude/modest-volhard-f77f2bfrom
claude/edi-document-kde-gate
Nov 4, 2026
Merged

PetrefiedThunder merged 5 commits into
claude/modest-volhard-f77f2bfrom
claude/edi-document-kde-gate

Conversation

@PetrefiedThunder

@PetrefiedThunder PetrefiedThunder commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

POST /api/v1/ingest/edi/document no longer returns 201 with accepted: 0. Each set now stores through the real gate, or gets a logged 422 before the ISA13 is reserved, so a corrected resend is not a 409 replay.

Stacked on #3818. Merge that first.

Before and after

Set Before After
861 Every document returned 201 accepted: 0 Stores. Missing KDEs return 422
856 201 accepted: 0 with no N1*SF or N1*ST name. A line with no product id stored EDI 856 Item 422
944, 945 945 with no tlc_source_reference returned 201 accepted: 0. Lines with no product id stored a placeholder 422
861, 944, 945 quantity and unit A blank unit stored units, and an 861 with no quantity element stored 1.0 422
850, 810 Every document returned 201 accepted: 0 Every document returns 422 (see below)

944 and 945 were not named in the original task. They are here because 945 hit the same bug when probed, and 944 uses the same check.

Changes

  • _preflight_document_required_kdes walks REQUIRED_KDES_BY_CTE for 861, 944, 945, 850 and 810 and runs before the ISA13 is reserved. It needs a positive quantity and a recognized unit on every line, so bad values return edi_<set>_missing_required_kdes here instead of edi_non_positive_quantity or edi_unmapped_unit_of_measure after the reservation.
  • 856 on /document uses fix(edi): store valid 856s through the required-KDE gate #3818's _legacy_856_placeholder_errors path, not this table.
  • 861 now emits receive_date (BRA02), immediate_previous_source (N1*SF name, or GLN as on 944), reference_document and tlc_source_reference.
  • tlc_source_reference on 861 and 945 follows the fix(edi): store valid 856s through the required-KDE gate #3818 decision: the caller supplies it, and it is never derived from the ship-from party.
  • location_override still does not fill ship_from_location. Only the location name uses it.
  • The 422 message on both routes now points to errors instead of suggesting overrides, because /document has no override for most of these KDEs.

850 and 810

Every 850 and 810 returns 422 on ship_date, ship_from_location and ship_to_location. This is the current behaviour, by maintainer decision. No value was invented: BEG05 is a PO date and BIG01 is an invoice date, and the N1 BY, SE and RI parties are commercial parties, not shipping locations. These sets stored nothing before this PR either.

How to handle them longer term is still open. This PR adds no 810 field extraction and keeps both sets as shipping CTEs.

Verification

All on merge commit cb17b769.

  • services/ingestion/tests -k edi: passed, with 3 skips that need REMEDIATION_EXCHANGE_PG_ADMIN_DSN.
  • Ran the production image with Postgres and Redis and a real tenant key. For 856 (blank unit, blank quantity), 861 (blank unit, bad unit, no quantity), 944 (blank unit) and 945 (blank unit) on /document, and 856 (blank unit, no ship-from name) on /: 422, then a corrected resend on the same ISA13 returned 201 accepted: 1, then replay returned 409. 850 and 810 returned 422.
  • Read the 9 stored rows back from fsma.traceability_events: each has quantity 200, unit cases, and the tlc_source_reference and reference document sent. 11 rejections were logged.
  • test_856_blank_sn1_quantity_is_rejected changed: its document now has parties and a product, so it still reaches the quantity check.
  • An 856 with no item line now reports quantity, unit_of_measure and product_description together.

Not changed

🤖 Generated with Claude Code


Note

Medium Risk
Changes FSMA traceability ingestion semantics on a production EDI route (what persists vs 422, interchange dedup timing), but the behavior is fail-closed validation rather than new persistence of inferred KDEs.

Overview
POST /api/v1/ingest/edi/document no longer succeeds with accepted: 0 when the real required-KDE gate would drop events. Valid 861, 944, and 945 documents now persist; bad ones return 422 with edi_<set>_missing_required_kdes, a logged rejection, and no ISA13 reservation, so a fixed resend is not treated as a 409 replay.

A new _preflight_document_required_kdes runs before interchange dedup for non-856 sets. It enforces REQUIRED_KDES_BY_CTE per line (product, positive quantity, recognized unit, tlc_source_reference, and set-specific locations/references) instead of letting the route synthesize placeholders like units, 1.0, or generic item names that the gate cannot catch. 856 on this route still uses _legacy_856_placeholder_errors, unified under _reject_missing_required_kdes.

861 events now carry gate-facing KDEs (receive_date, immediate_previous_source, reference_document, tlc_source_reference). 850 and 810 always 422 on ship_date, ship_from_location, and ship_to_location because those documents do not record shipments; nothing is stored (same effective outcome as before, but explicit). Docs and integration tests cover pre-reservation rejection, batch failure, and the commercial-document behavior.

Reviewed by Cursor Bugbot for commit 5074af6. Bugbot is set up for automated code reviews on this repo. Configure here.

PetrefiedThunder and others added 2 commits November 4, 2026 00:42
/document answered 201 with accepted: 0 for every 861, 850 and 810, and
for 856, 944 and 945 documents missing a required KDE. It also stored a
synthesized "EDI <set> Item" for a line with no product id.

- Check each set's required KDEs before the ISA13 is reserved. A miss is
  a logged 422, so the corrected resend is not a 409 replay.
- 861 now emits receive_date, immediate_previous_source,
  reference_document and tlc_source_reference. The TLC source comes from
  the caller's form field, as on 856.
- 850 and 810 always return 422. The PO date, invoice date and buyer and
  seller parties are not a ship date or shipping locations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The product check looped over no lines and passed, so these documents
reached FSMA validation after the ISA13 was reserved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
regengine Ready Ready Preview Nov 4, 2026 8:33pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review: 1
  • main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T08:46:44.366779Z 0299785 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

PR Blocker Checklist

Head SHA: cb17b769e5b2d61eb2cacf573476b97f4471520a

Status: Ready

  • All review threads resolved
  • No reviews requesting changes
  • No blocked review engines

…to claude/edi-document-kde-gate

Picks up the secrets pragma fix on the flagged curl line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-gate

Takes #3818's placeholder checks for 856 on /document, so 856 no longer goes
through the document preflight table.

The other line sets had the same hole. 861, 944 and 945 could still store a
blank unit as "units", and an 861 with no quantity element as 1.0. The document
preflight now needs a positive quantity and a recognized unit on every line, so
these return edi_<set>_missing_required_kdes before the ISA13 is reserved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to claude/edi-document-kde-gate

# Conflicts:
#	services/ingestion/app/edi_ingestion/routes.py

Co-authored-by: PetrefiedThunder <131207452+PetrefiedThunder@users.noreply.github.com>

@PetrefiedThunder PetrefiedThunder left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex re-review of head cb17b76 (headless codex exec, read-only). Focused on the delta from 5074af6, then a full pass over the PR diff vs merge-base with main.

Delta: the legacy 856 placeholder validation (_legacy_856_placeholder_errors) now runs in the pre-reservation gate in services/ingestion/app/edi_ingestion/routes.py, before the ISA13 is reserved, instead of after. A corrected resend is no longer blocked by a burned ISA13. test_edi_856_real_kde_gate.py now parametrizes that regression over both routes. Full PR: the required-KDE checks and canonical field mappings are sound, with no correctness or tenant-isolation issues found. Test data and storage fakes are explicit. This was a static review and the tests were not run.

Defects: none.

Note: this review does not clear the PR Blocker Checklist or Head SHA Check Guard. Failing checks at review time: Engine Review Pass.

REVIEW: PASS codex cb17b76

@PetrefiedThunder
PetrefiedThunder merged commit cb0ba0c into claude/modest-volhard-f77f2b Nov 4, 2026
9 checks passed
@PetrefiedThunder
PetrefiedThunder deleted the claude/edi-document-kde-gate branch November 4, 2026 20:38

This branch was successfully deployed

1 active deployment
Preview — cb17b769 Deployed Nov 4, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants