[Security/Reliability] Enforce ETL request-size limits before full body materialization

[seonghobae]

Buyer / availability gap

Fresh protected-source inspection at develop@c2511f6b9d716ef9ab8de60a91fffd826714a4c5 confirms that mightyETL has strong service-level admission limits in EtlBatchProperties (1 MiB default, 64 MiB hard ceiling) and validates UTF-8 payload bytes before database work. However, the synchronous HTTP adapter declares the request as @RequestBody String jsonInput. Spring's HTTP message conversion therefore has to read/materialize the request body as a Java String before EtlService.processData(...) can apply that byte limit.

The existing limit protects parsing, transformation and persistence work, but it is not a transport-bound pre-allocation limit. A very large or chunked JSON request can consume request-buffer/String heap before the repository's 1 MiB admission check runs. This is materially different from record-count/batch atomicity controls and should be treated as an availability/resource-exhaustion boundary.

RCA

• Immediate cause: payload-size enforcement begins only after MVC has converted the complete HTTP entity to String.
• Technical root cause: the same service method supports direct Java callers and HTTP callers, so semantic batch validation was implemented inside EtlService while transport-level byte admission remained implicit in the web container/framework.
• Control failure: tests prove oversized strings are rejected before JDBC, but do not prove an HTTP request larger than the configured limit is rejected before complete body materialization.
• Impact: unauthenticated/weakly protected or merely compromised callers can impose avoidable heap/GC pressure even though the eventual ETL service result is payload_too_large; direct ETL authentication remains separately tracked by [Product/Security Gap] Replace direct ETL HTTP Basic trust boundary #161.

Distinct remedies / feasibility

1. Preferred — bounded HTTP body admission at the real transport boundary. Enforce a configurable byte ceiling before complete MVC String conversion, including chunked/unknown-length requests, while preserving the existing EtlBatchProperties semantic limit for direct Java callers and defense in depth. A servlet filter/request wrapper, endpoint-specific streaming converter, or equivalent maintained Spring/Tomcat mechanism is acceptable only if tests prove where bytes are rejected.
2. Smaller but incomplete — reject known oversized Content-Length early. Useful as an optimization but insufficient alone because chunked or missing-length requests bypass it.
3. Alternative architecture — stream/parse the JSON array under a counting input boundary. Potentially stronger memory behavior but larger blast radius; adopt only with compatibility, ordering, atomic-batch and idempotency evidence.
4. Reject — relying on EtlService's current post-materialization byte count as transport protection. It does not bound the allocation that happens first.
5. Reject — arbitrary container-wide limits that silently break unrelated endpoints or diverge from xtrmetl.etl.max-payload-bytes. One authority or an explicitly documented stricter outer bound is required.

No new secret, external service, database migration or cross-repository mutation is required.

TDD acceptance

1. add a fail-first HTTP-boundary test that sends a request larger than the configured ETL limit and proves current MVC reaches/full-materializes the body before service admission;
2. add equivalent chunked/unknown-length coverage so Content-Length alone cannot satisfy the contract;
3. after repair, reject oversized input with deterministic 413 Payload Too Large or another explicitly versioned bounded API contract before complete request-body materialization;
4. preserve existing direct-call EtlService byte/record limits, whole-batch atomicity, idempotency hashing/ownership, RFC 9457 confidentiality and UTF-8 semantics;
5. keep the accepted limit derived from the canonical ETL payload authority or explicitly document any stricter outer transport ceiling and startup validation between the two;
6. prove requests at and immediately below the byte boundary still reach normal ETL processing;
7. do not buffer/copy the entire oversized request merely to discover it is oversized;
8. cover all new owned production branches/statements at exact 100% and run focused/full hosted tests plus exact-source CI/security/SBOM/SAST and independent review before integration.

Current implementation / writer safety

active_pr via Draft PR #248 on branch security/etl-http-payload-bound-c2511f6, still based directly on exact protected develop@c2511f6b9d716ef9ab8de60a91fffd826714a4c5. Current exact source head 4d928203f96c9f4dbcb2ac02846ae7ad531ba03f is five commits ahead and zero behind that base. It adds endpoint-scoped EtlPayloadAdmissionAdvice without touching EtlController, EtlService, or POM writer lanes.

TDD history is preserved rather than rewritten: RED fd8b330a09fb8ea6055974086241e0acc96ba2cc proved both known-length and unknown/chunked oversized requests reached the current defect; candidate 487d1ba950c26deb25495b367a4021a71be46180 exposed a hosted compile defect; c77503a6b8867a0afa7d3f9b2c6ab4b57d24412f corrected checked stream-error propagation; a5787de108ae5fcd0389a9b9746e69e7577348a4 added exact/below-boundary HTTP acceptance; current 4d928203... proves the transport wrapper rejects known oversize after zero entity bytes and unknown oversize after exactly configured-limit-plus-one bytes while accepting exact-limit input.

Current hosted CI 31428935345 is green across Ubuntu/macOS/Windows on synthetic merge 4fc2ce6075e0b7c4197953b8e98c8c5923d11070; macOS ran 282 ETL tests with zero failures, including 3 direct stream-admission tests and 4 HTTP-boundary tests. Dependency Review, SBOM, Semgrep and Security Scan aggregates are also green. Those aggregates are not merge authority: CI still reports JaCoCo Analyzed bundle 'etl-service' with 0 classes (#162/#164/#205), hard Trivy still warns Dependency version cannot be determined. Child dependencies will not be found. (#196), and source-executing jobs still validate the GitHub synthetic merge rather than accepted literal head identity. No formal review or qualifying independent non-author APPROVED review currently exists.

The branch remains path-disjoint from Draft PR #236 (EtlController.java), Draft PR #199 (EtlService.java), PR #164 (etl-service/pom.xml coverage gate), and canonical docs PR #149. Do not broaden #248 into those files while their writers remain active.

Relationships: #161 direct ETL authentication; #198/#199 batch data-integrity work; #219 failure-log privacy; #236 UTF-8 HTTP response contract; #162/#164/#205 coverage evidence; #196 scanner dependency-graph completeness.