[Data integrity] Make Flyway the sole production ETL schema-mutation authority

[seonghobae]

Buyer / migration-safety gap

Protected develop@622e5e6c3d534f230c390f10e3832efadfc01825 enables both Flyway migration management and Hibernate automatic schema mutation in the deployable ETL service:

spring:
  flyway:
    baseline-on-migrate: ${FLYWAY_BASELINE_ON_MIGRATE:false}
    baseline-version: 0
    clean-disabled: true
    validate-migration-naming: true
  jpa:
    hibernate:
      ddl-auto: update

The active durable-job work already depends on ordered, reviewed Flyway migrations and explicit rollback/forward-recovery evidence. A second runtime DDL authority (ddl-auto: update) makes acquisition-grade schema provenance ambiguous even if it currently happens to find no mapped entity requiring a mutation.

Spring Boot's maintained database-initialization guidance recommends a single mechanism for schema generation and states that when using a higher-level migration tool such as Flyway or Liquibase, it should be used alone to create/initialize the schema. ddl-auto values include none, validate, update, create, and create-drop; non-embedded databases with a schema manager otherwise default to none when not explicitly overridden.

Primary implementation reference to reverify at mutation time: Spring Boot Database Initialization documentation (docs.spring.io/spring-boot/how-to/data-initialization.html).

RCA

• Immediate cause: deployable ETL configuration explicitly forces Hibernate ddl-auto: update while also enabling Flyway.
• Technical root cause: database schema authority is split between versioned migration artifacts and runtime ORM introspection/mutation.
• Systemic/control cause: the repository has strong per-migration tests but no startup/configuration invariant saying protected schema changes may occur only through reviewed migration paths.
• Detection failure: current source contains no obvious @Entity production type, which can make ddl-auto:update appear inert and allow the unsafe authority to persist unnoticed. Inert today is not a valid schema-governance contract for a future entity addition.

Distinct remedies / feasibility

Preferred — Flyway-only mutation, optional Hibernate validation

• make Flyway the only production schema-mutation authority;
• set Hibernate DDL behavior to none or, only if entity-to-schema validation is actually useful and proven against the current non-JPA/JDBC architecture, validate;
• add a deterministic contract test rejecting update, create, and create-drop in deployable profiles;
• preserve test-only isolated schema creation only where a test profile explicitly owns it and cannot reach production data;
• update migration/rollback/Architecture/TRD/Test Strategy documentation accordingly.

Alternative — remove unused JPA schema-generation configuration/dependency surface

If source inventory proves the ETL service does not use JPA entities at all, remove the unused JPA schema-generation configuration (and eventually unused JPA dependency if complete source/runtime verification supports it). This can reduce attack/configuration surface, but must not be mixed into the first authority repair unless dependency removal is independently verified.

Reject

• keeping update because current mapped-entity count appears zero;
• allowing Hibernate to create missing tables while Flyway owns later migrations;
• adding broad baseline-on-migrate=true to hide drift;
• weakening migration validation or using ORM mutation as rollback;
• choosing validate without proving it is compatible with the JDBC-owned schema and does not create a false requirement for ORM mappings.

Current implementation classification

planned / known_gap. Before source mutation, recheck current active ETL application-config ownership and durable migration branches. The first TDD slice should be configuration-authority-only, not a broad JPA refactor.

Acceptance

1. fail-first test proves protected deployable config currently permits Hibernate update;
2. production/deployable profiles cannot run Hibernate automatic schema mutation (update, create, create-drop);
3. Flyway remains the sole production schema-changing startup mechanism;
4. every owned schema change continues to require versioned migration, migration test, rollout/rollback or forward-recovery evidence, and exact PostgreSQL rehearsal where material;
5. tests prevent a future entity/config change from silently re-enabling runtime DDL mutation;
6. clean bootstrap, existing-volume upgrade and active durable-job migrations remain compatible;
7. FLYWAY_BASELINE_ON_MIGRATE remains fail-closed by default and is not used to paper over unmanaged drift;
8. canonical TRD/Architecture/ADR/Data Model/Test Strategy/Migration/Traceability identifies one schema authority and distinguishes bootstrap compatibility SQL from versioned service migrations;
9. exact-source CI/security/non-vacuous coverage/review gates pass before protected integration.

Relationship: #150/#155 own legacy bootstrap retirement; #143–#148 own durable-job migration stack; #159 owns canonical architecture/data/migration traceability; #165 owns release/provenance.

[seonghobae]

2026-08-10 TDD execution evidence — PR #184

The first bounded implementation has now progressed from planned to active_pr without broadening the demonstrated boundary.

RED

• fail-first source: 65e69204f2396a47902d597b18ac9e5b7cc01b80;
• CI run 31345299326, macOS job 93325991714 checked out synthetic merge aef12a331de1977a88862f64eaa862fa2724e018 (Merge 65e6920... into develop@622e5e6...);
• production and test source compiled successfully;
• exactly FlywaySchemaAuthorityTest.deployableConfigurationMakesFlywayTheOnlySchemaMutationAuthority failed because ddl-auto: none was absent; 276 ETL tests ran with 1 failure, 0 errors, 0 skipped;
• therefore the RED reached the intended deployable configuration authority rather than failing in setup/import/runner infrastructure.

GREEN candidate

• current PR fix(data): make Flyway the sole schema mutation authority #184 source: 9d51ebfc6919d2a9b046f55868313ba134ee18b3;
• production delta is one line only: ddl-auto: update -> ddl-auto: none;
• CI 31345440443 is terminal-success on macOS/Ubuntu/Windows;
• Dependency Review 31345440452, SBOM 31345440488, SAST 31345440433, and Security Scan 31345440449 are terminal-success;
• macOS CI job 93326388060 ran 276 ETL tests with 0 failures/errors/skips and the new FlywaySchemaAuthorityTest passed.

The current CI checked out synthetic merge 8a6b72fce56942ec60c0e48bd743cce68cb3ee21 (Merge 9d51ebf... into develop@622e5e6...), so this remains integration-tree compatibility evidence rather than literal-source proof. The same job still reports Analyzed bundle 'etl-service' with 0 classes; #162/#164 therefore remains a prerequisite before this can satisfy the repository's non-vacuous 100% coverage contract.

Issue remains open until #184 reaches protected develop and canonical #149/#159 architecture/migration/test traceability is reconciled. No JPA dependency removal, Hibernate validate, Flyway baseline weakening, or migration rewrite was introduced.

[seonghobae]

Fresh acceptance reconciliation for active repair #184:

• protected/live base remains develop@622e5e6c3d534f230c390f10e3832efadfc01825;
• behavior RED 65e69204f2396a47902d597b18ac9e5b7cc01b80 proved ddl-auto: update at the deployable production boundary;
• behavior GREEN 9d51ebfc6919d2a9b046f55868313ba134ee18b3 changed only spring.jpa.hibernate.ddl-auto to none;
• documentation RED 7da4dfdccf1aa194fa36d0962aa19d1b8fac01d6 reached the real contract and failed only because source-backed migration-authority doctoring was absent;
• exact current source head d081e1080ce4d3bfd959fab6a408e76013eb8627 adds that doctoring with current Spring Boot/Flyway primary references, schema-history/checksum integrity, baseline/clean controls, bootstrap-vs-runtime authority, PostgreSQL-realistic testing, forward-recovery/rollback, standalone/MSA ownership, and privacy/security boundaries.

Current exact-source-head PR-event aggregates are all terminal-success: CI 31347187208, Dependency Review 31347187181, SBOM 31347187172, SAST 31347187169, Security Scan 31347187174. CI macOS 93331176847 checked out synthetic merge da78bbad6c1aed89f7a1dd2b8dcfda48f8f6a564; FlywaySchemaAuthorityTest passed 2/2, ETL ran 277 tests, CDC 106, gateway passed, full reactor BUILD SUCCESS.

This still does not close the issue: current source-executing CI is synthetic-merge evidence rather than literal-head proof, the inherited protected JaCoCo gate still reports Analyzed bundle 'etl-service' with 0 classes, and no qualifying independent non-author formal approval exists. #162/#164 remains the coverage-control prerequisite. PR #184 body is reconciled to the exact current state and remains Draft.