From 78f0929dd1c47dde7f443c52065d03a78c155a0a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 15:20:31 +0900 Subject: [PATCH 01/59] feat: establish ConceptWeave foundation --- .github/workflows/product.yml | 50 +++ .gitignore | 4 + AGENTS.md | 23 ++ ARCHITECTURE.md | 77 ++++ CHANGELOG.md | 18 + CLAUDE.md | 7 + Cargo.lock | 7 + Cargo.toml | 9 + OPERABILITY.md | 23 ++ README.md | 50 ++- SECURITY.md | 33 ++ TEST_STRATEGY.md | 40 ++ contracts/semantic-candidate.schema.json | 63 ++++ crates/conceptweave-domain/Cargo.toml | 10 + crates/conceptweave-domain/src/lib.rs | 363 +++++++++++++++++++ docs/CONTEXT_MAP.md | 16 + docs/PRD.md | 81 +++++ docs/TRD.md | 58 +++ docs/UBIQUITOUS_LANGUAGE.md | 18 + docs/UML.md | 40 ++ docs/adr/0001-product-boundary.md | 22 ++ docs/adr/0002-truth-publication-lifecycle.md | 23 ++ docs/adr/0003-standards-llm-boundary.md | 19 + docs/adr/README.md | 5 + docs/doctoring/REFERENCES.md | 42 +++ docs/product-technical-gap-baseline.md | 41 +++ rust-toolchain.toml | 4 + scripts/check_coverage.sh | 11 + 28 files changed, 1155 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/product.yml create mode 100644 .gitignore create mode 100644 AGENTS.md create mode 100644 ARCHITECTURE.md create mode 100644 CHANGELOG.md create mode 100644 CLAUDE.md create mode 100644 Cargo.lock create mode 100644 Cargo.toml create mode 100644 OPERABILITY.md create mode 100644 SECURITY.md create mode 100644 TEST_STRATEGY.md create mode 100644 contracts/semantic-candidate.schema.json create mode 100644 crates/conceptweave-domain/Cargo.toml create mode 100644 crates/conceptweave-domain/src/lib.rs create mode 100644 docs/CONTEXT_MAP.md create mode 100644 docs/PRD.md create mode 100644 docs/TRD.md create mode 100644 docs/UBIQUITOUS_LANGUAGE.md create mode 100644 docs/UML.md create mode 100644 docs/adr/0001-product-boundary.md create mode 100644 docs/adr/0002-truth-publication-lifecycle.md create mode 100644 docs/adr/0003-standards-llm-boundary.md create mode 100644 docs/adr/README.md create mode 100644 docs/doctoring/REFERENCES.md create mode 100644 docs/product-technical-gap-baseline.md create mode 100644 rust-toolchain.toml create mode 100755 scripts/check_coverage.sh diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml new file mode 100644 index 00000000..4caa8a1e --- /dev/null +++ b/.github/workflows/product.yml @@ -0,0 +1,50 @@ +name: Product + +on: + pull_request: + push: + branches: [main] + +permissions: + contents: read + +jobs: + rust-quality: + runs-on: ubuntu-latest + steps: + - name: Checkout exact revision + uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + + - name: Show pinned Rust toolchain + run: rustc --version && cargo --version + + - name: Format + run: cargo fmt --all --check + + - name: Clippy + run: cargo clippy --workspace --all-targets -- -D warnings + + - name: Test + run: cargo test --workspace + + - name: Public documentation + env: + RUSTDOCFLAGS: -D warnings + run: cargo doc --workspace --no-deps + + - name: Install cargo-llvm-cov + uses: taiki-e/install-action@1ed6d7be6168f6c9046541087ff549b6bc581fdf # v2.87.2 + with: + tool: cargo-llvm-cov + + - name: Exact owned coverage + run: ./scripts/check_coverage.sh + + - name: Validate public JSON contract + run: jq -e . contracts/semantic-candidate.schema.json >/dev/null + + - name: Lockfile freshness + run: cargo generate-lockfile && git diff --exit-code -- Cargo.lock + + - name: Clean working tree + run: git diff --exit-code diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..c11a1dfc --- /dev/null +++ b/.gitignore @@ -0,0 +1,4 @@ +/target +*.profraw +coverage.json +.DS_Store diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..67347566 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,23 @@ +# AGENTS.md — ConceptWeave + +Read the organization `ContextualWisdomLab/.github` master context and product goal directive before material work. Live GitHub state and this repository's accepted ADRs override remembered chat state. + +## Product boundary + +ConceptWeave owns automatic, evidence-bound **Semantic Model Engineering**. Do not turn it into a semantic catalog, lineage engine, generic LLM gateway, browser crawler, or another product's system of record. + +## Development rules + +- Apply DDD continuously; maintain the Context Map and Ubiquitous Language. +- Rust 1.98.0 is the production baseline for core logic. Production mathematical/vector/model-scoring computation, if introduced, remains Rust-first. +- `conceptweave-domain` has no provider/network/database dependencies. +- External products and providers enter through versioned ports and Anti-Corruption Layers. +- LLM work uses `contextual-orchestrator`; model output is proposal evidence, never semantic authority. +- No direct cross-service application-table SQL. +- New database objects, when introduced, use descriptive two-or-more-word `snake_case` names and 3NF by default. +- Preserve source evidence, truth status, and publication state separately. +- Published semantic truth is immutable; correction uses supersession/new release. +- Public Rust APIs require beginner-readable documentation. +- Owned production coverage target is 100% line/function/region/branch where tooling exposes it. +- Never suppress deprecation warnings; fix causes. +- Never force-push shared branches, self-approve, fabricate checks, or weaken branch protection. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 00000000..c5dd4993 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,77 @@ +# ConceptWeave Architecture + +## Product responsibility + +ConceptWeave owns the process that turns observed enterprise evidence into governed semantic-model releases. It does not own source-system truth or downstream catalog/query experiences. + +```mermaid +flowchart LR + S[Source systems and artifacts] --> O[Source Observation] + O --> D[Semantic Discovery] + D --> V[Model Validation] + V --> G[Governance & Publication] + G --> P[Versioned semantic release] + + CO[contextual-orchestrator] -. proposal assistance .-> D + LW[LineageWeave] -. inferred/proposed lineage .-> O + CG[context-graph-contracts] -. shared graph/provenance contracts .-> P + P --> SDP[semantic-data-portal] + P --> GRC[governance-risk-compliance] + P --> EA[enterprise-architecture-core] +``` + +## DDD context map + +| Context | Type | Owns | Does not own | +| --- | --- | --- | --- | +| Source Observation | Supporting | immutable observations, parser receipts, evidence locations | source-system business truth | +| Semantic Discovery | Core | candidate generation and evidence binding | publication authority | +| Model Validation | Supporting | deterministic validation reports | human review decisions | +| Governance & Publication | Core | proposal lifecycle, review receipts, releases, supersession | catalog/search runtime | +| Interoperability | Supporting | versioned import/export and ACL adapters | foreign product internals | + +## Aggregate boundaries + +### SemanticCandidate + +Smallest consistency boundary for a single proposed semantic artifact and its evidence-bound publication state. It cannot jump directly from Draft to Published. + +### SemanticModelRelease (planned) + +Immutable publication aggregate containing approved candidate identities, release version, artifact digests, validation receipts, reviewer receipts, and supersession metadata. It will reference candidates rather than copy foreign source records. + +## Truth model + +- `observed`: exact source fact; +- `inferred`: derived candidate; +- `proposed`: submitted for governance; +- `authoritative`: explicitly reviewed and published; +- `superseded`: formerly authoritative and replaced; +- `rejected`: explicitly rejected. + +Truth status and publication workflow are distinct. A source observation can be authoritative in its source domain without making an inferred semantic interpretation authoritative. + +## Integration boundaries + +- `contextual-orchestrator`: LLM/model routing only. +- `LineageWeave`: inferred/proposed lineage evidence only. +- `semantic-data-portal`: published semantic artifact consumer/governance/catalog plane; it is not ConceptWeave's internal database. +- `context-graph-contracts`: shared cross-product identifiers, truth/provenance/event contracts where adopted. +- Keyverse: future identity/tenant authentication boundary. + +No direct cross-service application-table SQL is permitted. + +## Foundation directory structure + +```text +crates/ + conceptweave-domain/ # Core domain contract only +contracts/ # Versioned public schemas +docs/ + adr/ # Binding architecture decisions + doctoring/ # Standards/research evidence +scripts/ # Deterministic repository-quality helpers +.github/workflows/ # CI evidence +``` + +Adapters and application services are added only when their bounded responsibility exists; generic `utils`, `helpers`, or `services` dumping grounds are prohibited. diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000..8910d6fa --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,18 @@ +# Changelog + +All notable changes to ConceptWeave are documented here. + +## Unreleased + +### Added + +- Initial ConceptWeave product, DDD, security, test, and operability baselines. +- Rust 1.98.0 `conceptweave-domain` foundation with evidence-bound semantic candidate contracts. +- Fail-closed Draft -> Proposed -> Validated -> Reviewed -> Published lifecycle with explicit rejection and supersession. +- Draft 2020-12 JSON Schema for the semantic-candidate public contract. +- Standards and research doctoring covering stable W3C ontology standards, 2026 RDF/SHACL work in progress, Apache Ossie, and recent LLM ontology-engineering research. + +### Security + +- Model-generated semantics remain non-authoritative until deterministic validation and authorized review. +- Unsafe Rust is forbidden in the core domain crate. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..d8db2650 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,7 @@ +# CLAUDE.md — ConceptWeave + +Follow `AGENTS.md`, `ARCHITECTURE.md`, accepted ADRs, and the organization master context before making changes. + +ConceptWeave's core invariant is: **inference is not authority**. Every generated concept, relation, constraint, dimension, measure, or physical mapping must retain evidence and pass the explicit governance lifecycle before publication. + +Keep domain logic in bounded domain modules, LLM/provider logic behind ports/adapters, and source/consumer systems independent. Prefer deterministic validation and explicit abstention over plausible unsupported output. diff --git a/Cargo.lock b/Cargo.lock new file mode 100644 index 00000000..451324f0 --- /dev/null +++ b/Cargo.lock @@ -0,0 +1,7 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "conceptweave-domain" +version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml new file mode 100644 index 00000000..b5e817a6 --- /dev/null +++ b/Cargo.toml @@ -0,0 +1,9 @@ +[workspace] +members = ["crates/conceptweave-domain"] +resolver = "2" + +[workspace.package] +version = "0.1.0" +edition = "2024" +rust-version = "1.98" +repository = "https://github.com/ContextualWisdomLab/ConceptWeave" diff --git a/OPERABILITY.md b/OPERABILITY.md new file mode 100644 index 00000000..b3302610 --- /dev/null +++ b/OPERABILITY.md @@ -0,0 +1,23 @@ +# Operability Baseline + +ConceptWeave has no production network service or durable database in the foundation slice. This document defines requirements before either is introduced. + +## Runtime requirements + +- explicit startup/readiness/liveness semantics; +- bounded source job queues, deadlines, cancellation, retry classification, and idempotency; +- persistent job receipts before accepting asynchronous work; +- OpenTelemetry sender/receiver ownership documented using the CWL shared observability contract; +- detailed structured error messages with safe identifiers, failure boundary, cause code, retryability, impact, and next action; +- no secrets or unnecessary raw PII in telemetry; +- backup/restore and migration rehearsal before durable persistence is production-ready; +- graceful drain of source parsing, model calls, validation, and publication jobs; +- deterministic replay from immutable source snapshot + extractor/config revisions. + +## Degraded modes + +- LLM unavailable: deterministic observation/validation remains available; discovery may return a typed `model_assistance_unavailable` result rather than fabricate candidates. +- external research unavailable: internal source modeling remains available and reports the missing evidence channel. +- downstream catalog unavailable: publication retains a durable release/outbox receipt and does not lose the governed release. + +Concrete SLO/RPO/RTO values require measured runtime evidence and are not guessed in the foundation. diff --git a/README.md b/README.md index f6c0a4d6..3ec89033 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,51 @@ # ConceptWeave -Automatic ontology and semantic-layer engineering for governed enterprise meaning. +**Automatic ontology and semantic-layer engineering for governed enterprise meaning.** -> This repository was initialized with a minimal protected baseline. Substantive architecture and implementation changes are introduced through pull requests. +ConceptWeave turns heterogeneous enterprise evidence—schemas, APIs, event contracts, documents, code structure, existing vocabularies, and lineage—into **reviewable semantic-model candidates**. It does not make model-generated meaning authoritative by itself. Candidates must retain source evidence, pass deterministic validation, and move through an explicit governance lifecycle before publication. + +## Product boundary + +ConceptWeave owns **semantic model engineering**: + +`observe -> discover -> propose -> validate -> review -> publish` + +It does **not** own: + +- enterprise catalog/search/runtime consumption (`semantic-data-portal`), +- lineage reconstruction (`LineageWeave`), +- cross-product graph/event contracts (`context-graph-contracts`), +- LLM provider routing (`contextual-orchestrator`), or +- the authoritative business data of source systems. + +External source-analysis tools may be integrated behind adapters, but no external fork is treated as ConceptWeave product authority. + +## First release target + +The first vertical is a **relational-schema-to-governed-semantic-model proposal**: + +1. ingest an immutable schema snapshot; +2. derive observed physical entities and relationships; +3. propose concepts, taxonomy/semantic relations, dimensions, measures, constraints, and physical mappings; +4. bind every proposal to exact source evidence; +5. validate structure and consistency; +6. require steward review before publication; and +7. export a versioned semantic package suitable for ontology and analytics consumers. + +Planned publication targets include OWL/RDFS/SKOS, SHACL, JSON-LD, and an Apache Ossie-compatible semantic-model projection when the emerging specification is sufficiently stable for the required subset. + +## Current state + +This foundation PR establishes the Rust domain contract, candidate truth/publication lifecycle, JSON Schema, DDD architecture, standards/research baseline, security/test/operability baselines, and CI. Source adapters, LLM-assisted induction, persistence, reasoning, review UI, and publication adapters remain explicit product gaps. + +## Rust + +The repository is pinned to Rust 1.98.0. The current core has no third-party runtime dependencies. + +```bash +cargo fmt --all --check +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace +``` + +See [`docs/PRD.md`](docs/PRD.md), [`docs/TRD.md`](docs/TRD.md), and [`ARCHITECTURE.md`](ARCHITECTURE.md). diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..9a8fa55a --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,33 @@ +# Security Baseline + +## Trust boundaries + +All source artifacts, generated candidate payloads, external ontology files, model outputs, and future web-retrieved content are untrusted input. + +## Required controls + +- source size, type, nesting, archive/decompression, and parser-time bounds; +- immutable source digests and parser/extractor provenance; +- no credentials, secrets, tokens, DSNs, or raw authorization material in semantic evidence; +- prompt-injection text is source data, never tool or policy instruction; +- LLM calls only through `contextual-orchestrator` with minimum necessary context; +- outbound retrieval, when introduced, uses a reviewed SSRF/DNS-rebinding-safe CWL egress boundary; +- no source-system writes from discovery or validation; +- reviewed authorization required before publication; +- future tenant isolation applies to source snapshots, candidates, review receipts, releases, exports, and object storage; +- published artifacts retain origin/provenance and cannot silently overwrite prior releases. + +## Threats tracked from foundation + +1. semantic poisoning by malicious source text; +2. hallucinated concepts/relations treated as facts; +3. ontology import cycles or reasoning/resource exhaustion; +4. unsafe generated query/expression execution; +5. cross-tenant evidence exposure; +6. provenance stripping during export; +7. malicious or oversized schema/API artifacts; +8. external-source SSRF or credential leakage; +9. model/provider compromise or unexpected retention; +10. governance bypass from Proposed/Validated directly to Published. + +Security findings become tests before the related runtime capability can be marked release-ready. diff --git a/TEST_STRATEGY.md b/TEST_STRATEGY.md new file mode 100644 index 00000000..6d137795 --- /dev/null +++ b/TEST_STRATEGY.md @@ -0,0 +1,40 @@ +# Test Strategy + +## Foundation gates + +- Rust formatting and Clippy with warnings denied; +- unit tests for every domain lifecycle branch; +- owned production line/function/region and LLVM branch coverage target of 100%; +- JSON Schema syntax validation; +- lockfile freshness and clean-tree verification; +- public Rust documentation with `missing_docs` denied. + +## Future product test families + +### Source observation + +Realistic PostgreSQL schema snapshots, OpenAPI/AsyncAPI fixtures, malformed contracts, deep nesting, invalid encoding, duplicate identifiers, archive bombs, parser cancellation, and exact digest/location provenance. + +### Ontology and semantic discovery + +Golden concept/type/taxonomy/relation sets; mapping precision/recall; multilingual labels; synonyms/homonyms; false friends; unrelated sources; cross-domain collisions; explicit no-answer cases. + +### Semantic measures + +Exact deterministic calculations, grain correctness, join/cardinality safety, units, null semantics, time windows, currency/unit conversions through approved deterministic layers, and no LLM arithmetic authority. + +### Validation/reasoning + +OWL consistency where supported, SHACL conformance, cycle constraints, unsatisfiable classes, contradictory ranges/domains, duplicate measures, and bounded reasoner resources. + +### Governance + +No bypass of Reviewed before Published, immutable published releases, rejection, supersession, stale-review protection, maker-checker requirements where configured, and exact audit receipts. + +### Security + +Prompt injection, malicious ontology/source content, SSRF, cross-tenant leakage, secret leakage, expression injection, resource exhaustion, replay, malformed source provenance, and hostile export values. + +### Evaluation + +Model-backed evaluation must include deterministic fixtures and human-reviewed expert cases. Report extraction recall, semantic precision, structural validity, mapping accuracy, citation/provenance completeness, and abstention quality separately rather than collapsing them into one opaque score. diff --git a/contracts/semantic-candidate.schema.json b/contracts/semantic-candidate.schema.json new file mode 100644 index 00000000..7e090a45 --- /dev/null +++ b/contracts/semantic-candidate.schema.json @@ -0,0 +1,63 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://schemas.contextualwisdomlab.org/conceptweave/semantic-candidate/0.1.0", + "title": "ConceptWeave Semantic Candidate", + "type": "object", + "additionalProperties": false, + "required": [ + "candidate_id", + "kind", + "truth_status", + "publication_state", + "evidence" + ], + "properties": { + "candidate_id": {"type": "string", "minLength": 1}, + "kind": { + "enum": [ + "concept", + "taxonomy_relation", + "semantic_relation", + "constraint", + "dimension", + "measure", + "physical_mapping" + ] + }, + "truth_status": { + "enum": [ + "observed", + "inferred", + "proposed", + "authoritative", + "superseded", + "rejected" + ] + }, + "publication_state": { + "enum": [ + "draft", + "proposed", + "validated", + "reviewed", + "published", + "superseded", + "rejected" + ] + }, + "evidence": { + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "additionalProperties": false, + "required": ["source_id", "source_digest", "location"], + "properties": { + "source_id": {"type": "string", "minLength": 1}, + "source_digest": {"type": "string", "minLength": 1}, + "location": {"type": "string", "minLength": 1} + } + } + } + } +} diff --git a/crates/conceptweave-domain/Cargo.toml b/crates/conceptweave-domain/Cargo.toml new file mode 100644 index 00000000..2a3f12e3 --- /dev/null +++ b/crates/conceptweave-domain/Cargo.toml @@ -0,0 +1,10 @@ +[package] +name = "conceptweave-domain" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +repository.workspace = true +description = "Core domain contracts for governed ontology and semantic-layer engineering" + +[lib] +path = "src/lib.rs" diff --git a/crates/conceptweave-domain/src/lib.rs b/crates/conceptweave-domain/src/lib.rs new file mode 100644 index 00000000..a8fe48d4 --- /dev/null +++ b/crates/conceptweave-domain/src/lib.rs @@ -0,0 +1,363 @@ +#![forbid(unsafe_code)] +#![deny(missing_docs)] +//! Core domain contracts for ConceptWeave. +//! +//! ConceptWeave separates observed source evidence from inferred semantic +//! candidates and from reviewed, published semantic-model truth. This crate +//! contains only that domain contract; adapters, persistence, LLM orchestration, +//! and publication formats belong to other bounded contexts. + +use core::fmt; + +/// The kind of source material observed by ConceptWeave. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum SourceKind { + /// A relational schema or database-introspection snapshot. + RelationalSchema, + /// An OpenAPI contract. + OpenApi, + /// An AsyncAPI or event contract. + AsyncApi, + /// Human-authored documentation or a business glossary. + Document, + /// Source-code structure observed through a bounded adapter. + SourceCode, + /// An existing ontology or controlled vocabulary. + ExistingOntology, + /// Provenance or lineage evidence produced by another system. + Lineage, +} + +/// The semantic artifact that a candidate proposes. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum CandidateKind { + /// A domain concept or class. + Concept, + /// A broader/narrower or parent/child taxonomy relation. + TaxonomyRelation, + /// A non-taxonomic semantic relation or object property. + SemanticRelation, + /// A data-quality, cardinality, or semantic constraint. + Constraint, + /// An analytical dimension. + Dimension, + /// A governed analytical measure or metric definition. + Measure, + /// A mapping between a physical source element and a semantic concept. + PhysicalMapping, +} + +/// The epistemic status of a fact or relation. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum TruthStatus { + /// Directly observed from a source without semantic inference. + Observed, + /// Derived by deterministic or model-assisted inference. + Inferred, + /// Explicitly proposed for governance review. + Proposed, + /// Approved and published by the owning governance process. + Authoritative, + /// Previously authoritative but replaced by a newer fact or release. + Superseded, + /// Explicitly rejected by validation or governance review. + Rejected, +} + +/// The governance state of a semantic candidate. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum PublicationState { + /// Newly discovered and not yet submitted for validation. + Draft, + /// Submitted as a candidate for validation. + Proposed, + /// Passed deterministic validation and consistency checks. + Validated, + /// Reviewed by an authorized semantic steward or equivalent workflow. + Reviewed, + /// Published as governed semantic truth. + Published, + /// Replaced by a later published release. + Superseded, + /// Rejected and no longer eligible for publication. + Rejected, +} + +/// A stable reference to the evidence supporting a semantic candidate. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct EvidenceReference { + /// Stable identifier of the observed source snapshot or artifact. + pub source_id: String, + /// Content digest of the exact source revision used as evidence. + pub source_digest: String, + /// Human- and machine-readable location within the source artifact. + pub location: String, +} + +impl EvidenceReference { + /// Creates an evidence reference, rejecting blank identity fields. + pub fn new( + source_id: impl Into, + source_digest: impl Into, + location: impl Into, + ) -> Result { + let source_id = source_id.into(); + let source_digest = source_digest.into(); + let location = location.into(); + if source_id.trim().is_empty() { + return Err(ContractError::EmptyField("source_id")); + } + if source_digest.trim().is_empty() { + return Err(ContractError::EmptyField("source_digest")); + } + if location.trim().is_empty() { + return Err(ContractError::EmptyField("location")); + } + Ok(Self { + source_id, + source_digest, + location, + }) + } +} + +/// A governed candidate for an ontology or semantic-layer artifact. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct SemanticCandidate { + /// Stable candidate identifier within the owning tenant or workspace. + pub candidate_id: String, + /// Kind of semantic artifact proposed by the candidate. + pub kind: CandidateKind, + /// Current epistemic status. + pub truth_status: TruthStatus, + /// Current governance/publication state. + pub publication_state: PublicationState, + /// Exact source evidence supporting the candidate. + pub evidence: Vec, +} + +impl SemanticCandidate { + /// Creates an inferred draft candidate with at least one evidence reference. + pub fn new( + candidate_id: impl Into, + kind: CandidateKind, + evidence: Vec, + ) -> Result { + let candidate_id = candidate_id.into(); + if candidate_id.trim().is_empty() { + return Err(ContractError::EmptyField("candidate_id")); + } + if evidence.is_empty() { + return Err(ContractError::MissingEvidence); + } + let publication_state = PublicationState::Draft; + Ok(Self { + candidate_id, + kind, + truth_status: truth_for_state(publication_state), + publication_state, + evidence, + }) + } + + /// Moves the candidate through the fail-closed governance lifecycle. + pub fn transition(&mut self, target: PublicationState) -> Result<(), ContractError> { + let from = self.publication_state; + if !ALLOWED_TRANSITIONS.contains(&(from, target)) { + return Err(ContractError::InvalidTransition { from, to: target }); + } + self.publication_state = target; + self.truth_status = truth_for_state(target); + Ok(()) + } + + /// Returns whether the candidate is immediately eligible for publication. + pub fn is_publishable(&self) -> bool { + self.publication_state == PublicationState::Reviewed && !self.evidence.is_empty() + } +} + +const ALLOWED_TRANSITIONS: &[(PublicationState, PublicationState)] = &[ + (PublicationState::Draft, PublicationState::Proposed), + (PublicationState::Draft, PublicationState::Rejected), + (PublicationState::Proposed, PublicationState::Validated), + (PublicationState::Proposed, PublicationState::Rejected), + (PublicationState::Validated, PublicationState::Reviewed), + (PublicationState::Validated, PublicationState::Rejected), + (PublicationState::Reviewed, PublicationState::Published), + (PublicationState::Reviewed, PublicationState::Rejected), + (PublicationState::Published, PublicationState::Superseded), +]; + +fn truth_for_state(state: PublicationState) -> TruthStatus { + match state { + PublicationState::Draft | PublicationState::Validated | PublicationState::Reviewed => { + TruthStatus::Inferred + } + PublicationState::Proposed => TruthStatus::Proposed, + PublicationState::Published => TruthStatus::Authoritative, + PublicationState::Superseded => TruthStatus::Superseded, + PublicationState::Rejected => TruthStatus::Rejected, + } +} + +/// A domain-contract validation failure. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ContractError { + /// A required identity or evidence field was blank. + EmptyField(&'static str), + /// A candidate was created without any supporting evidence. + MissingEvidence, + /// A governance state transition attempted to skip or reverse required review. + InvalidTransition { + /// State before the rejected transition. + from: PublicationState, + /// Requested target state. + to: PublicationState, + }, +} + +impl fmt::Display for ContractError { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::EmptyField(field) => write!(formatter, "required field `{field}` is blank"), + Self::MissingEvidence => write!(formatter, "semantic candidates require source evidence"), + Self::InvalidTransition { from, to } => write!( + formatter, + "publication transition from {from:?} to {to:?} is not permitted" + ), + } + } +} + +impl std::error::Error for ContractError {} + +#[cfg(test)] +mod tests { + use super::*; + + fn evidence() -> EvidenceReference { + EvidenceReference::new("source-1", "sha256:abc", "schema.orders.total").unwrap() + } + + fn candidate() -> SemanticCandidate { + SemanticCandidate::new("candidate-1", CandidateKind::Concept, vec![evidence()]).unwrap() + } + + #[test] + fn evidence_reference_accepts_valid_values() { + let reference = evidence(); + assert_eq!(reference.source_id, "source-1"); + assert_eq!(reference.source_digest, "sha256:abc"); + assert_eq!(reference.location, "schema.orders.total"); + } + + #[test] + fn evidence_reference_rejects_each_blank_field() { + assert_eq!( + EvidenceReference::new(" ", "digest", "location"), + Err(ContractError::EmptyField("source_id")) + ); + assert_eq!( + EvidenceReference::new("source", " ", "location"), + Err(ContractError::EmptyField("source_digest")) + ); + assert_eq!( + EvidenceReference::new("source", "digest", " "), + Err(ContractError::EmptyField("location")) + ); + } + + #[test] + fn candidate_requires_identity_and_evidence() { + assert_eq!( + SemanticCandidate::new(" ", CandidateKind::Concept, vec![evidence()]), + Err(ContractError::EmptyField("candidate_id")) + ); + assert_eq!( + SemanticCandidate::new("candidate", CandidateKind::Concept, vec![]), + Err(ContractError::MissingEvidence) + ); + } + + #[test] + fn reviewed_candidate_can_be_published() { + let mut candidate = candidate(); + assert_eq!(candidate.truth_status, TruthStatus::Inferred); + assert!(!candidate.is_publishable()); + + candidate.transition(PublicationState::Proposed).unwrap(); + assert_eq!(candidate.truth_status, TruthStatus::Proposed); + candidate.transition(PublicationState::Validated).unwrap(); + assert_eq!(candidate.truth_status, TruthStatus::Inferred); + candidate.transition(PublicationState::Reviewed).unwrap(); + assert_eq!(candidate.truth_status, TruthStatus::Inferred); + assert!(candidate.is_publishable()); + candidate.transition(PublicationState::Published).unwrap(); + assert_eq!(candidate.truth_status, TruthStatus::Authoritative); + assert!(!candidate.is_publishable()); + } + + #[test] + fn lifecycle_rejects_skipped_and_post_rejection_transitions() { + let mut candidate = candidate(); + assert_eq!( + candidate.transition(PublicationState::Published), + Err(ContractError::InvalidTransition { + from: PublicationState::Draft, + to: PublicationState::Published, + }) + ); + candidate.transition(PublicationState::Rejected).unwrap(); + assert_eq!(candidate.truth_status, TruthStatus::Rejected); + assert_eq!( + candidate.transition(PublicationState::Proposed), + Err(ContractError::InvalidTransition { + from: PublicationState::Rejected, + to: PublicationState::Proposed, + }) + ); + } + + #[test] + fn published_candidate_can_only_be_superseded() { + let mut candidate = candidate(); + for state in [ + PublicationState::Proposed, + PublicationState::Validated, + PublicationState::Reviewed, + PublicationState::Published, + ] { + candidate.transition(state).unwrap(); + } + candidate.transition(PublicationState::Superseded).unwrap(); + assert_eq!(candidate.truth_status, TruthStatus::Superseded); + assert_eq!( + candidate.transition(PublicationState::Rejected), + Err(ContractError::InvalidTransition { + from: PublicationState::Superseded, + to: PublicationState::Rejected, + }) + ); + } + + #[test] + fn contract_errors_explain_the_failure() { + assert_eq!( + ContractError::EmptyField("field").to_string(), + "required field `field` is blank" + ); + assert_eq!( + ContractError::MissingEvidence.to_string(), + "semantic candidates require source evidence" + ); + assert_eq!( + ContractError::InvalidTransition { + from: PublicationState::Draft, + to: PublicationState::Published, + } + .to_string(), + "publication transition from Draft to Published is not permitted" + ); + } +} diff --git a/docs/CONTEXT_MAP.md b/docs/CONTEXT_MAP.md new file mode 100644 index 00000000..5ea47792 --- /dev/null +++ b/docs/CONTEXT_MAP.md @@ -0,0 +1,16 @@ +# Context Map + +## Internal relationships + +- Source Observation -> Semantic Discovery: **Customer/Supplier**; Discovery consumes immutable observation contracts. +- Semantic Discovery -> Model Validation: **Conformist to published candidate contract**; validation must not rewrite discovery evidence. +- Model Validation -> Governance & Publication: **Customer/Supplier**; governance consumes deterministic validation receipts. +- Governance & Publication -> Interoperability: **Published Language**; adapters consume immutable release contracts. + +## External relationships + +- contextual-orchestrator -> Semantic Discovery: **Anti-Corruption Layer**. Model/provider envelopes never enter the domain model directly. +- LineageWeave -> Source Observation: **Anti-Corruption Layer**. Inferred/proposed lineage remains explicitly non-authoritative until ConceptWeave governance evaluates it. +- context-graph-contracts <-> Interoperability: **Shared Kernel only for versioned public contracts**, kept minimal. +- semantic-data-portal <- Interoperability: **Published Language**. SDP consumes releases; ConceptWeave does not read SDP application tables. +- Keyverse -> future delivery layer: **Anti-Corruption Layer** for verified identity/tenant context. diff --git a/docs/PRD.md b/docs/PRD.md new file mode 100644 index 00000000..28465202 --- /dev/null +++ b/docs/PRD.md @@ -0,0 +1,81 @@ +# ConceptWeave Product Requirements Document + +## 1. Product statement + +ConceptWeave converts heterogeneous enterprise evidence into a governed ontology and semantic layer without collapsing observed facts, model inference, and human-approved meaning into the same truth state. + +## 2. Buyer problem + +Enterprise teams repeatedly hand-build business glossaries, ontologies, metric definitions, semantic mappings, and data relationships from database schemas, API contracts, documents, and tribal knowledge. The work is slow, inconsistent across tools, difficult to audit, and unsafe to delegate entirely to an LLM because inferred semantics can be plausible but wrong. + +## 3. Primary buyers and users + +- enterprise data architects and semantic-modeling teams; +- data governance and catalog teams; +- analytics/BI platform owners; +- AI/RAG platform teams that require governed machine-readable context; +- risk/compliance and enterprise-architecture teams that need traceable semantic models. + +## 4. Core job to be done + +Given an enterprise source estate, produce a **reviewable semantic model proposal** in which every concept, relationship, constraint, dimension, measure, and physical mapping is linked to exact evidence and can be validated, rejected, reviewed, published, superseded, and reproduced. + +## 5. Functional requirements + +### FR-1 Source observation + +Accept immutable snapshots or versioned contracts for relational schema, OpenAPI, AsyncAPI/event models, documents/glossaries, source-code structure, existing ontology/vocabulary, and lineage/provenance. Raw source authority remains with its owning system. + +### FR-2 Candidate discovery + +Produce candidates for concepts, taxonomies, non-taxonomic relations, semantic constraints, dimensions, measures, and physical-to-semantic mappings. Each candidate starts as inferred rather than authoritative. + +### FR-3 Evidence and provenance + +Every candidate must retain source identity, digest, location, observation time, parser/extractor revision, and discovery method. Unsupported candidates fail closed. + +### FR-4 Deterministic validation + +Validate syntax, identifiers, relationship cardinality, mapping completeness, duplicate/contradictory definitions, ontology consistency where supported, semantic-measure contracts, and publication schema before review. + +### FR-5 Governed review + +A candidate cannot become authoritative solely because an LLM or automated extractor produced it. The publication lifecycle is Draft -> Proposed -> Validated -> Reviewed -> Published, with explicit rejection and supersession paths. + +### FR-6 Publication + +Publish versioned artifacts for ontology and semantic-layer consumers while retaining the exact input snapshot and proposal/review receipts that produced the release. + +### FR-7 Interoperability + +Support stable adapters for `semantic-data-portal`, `LineageWeave`, `context-graph-contracts`, and other CWL products without direct cross-service application-table SQL. + +### FR-8 LLM assistance + +All LLM-backed induction uses `contextual-orchestrator`. Model output is untrusted proposal data and may not skip deterministic validation or review. + +## 6. First vertical slice + +Relational schema snapshot -> observed tables/columns/foreign keys -> concept/relation/dimension/measure/mapping candidates -> evidence-bound validation report -> reviewable proposal package. + +## 7. Non-goals for v0.1 + +- replacing `semantic-data-portal` as the enterprise catalog; +- arbitrary write access to source systems; +- automatic publication without review; +- treating vector similarity as semantic truth; +- copying every external ontology into one CWL namespace; +- building a generic LLM gateway or browser crawler; +- claiming an emerging draft semantic-layer format is a stable standard. + +## 8. Acceptance criteria for the first commercial candidate + +- 100% owned production line/function/region and branch coverage where tooling exposes it; +- candidate-to-source provenance completeness of 100%; +- zero publication paths that bypass reviewed state; +- zero silent inferred-to-authoritative promotion; +- deterministic replay of the same immutable source snapshot and extraction configuration; +- cross-tenant access denial when tenancy is introduced; +- malformed/hostile source contracts rejected with bounded resource use; +- semantic-model release can be reproduced from source receipts and approved proposal receipts; +- buyer can inspect why each published artifact exists and which evidence supported it. diff --git a/docs/TRD.md b/docs/TRD.md new file mode 100644 index 00000000..ad5e1415 --- /dev/null +++ b/docs/TRD.md @@ -0,0 +1,58 @@ +# ConceptWeave Technical Requirements Document + +## 1. Architectural style + +ConceptWeave starts as a Rust-first modular monolith with explicit bounded contexts and ports. Network-service extraction is deferred until independent scaling, trust, or deployment boundaries are demonstrated. + +## 2. Bounded contexts + +1. **Source Observation** — immutable source snapshots and parser receipts. +2. **Semantic Discovery** — evidence-bound candidate generation. +3. **Model Validation** — deterministic structural, ontology, constraint, and semantic-model validation. +4. **Governance & Publication** — review decisions, immutable releases, supersession. +5. **Interoperability** — import/export adapters and CWL anti-corruption layers. + +The Core Domain is **Semantic Model Engineering**, represented by the discovery-to-publication lifecycle. Identity, LLM routing, outbound web access, observability, and catalog consumption are external/generic responsibilities. + +## 3. Dependency direction + +`domain <- application <- ports/contracts <- adapters <- delivery` + +Domain code must not import web frameworks, databases, provider SDKs, LLM SDKs, or another CWL product's internals. + +## 4. Source observation contract + +Every observed source will eventually carry at least: + +- source snapshot identifier; +- source kind; +- immutable content digest; +- source authority; +- observed/recorded time; +- parser/extractor version; +- tenant/workspace scope when tenancy exists; +- bounded source locations for extracted evidence. + +## 5. Candidate contract + +The initial Rust and JSON contracts cover candidate kind, truth status, publication state, and source evidence. Later revisions add ontology IRIs, language-tagged labels, relation endpoints, cardinality, units, measure expressions, physical mappings, confidence/evaluation receipts, and temporal validity without breaking v0.1 consumers. + +## 6. LLM boundary + +LLM calls go through `contextual-orchestrator`. The application sends bounded evidence/context and receives structured proposals. LLM output is never a database command, publication decision, validation result, or source-system mutation. Deterministic checks must be able to reject the output without another model call. + +## 7. Standards strategy + +Stable publication targets use stable recommendations first: RDF 1.1, OWL 2, SKOS, SHACL 1.0, JSON-LD 1.1, and PROV-O as applicable. RDF 1.2 and SHACL 1.2 are tracked as 2026 drafts/candidate work and are not silently treated as final standards. Apache Ossie (incubating; formerly OSI) is tracked as an emerging semantic-model exchange format for metrics, dimensions, relationships, and datasets. + +## 8. Persistence + +No durable product database is claimed by the foundation slice. When persistence is introduced it must be PostgreSQL, 3NF by default, use descriptive two-or-more-word `snake_case` objects, preserve business/effective time separately from system-recorded time when facts vary over time, enforce tenant-scoped references, and use explicit migration ownership rather than runtime DDL races. + +## 9. Security + +Source artifacts are untrusted input. Adapters must enforce source size/type bounds, parser timeouts, archive/decompression limits, SSRF-safe outbound access where external retrieval exists, and prompt-injection isolation for LLM-assisted extraction. Credentials and raw secrets never become semantic evidence. + +## 10. Evaluation + +Evaluation must separate extraction recall, semantic correctness, structural correctness, ontology consistency, mapping accuracy, measure correctness, and governance outcomes. Model-judge scores may supplement but never replace deterministic golden fixtures and human-reviewed expert cases. diff --git a/docs/UBIQUITOUS_LANGUAGE.md b/docs/UBIQUITOUS_LANGUAGE.md new file mode 100644 index 00000000..0c1d0c25 --- /dev/null +++ b/docs/UBIQUITOUS_LANGUAGE.md @@ -0,0 +1,18 @@ +# Ubiquitous Language + +| Term | Meaning | +| --- | --- | +| Source Snapshot | Immutable revision of source evidence observed by ConceptWeave. | +| Observation | Deterministically extracted fact from a Source Snapshot. | +| Evidence Reference | Stable source identity, digest, and location supporting a candidate. | +| Semantic Candidate | Evidence-bound proposal for a concept, relation, constraint, dimension, measure, or physical mapping. | +| Semantic Model Proposal | Versioned collection of candidates presented for validation/review. | +| Validation Report | Deterministic result describing structural or semantic contract validity; not a review decision. | +| Review Decision | Authorized accept/reject decision over validated candidates or a model proposal. | +| Semantic Model Release | Immutable governed publication artifact. | +| Truth Status | Epistemic classification: observed, inferred, proposed, authoritative, superseded, rejected. | +| Publication State | Governance workflow state: draft, proposed, validated, reviewed, published, superseded, rejected. | +| Physical Mapping | Mapping from a physical schema/API/event element to a semantic concept or field. | +| Dimension | Governed categorical or temporal axis used to group/filter analytical facts. | +| Measure | Governed calculation with explicit expression, grain, units, null semantics, and evidence. | +| Semantic Steward | Authorized reviewer responsible for accepting or rejecting semantic meaning. | diff --git a/docs/UML.md b/docs/UML.md new file mode 100644 index 00000000..a9559e7f --- /dev/null +++ b/docs/UML.md @@ -0,0 +1,40 @@ +# UML and lifecycle views + +## Candidate state machine + +```mermaid +stateDiagram-v2 + [*] --> Draft + Draft --> Proposed + Draft --> Rejected + Proposed --> Validated + Proposed --> Rejected + Validated --> Reviewed + Validated --> Rejected + Reviewed --> Published + Reviewed --> Rejected + Published --> Superseded + Rejected --> [*] + Superseded --> [*] +``` + +## Foundation sequence + +```mermaid +sequenceDiagram + participant Source + participant Observation + participant Discovery + participant Validator + participant Steward + participant Publisher + + Source->>Observation: immutable snapshot + Observation->>Discovery: observations + evidence refs + Discovery->>Validator: inferred candidate proposal + Validator-->>Discovery: validation report + Validator->>Steward: validated proposal + Steward->>Publisher: reviewed acceptance + Publisher-->>Source: no source mutation + Publisher-->>Steward: immutable release receipt +``` diff --git a/docs/adr/0001-product-boundary.md b/docs/adr/0001-product-boundary.md new file mode 100644 index 00000000..e4a0d6c4 --- /dev/null +++ b/docs/adr/0001-product-boundary.md @@ -0,0 +1,22 @@ +# ADR 0001 — Product and bounded-context boundary + +**Status:** Accepted + +## Context + +CWL already has products that reconstruct lineage, operate semantic catalogs, define shared graph contracts, and route LLM calls. Placing automatic semantic-model engineering inside any one of those products would blur system-of-record and reuse boundaries. + +## Decision + +ConceptWeave owns **Semantic Model Engineering**: observing source evidence, discovering semantic candidates, validating them, governing review, and publishing versioned ontology/semantic-layer releases. + +`semantic-data-portal` remains the consumer/catalog/governance plane for published semantic context. `LineageWeave` remains an inference/lineage evidence producer. `context-graph-contracts` remains a contract-only interoperability repository. `contextual-orchestrator` remains the LLM routing boundary. + +External code-analysis or graph-generation tools may be optional source adapters, but external forks are not ConceptWeave product authority or required internal dependencies. + +## Consequences + +- ConceptWeave can be used by GRC, EA, analytics, HR, billing, or other products without copying their authoritative data. +- No direct cross-service application-table SQL. +- Source systems remain authoritative for source facts. +- Published semantic releases are authoritative only within their explicitly reviewed semantic scope. diff --git a/docs/adr/0002-truth-publication-lifecycle.md b/docs/adr/0002-truth-publication-lifecycle.md new file mode 100644 index 00000000..ac35e119 --- /dev/null +++ b/docs/adr/0002-truth-publication-lifecycle.md @@ -0,0 +1,23 @@ +# ADR 0002 — Evidence, truth, and publication lifecycle + +**Status:** Accepted + +## Context + +Automatic ontology learning and LLM-assisted semantic modeling can produce plausible but incorrect concepts and relations. A single `confidence` number cannot establish authority. + +## Decision + +Separate epistemic truth status from governance publication state. Every semantic candidate requires exact source evidence. New candidates are inferred drafts. Publication requires the ordered lifecycle: + +`Draft -> Proposed -> Validated -> Reviewed -> Published` + +At Draft/Proposed/Validated/Reviewed stages the artifact is not authoritative. `Published` changes semantic truth to authoritative within the release scope. Rejection is explicit. Published facts are never overwritten; a replacement creates a new release and marks the old release/candidate superseded. + +No LLM, embedding similarity, graph centrality, or automated extractor may directly create authoritative semantic truth. + +## Consequences + +- Review and validation receipts become first-class future persistence objects. +- Replay can reconstruct why a semantic release exists. +- Consumers can filter `authoritative`, `observed`, `inferred`, and `proposed` data without conflating them. diff --git a/docs/adr/0003-standards-llm-boundary.md b/docs/adr/0003-standards-llm-boundary.md new file mode 100644 index 00000000..8a56cfa9 --- /dev/null +++ b/docs/adr/0003-standards-llm-boundary.md @@ -0,0 +1,19 @@ +# ADR 0003 — Standards and LLM engineering boundary + +**Status:** Accepted + +## Context + +ConceptWeave must publish portable semantic artifacts while standards evolve and LLM-based ontology engineering remains an active research area. + +## Decision + +1. Stable ontology publication targets are RDF 1.1, OWL 2, SKOS, SHACL 1.0, JSON-LD 1.1, and PROV-O as applicable. +2. RDF 1.2 and SHACL 1.2 are tracked as 2026 W3C in-progress work and may be implemented behind explicit experimental/versioned adapters; they are not labeled final standards. +3. Apache Ossie (incubating, formerly Open Semantic Interchange) is tracked as an emerging vendor-neutral semantic-model exchange format for datasets, fields/dimensions, relationships, and metrics. Any adapter is explicitly version-bound until the required specification subset is stable. +4. LLM-backed ontology learning, matching, labeling, and candidate generation must use `contextual-orchestrator` and produce structured proposals with evidence. LLMs do not validate, approve, or publish semantic truth. +5. Evaluation combines deterministic conformance/consistency checks, benchmark fixtures, and human-reviewed cases. Model-as-judge evidence is supplementary only. + +## Consequences + +ConceptWeave is standards-oriented without falsely promoting drafts to Recommendations, and model assistance can improve recall while governance remains fail-closed. diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 00000000..291702a3 --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,5 @@ +# Architecture Decision Records + +- [ADR 0001 — Product and bounded-context boundary](0001-product-boundary.md) +- [ADR 0002 — Evidence, truth, and publication lifecycle](0002-truth-publication-lifecycle.md) +- [ADR 0003 — Standards and LLM engineering boundary](0003-standards-llm-boundary.md) diff --git a/docs/doctoring/REFERENCES.md b/docs/doctoring/REFERENCES.md new file mode 100644 index 00000000..f42a762c --- /dev/null +++ b/docs/doctoring/REFERENCES.md @@ -0,0 +1,42 @@ +# Standards and Research References + +This file records the initial evidence basis for ConceptWeave architecture decisions. Stable Recommendations and in-progress specifications are deliberately distinguished. + +## Stable standards / recommendations + +Miles, A., & Bechhofer, S. (Eds.). (2009). *SKOS Simple Knowledge Organization System Reference*. World Wide Web Consortium. https://www.w3.org/TR/skos-reference/ + +W3C OWL Working Group. (2012). *OWL 2 Web Ontology Language document overview (Second Edition)*. World Wide Web Consortium. https://www.w3.org/TR/owl2-overview/ + +W3C Provenance Working Group. (2013). *PROV-O: The PROV Ontology*. World Wide Web Consortium. https://www.w3.org/TR/prov-o/ + +W3C RDF Working Group. (2014). *RDF 1.1 concepts and abstract syntax*. World Wide Web Consortium. https://www.w3.org/TR/rdf11-concepts/ + +W3C Data Shapes Working Group. (2017). *Shapes Constraint Language (SHACL)*. World Wide Web Consortium. https://www.w3.org/TR/shacl/ + +World Wide Web Consortium. (2020). *JSON-LD 1.1*. https://www.w3.org/TR/json-ld11/ + +## In-progress / emerging specifications tracked, not claimed as final standards + +W3C RDF-star Working Group. (2026). *RDF 1.2 concepts and abstract data model* (Candidate Recommendation Snapshot, April 7, 2026). World Wide Web Consortium. https://www.w3.org/TR/rdf12-concepts/ + +W3C Data Shapes Working Group. (2026). *SHACL 1.2 Core* (Working Draft, August 3, 2026). World Wide Web Consortium. https://www.w3.org/TR/shacl12-core/ + +Apache Software Foundation. (2026). *Apache Ossie (incubating)*. https://ossie.apache.org/ +Formerly Open Semantic Interchange (OSI); tracked as an emerging vendor-neutral semantic-model exchange specification rather than a W3C/ISO standard. + +## Research basis + +Babaei Giglou, H., D'Souza, J., & Auer, S. (2023). LLMs4OL: Large language models for ontology learning. In *The Semantic Web – ISWC 2023* (pp. 408–427). Springer. https://doi.org/10.1007/978-3-031-47240-4_22 + +Hertling, S., & Paulheim, H. (2023). OLaLa: Ontology matching with large language models. In *Proceedings of the 12th Knowledge Capture Conference 2023* (pp. 131–139). Association for Computing Machinery. https://doi.org/10.1145/3587259.3627571 + +Lo, A., Jiang, A. Q., Li, W., & Jamnik, M. (2024). *End-to-end ontology learning with large language models* [Preprint]. arXiv. https://arxiv.org/abs/2410.23584 + +Li, J., Garijo, D., & Poveda-Villalón, M. (2026). Large language models for ontology engineering: A systematic literature review. *Semantic Web, 17*(4), 1–45. https://doi.org/10.1177/22104968261465514 + +## Decision implications + +- LLMs can assist ontology learning, matching, modeling, and maintenance, but current research does not justify automatic authority promotion. +- Evaluation practices across LLM ontology engineering are still heterogeneous; ConceptWeave therefore requires reproducible deterministic checks plus human-reviewed benchmark cases. +- Modular ontology engineering and explicit source provenance are preferred over one opaque prompt that attempts to generate an entire enterprise semantic layer in a single step. diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md new file mode 100644 index 00000000..734b43ee --- /dev/null +++ b/docs/product-technical-gap-baseline.md @@ -0,0 +1,41 @@ +# Product / Technical Gap Baseline + +**Snapshot:** 2026-09-01 + +## Shipped on protected `main` + +Only the repository bootstrap README exists before the foundation PR. No production capability is claimed. + +## Active foundation slice + +| Area | Status | Evidence / next action | +| --- | --- | --- | +| Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define Semantic Model Engineering and CWL boundaries. | +| Truth/publication lifecycle | ACTIVE_PR | Rust domain contract + JSON Schema enforce evidence and reviewed-before-published lifecycle. | +| Rust baseline | ACTIVE_PR | Rust 1.98.0 workspace, unsafe forbidden, public docs required. | +| Quality gate | ACTIVE_PR | Product workflow for fmt/clippy/tests/docs/coverage/schema/lock/clean-tree. | +| Standards/research | ACTIVE_PR | Stable-vs-draft standards and ontology-engineering research doctoring. | +| Security/test/operability | ACTIVE_PR | Baselines added; no production service claimed. | + +## P0 product gaps after foundation + +1. **Source Observation vertical** — relational schema snapshot contract, real PostgreSQL introspection adapter, immutable digest/location receipts, hostile-input bounds. +2. **Ontology induction** — deterministic observations plus contextual-orchestrator structured candidate generation for concepts, taxonomy, and non-taxonomic relations. +3. **Semantic-layer induction** — dimensions, measures, grain, units, relationships, and physical mappings with deterministic calculation contracts. +4. **Validation engine** — RDF/OWL/SKOS/SHACL publication validation, consistency checks, duplicate/conflict detection, bounded reasoning. +5. **Governance persistence** — PostgreSQL 3NF candidates, evidence, validation receipts, review decisions, releases, transactional outbox, bitemporal history where applicable. +6. **Review workflow** — Keyverse tenant/role/purpose context, steward review, maker-checker where required, stale decision protection, immutable publication receipt. +7. **Publication adapters** — OWL/RDFS/SKOS/SHACL/JSON-LD and version-bound Apache Ossie semantic-model export. +8. **CWL integration** — `semantic-data-portal`, `LineageWeave`, `context-graph-contracts`, GRC, and EA through published contracts only. +9. **Evaluation harness** — ontology-learning/matching golden fixtures, structural/semantic metrics, human-reviewed cases, replay reproducibility, multilingual cases. +10. **Secure external research** — SearXNG discovery and safe source fetch through the correct CWL egress boundary for ontology grounding, never search snippets as truth. +11. **Observability** — shared CWL OpenTelemetry import/bootstrap contract, detailed structured logs, SIEM security-event projection where applicable. +12. **Release** — SBOM, provenance, signed artifacts, migration/backup/restore evidence, versioned changelog, protected release pipeline. + +## DDD fitness gaps + +- No generic `utils/helpers/services/common` domain buckets are permitted. +- Adapters must remain outside `conceptweave-domain`. +- Foreign product DTOs require Anti-Corruption Layers. +- `semantic-data-portal` must not become ConceptWeave persistence, and ConceptWeave must not become an SDP clone. +- External forks/tools can be optional adapters but are not CWL-owned product authorities. diff --git a/rust-toolchain.toml b/rust-toolchain.toml new file mode 100644 index 00000000..45626cc0 --- /dev/null +++ b/rust-toolchain.toml @@ -0,0 +1,4 @@ +[toolchain] +channel = "1.98.0" +profile = "minimal" +components = ["clippy", "llvm-tools-preview", "rustfmt"] diff --git a/scripts/check_coverage.sh b/scripts/check_coverage.sh new file mode 100755 index 00000000..8bba62e0 --- /dev/null +++ b/scripts/check_coverage.sh @@ -0,0 +1,11 @@ +#!/usr/bin/env bash +set -euo pipefail + +cargo llvm-cov --workspace --branch --json --output-path coverage.json + +jq -e ' + .data[0].totals.lines.percent == 100 and + .data[0].totals.functions.percent == 100 and + .data[0].totals.regions.percent == 100 and + .data[0].totals.branches.percent == 100 +' coverage.json >/dev/null From 0081a67e48dc517270f50d32e4e241033222bc1c Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 15:23:19 +0900 Subject: [PATCH 02/59] fix: bind Product CI to exact source head --- .github/workflows/product.yml | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index 4caa8a1e..6b52b599 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -12,8 +12,17 @@ jobs: rust-quality: runs-on: ubuntu-latest steps: - - name: Checkout exact revision + - name: Checkout exact source revision uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + fetch-depth: 0 + persist-credentials: false + + - name: Verify exact source revision + env: + EXPECTED_SHA: ${{ github.event.pull_request.head.sha || github.sha }} + run: test "$(git rev-parse HEAD)" = "$EXPECTED_SHA" - name: Show pinned Rust toolchain run: rustc --version && cargo --version From 8135395d096e1cf038a40ca13db16181efeb716a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 15:25:02 +0900 Subject: [PATCH 03/59] test: cover evidence publication invariant --- .../tests/publication_invariants.rs | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 crates/conceptweave-domain/tests/publication_invariants.rs diff --git a/crates/conceptweave-domain/tests/publication_invariants.rs b/crates/conceptweave-domain/tests/publication_invariants.rs new file mode 100644 index 00000000..528c8f34 --- /dev/null +++ b/crates/conceptweave-domain/tests/publication_invariants.rs @@ -0,0 +1,19 @@ +use conceptweave_domain::{CandidateKind, EvidenceReference, PublicationState, SemanticCandidate}; + +#[test] +fn reviewed_candidate_without_evidence_is_not_publishable() { + let evidence = EvidenceReference::new("source-1", "sha256:abc", "public.orders").unwrap(); + let mut candidate = + SemanticCandidate::new("candidate-1", CandidateKind::Concept, vec![evidence]).unwrap(); + + for state in [ + PublicationState::Proposed, + PublicationState::Validated, + PublicationState::Reviewed, + ] { + candidate.transition(state).unwrap(); + } + + candidate.evidence.clear(); + assert!(!candidate.is_publishable()); +} From 8f3253371b4770571966a52e26db274b58d61585 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:47:59 +0900 Subject: [PATCH 04/59] test: reject publication after evidence loss --- .../tests/publication_invariants.rs | 25 ++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/crates/conceptweave-domain/tests/publication_invariants.rs b/crates/conceptweave-domain/tests/publication_invariants.rs index 528c8f34..8e9c0eb6 100644 --- a/crates/conceptweave-domain/tests/publication_invariants.rs +++ b/crates/conceptweave-domain/tests/publication_invariants.rs @@ -1,7 +1,8 @@ -use conceptweave_domain::{CandidateKind, EvidenceReference, PublicationState, SemanticCandidate}; +use conceptweave_domain::{ + CandidateKind, ContractError, EvidenceReference, PublicationState, SemanticCandidate, +}; -#[test] -fn reviewed_candidate_without_evidence_is_not_publishable() { +fn reviewed_candidate() -> SemanticCandidate { let evidence = EvidenceReference::new("source-1", "sha256:abc", "public.orders").unwrap(); let mut candidate = SemanticCandidate::new("candidate-1", CandidateKind::Concept, vec![evidence]).unwrap(); @@ -13,7 +14,25 @@ fn reviewed_candidate_without_evidence_is_not_publishable() { ] { candidate.transition(state).unwrap(); } + candidate +} +#[test] +fn reviewed_candidate_without_evidence_is_not_publishable() { + let mut candidate = reviewed_candidate(); candidate.evidence.clear(); + assert!(!candidate.is_publishable()); } + +#[test] +fn reviewed_candidate_without_evidence_cannot_transition_to_published() { + let mut candidate = reviewed_candidate(); + candidate.evidence.clear(); + + assert_eq!( + candidate.transition(PublicationState::Published), + Err(ContractError::MissingEvidence) + ); + assert_eq!(candidate.publication_state, PublicationState::Reviewed); +} From 0d66b89d05e5477ad4f9629f4747ff44a1f76e33 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:51:29 +0900 Subject: [PATCH 05/59] docs: map Consensus research to product capabilities --- .../RESEARCH_CAPABILITY_TRACEABILITY.md | 93 +++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md diff --git a/docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md b/docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md new file mode 100644 index 00000000..ea21e56c --- /dev/null +++ b/docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md @@ -0,0 +1,93 @@ +# Research-to-capability traceability + +Snapshot: 2026-09-01 + +This register turns the current Consensus search set into product decisions. A paper is not considered "used" merely because it appears in the bibliography: it must be tied to a bounded context, an accepted or rejected design implication, and an executable evaluation family. Publication metadata below follows the current Consensus record; preprints are not silently promoted to peer-reviewed evidence. + +## Classification rules + +- `generation`: Source Observation, Semantic Discovery, or LLM Proposal capabilities that create ontology/semantic-model candidates. +- `client`: Model Alignment or Client Consumption capabilities that match, resolve, compare, explain, or consume governed releases. +- `bridge`: connects generated ontology/schema mappings to a downstream virtual/semantic consumption model. +- `cross_cutting`: evaluation, versioning, governance, reproducibility, or human-validation evidence shared by both tracks. +- `adopt`: becomes a product/test requirement. +- `adapt`: informs the design but is constrained by ConceptWeave authority/security contracts. +- `research_only`: retained as evidence but not adopted as a production rule. + +## Current research register + +| Study | Class | Product implication | Decision | Implementation owner | Evaluation / test family | Limitation carried into ConceptWeave | +| --- | --- | --- | --- | --- | --- | --- | +| Shimizu & Hitzler (2024), *Accelerating Knowledge Graph and Ontology Engineering with Large Language Models* | generation / cross_cutting | Keep ontology engineering modular: modeling, extension, population, alignment, disambiguation are separate operations rather than one opaque prompt. | adopt | Semantic Discovery, LLM Proposal, Model Alignment | per-operation fixtures; orchestration receipts; module-boundary fitness | Consensus currently records an arXiv publication; use as architecture evidence, not sole production truth. | +| Trajanoska, Stojanov, & Trajanov (2023), *Enhancing Knowledge Graph Construction Using Large Language Models* | generation | Compare LLM-assisted entity/relation extraction and ontology proposals with deterministic/specialized baselines. | adapt | Semantic Discovery, LLM Proposal | entity/relation precision-recall-F1; KG/ontology relevance; abstention | Demonstration domain is not enterprise GRC; no automatic generalization of reported accuracy. | +| Val-Calvo et al. (2025), *OntoGenix* | generation | Treat dataset-to-ontology work as staged preprocessing -> planning -> building -> refinement -> mapping, with explicit failure on complex modeling. | adopt | Source Observation, Semantic Discovery, Model Validation | stage receipts; coherent-model fixtures; complex-model abstention | Human modeling remains stronger for complex cases; model output stays proposed. | +| Lo, Jiang, Li, & Jamnik (2024), *End-to-End Ontology Learning with Large Language Models* | generation | Include end-to-end taxonomy generation as a benchmark strategy, not as the sole architecture. Measure semantic and structural similarity. | adapt | Semantic Discovery, Evaluation | taxonomy-edge P/R/F1; graph structural similarity; domain-transfer fixture | Consensus currently records arXiv; fine-tuned end-to-end generation does not remove governance review. | +| Giglou, D'Souza, & Auer (2023), *LLMs4OL* | generation | Term typing, taxonomy discovery, and non-taxonomic relation extraction are first-class Generation tasks. | adopt | Semantic Discovery, LLM Proposal | LLMs4OL-style task fixtures across heterogeneous domains | Zero-shot results vary by domain; no model family becomes canonical product truth. | +| Giglou, D'Souza, & Auer (2024), *LLMs4OL 2024 Overview* | generation / cross_cutting | Preserve challenge-style standardized task definitions and reusable benchmark splits. | adopt | Evaluation | challenge-compatible term typing / taxonomy / relation suites | Challenge evidence measures tasks, not enterprise governance or source authority. | +| Phuttaamart, Kertkeidkachorn, & Trongratsameethong (2024), *The Ghost at LLMs4OL 2024 Task A* | generation | Track prompt/prompt-tuning sensitivity explicitly for term typing. | adapt | LLM Proposal, Evaluation | term-typing per-domain accuracy/F1; prompt sensitivity | GeoNames degradation is a concrete warning against aggregate-only scores. | +| Zhang et al. (2025), *OLIVE: Ontology Learning With Integrated Vector Embeddings* | generation | Vector/LLM workflows may assist relationship discovery and OWL drafting, but vectors remain candidate evidence rather than semantic authority. | adapt | Semantic Discovery, publication adapters | candidate quality; OWL syntax/shape validation | Prompt-driven retrieval and vector similarity cannot define truth status. | +| Lippolis et al. (2025), *Ontology Generation using Large Language Models* | generation / cross_cutting | Competency questions and user stories can drive ontology drafts; assess multiple structural criteria plus expert qualitative review. | adopt | LLM Proposal, Model Validation | competency-question coverage; structural criteria; expert/steward edit distance | Reported quality varies by model/prompt; generated OWL remains draft/proposed. | +| Giglou et al. (2026), *OntoLearner* | generation / cross_cutting | Add cross-domain standardized benchmarking and measure failure against ontology complexity, not just model size. | adopt | Evaluation | multi-domain term/taxonomy/relation benchmark; complexity-stratified error analysis | Tool/library is research infrastructure, not a required runtime dependency. | +| Hertling & Paulheim (2023), *OLaLa* | client | Ontology matching needs explicit prompt representation, examples, existing correspondences, and candidate-generation choices. | adopt | Model Alignment | OAEI-style matching P/R/F1; zero/few-shot comparison | LLM result is a correspondence candidate, never automatic authoritative alignment. | +| Giglou, D'Souza, Engel, & Auer (2024), *LLMs4OM* | client | Retrieve first, then match; compare concept-only, parent-context, and child-context representations. | adopt | Model Alignment, Client Consumption | retrieval recall; matching P/R/F1 across representation variants | Consensus currently records arXiv; client must remain functional without LLM matching. | +| Sousa, Lima, & Trojahn (2025), *Complex Ontology Matching with Large Language Model Embeddings* | client | Support expressive correspondence proposals using local subgraph/neighborhood evidence, not label similarity alone. | adapt | Model Alignment | complex-correspondence F1; subgraph ablations | Embedding-space/model compatibility must be explicit; reported gains do not authorize cross-model vector comparison. | +| Taboada et al. (2025), MILA | client | Use programmed retrieval/search to prune candidates and reserve LLM calls for uncertain cases. | adopt | Model Alignment, Client Consumption | candidate recall; final P/R/F1; LLM-call reduction vs naive prompting | Consensus currently records arXiv; algorithmic search cannot bypass evidence/truth-state rules. | +| Barcelos, French, & Wu (2025), *KROMA* | client | Targeted knowledge retrieval, structural context, and refinement should precede context-augmented LLM matching. | adopt | Model Alignment | candidate pruning recall; prompt-enrichment ablation; communication cost | Consensus currently records arXiv; RAG context is not source authority. | +| Qiang, Wang, & Taylor (2023), *Agent-OM* | client | Separate retrieval and matching responsibilities and expose bounded matching tools rather than one monolithic agent prompt. | adapt | Model Alignment, contextual-orchestrator ACL | OAEI simple/complex/few-shot tracks; tool-call receipts | Agent autonomy does not include release publication or business authorization. | +| Song, Chen, & Schmidt (2025), *GenOM* | client | Generated concept descriptions can enrich retrieval/matching, but exact lexical evidence remains a useful deterministic precision signal. | adopt | Model Alignment | OAEI Bio-ML; definition-quality criteria; retrieval/matching ablations | Biomedical results require enterprise-domain replication before broader claims. | +| Qiang & Taylor (2024), *OM4OV* | client / cross_cutting | Ontology/release version comparison needs explicit update-entity detection and explanations; do not equate versioning with ordinary matching. | adopt | Release Contract, Client Consumption | release-diff correctness; added/removed/changed entity detection; false-match explanation | Consensus currently records arXiv; ConceptWeave needs its own compatibility semantics. | +| Qiang et al. (2024), *OAEI-LLM* | client / cross_cutting | LLM-specific ontology-matching hallucinations require a dedicated benchmark dimension. | adopt | Evaluation | OAEI-LLM hallucination categories; abstention quality | Benchmark does not replace enterprise GRC golden fixtures. | +| Qiang et al. (2025), *OAEI-LLM-T* | client / cross_cutting | Add TBox/schema hallucination tests for matching and alignment. | adopt | Evaluation | TBox hallucination leaderboard/categories | Duplicate preprint/proceedings variants count as one study in this register. | +| Qiang, Wang, & Taylor (2026), *Crowd-OM* | cross_cutting | Human validation quality needs explicit trust/coherence/history controls when review scales beyond one steward. | adapt | Governance & Publication | inter-reviewer disagreement; coherence; adjudication receipts | Crowdsourcing is optional; domain-owner/steward authority remains product policy. | +| Qiang, Taylor, & Wang (2024), *How Does A Text Preprocessing Pipeline Affect Ontology Syntactic Matching?* | client | Keep deterministic tokenization/normalization as inspectable evidence; avoid assuming stopword/stemming pipelines always improve matching; LLM repair is secondary. | adopt | Model Alignment | OAEI preprocessing ablations; false-mapping regressions | No generic stopword/stemming heuristic is promoted to semantic truth. | +| Khalov & Ataeva (2025), *Automating Ontology Mapping in IT Service Management* | client | Lexical, embeddings, graph structure, and LLM signals may be compared as candidate features. | research_only | Model Alignment research adapter | feature ablation if reproduced | Reported validation uses an LLM surrogate expert and no annotated gold; it cannot ground production acceptance. | +| Xiao et al. (2025), *LLM4VKG* | bridge | Schema analysis + ontology development + mapping creation must flow into a stable downstream consumption contract and tolerate incomplete ontology inputs without inventing truth. | adopt | Generation↔Client seam | RODI-style mapping F1; incomplete-ontology fixtures; GRC round-trip | VKG execution remains in consuming systems; ConceptWeave does not become their query/database authority. | +| Li, Garijo, & Poveda-Villalón (2026), systematic literature review | cross_cutting | Standardize task definitions, datasets, metrics, prompt/model receipts, and human-expert review; disclose reproducibility gaps. | adopt | all bounded contexts / Evaluation | reproducibility manifest; benchmark disclosure; provider/prompt sensitivity | Literature reports heterogeneous protocols; no paper/prompt becomes a universal algorithm. | +| Du, An, Wang, & Liu (2024), ontology-learning review | cross_cutting | Keep shallow/deep/LLM methods comparable rather than treating LLMs as the only valid generation family. | research_only | Evaluation | baseline taxonomy of method families | Secondary review; the 2026 systematic review is the stronger cross-cutting evidence base. | + +## GRC reference flow + +`ContextualWisdomLab/governance-risk-compliance` is the first enterprise golden/reference scenario, not a special-case algorithm. The same immutable GRC fixture must exercise both tracks: + +`GRC source contract -> observed facts -> generation candidates -> validation/steward review -> semantic_release -> client validation/resolution/diff/query-plan -> GRC deterministic calculation`. + +Acceptance must prove that ConceptWeave never becomes the GRC system of record, that proposed/inferred relations do not mutate authoritative GRC records, that release validation works offline, and that release upgrades identify affected GRC queries explicitly. Public OAEI/RODI/LLMs4OL-style benchmarks remain necessary because one enterprise fixture cannot establish general matching or learning performance. + +## Consensus records used in this snapshot + +The following canonical Consensus records were fetched before recording product implications: + +- https://consensus.app/papers/accelerating-knowledge-graph-and-ontology-engineering-shimizu-hitzler/82d868ee8f7953108246241e28d5e339/?utm_source=chatgpt +- https://consensus.app/papers/enhancing-knowledge-graph-construction-using-large-trajanoska-stojanov/80ffe83041735fdf94bf4b60dd32ba1a/?utm_source=chatgpt +- https://consensus.app/papers/ontogenix-leveraging-large-language-models-for-enhanced-val-calvo-aranguren/2c2771b0905a5292b6addb6b299bda17/?utm_source=chatgpt +- https://consensus.app/papers/endtoend-ontology-learning-with-large-language-models-lo-jiang/c3543c4051ac5bd7bf6932020e8d5120/?utm_source=chatgpt +- https://consensus.app/papers/llms4ol-large-language-models-for-ontology-learning-giglou-d’souza/971c6331c7cd5e24a3a547d5a938b40d/?utm_source=chatgpt +- https://consensus.app/papers/llms4ol-2024-overview-the-1st-large-language-models-for-giglou-d’souza/3ee443141c0d51bf9a9a8f2257070f04/?utm_source=chatgpt +- https://consensus.app/papers/the-ghost-at-llms4ol-2024-task-a-prompttuningbased-large-phuttaamart-kertkeidkachorn/9d0fce91b8ba550fa308ec889bd7056e/?utm_source=chatgpt +- https://consensus.app/papers/olive-ontology-learning-with-integrated-vector-zhang-dalal/1d96dab8b5c45b9fbbf19ecf9d39bc23/?utm_source=chatgpt +- https://consensus.app/papers/ontology-generation-using-large-language-models-lippolis-saeedizade/f3dd9e0944c253e3962b9ae9f4dc7867/?utm_source=chatgpt +- https://consensus.app/papers/ontolearner-a-modular-python-library-for-ontology-giglou-d’souza/63f55ff320b759d0a5e6e2b79fe4e37a/?utm_source=chatgpt +- https://consensus.app/papers/olala-ontology-matching-with-large-language-models-hertling-paulheim/53331022346755a49bdbb5455ae13b8c/?utm_source=chatgpt +- https://consensus.app/papers/llms4om-matching-ontologies-with-large-language-models-giglou-d’souza/a45561fbd0b25041a04df0f2fa49440b/?utm_source=chatgpt +- https://consensus.app/papers/complex-ontology-matching-with-large-language-model-sousa-lima/7244613a2f595e9d9ded3c8e62300d99/?utm_source=chatgpt +- https://consensus.app/papers/ontology-matching-with-large-language-models-and-taboada-martínez/7cff568231f455de89f26311b6be0d26/?utm_source=chatgpt +- https://consensus.app/papers/kroma-ontology-matching-with-knowledge-retrieval-and-barcelos-french/4669219e2e1c54ea8af442fbc690f922/?utm_source=chatgpt +- https://consensus.app/papers/agentom-leveraging-llm-agents-for-ontology-matching-qiang-wang/1ff1e2abb0f255299ecb808951ceaf6b/?utm_source=chatgpt +- https://consensus.app/papers/genom-ontology-matching-with-description-generation-and-song-chen/8587c3ae332a516d8426504b1f64447c/?utm_source=chatgpt +- https://consensus.app/papers/om4ov-leveraging-ontology-matching-for-ontology-qiang-taylor/bc311ce1e88c52a1ad9485037371b2e0/?utm_source=chatgpt +- https://consensus.app/papers/oaeillm-a-benchmark-dataset-for-understanding-large-qiang-taylor/e71db19036e651e69c2b5cee75d36935/?utm_source=chatgpt +- https://consensus.app/papers/oaeillmt-a-tbox-benchmark-dataset-for-understanding-large-qiang-taylor/168a617397d8509ba9fe67e9889f2cab/?utm_source=chatgpt +- https://consensus.app/papers/crowdom-crowdsourcing-for-ontology-matching-validation-qiang-wang/55e7bc49f40d56c7994ffb1e28d1e0fc/?utm_source=chatgpt +- https://consensus.app/papers/how-does-a-text-preprocessing-pipeline-affect-ontology-qiang-taylor/b21e8fc85c665d108ec0022368151aca/?utm_source=chatgpt +- https://consensus.app/papers/automating-ontology-mapping-in-it-service-management-a-khalov-ataeva/74e0b948f0a8599a8ba223f23eeac3cc/?utm_source=chatgpt +- https://consensus.app/papers/llm4vkg-leveraging-large-language-models-for-virtual-xiao-ren/c6486ba49d125d66ad70bb4f97df5dc7/?utm_source=chatgpt +- https://consensus.app/papers/large-language-models-for-ontology-engineering-a-li-garijo/3087bb8f7cd0500d917f89d8a92559e5/?utm_source=chatgpt +- https://consensus.app/papers/a-short-review-for-ontology-learning-stride-to-large-du-an/ad3e2c6bf660569ca1effb7b6d31a6f7/?utm_source=chatgpt + +## Next executable consequences + +1. Issue #2 Generation evaluation must expose task-level metrics rather than one aggregate "ontology quality" score. +2. Issue #3 Client work must implement retrieval-before-prompt, structural/neighborhood evidence, deterministic lexical evidence, explicit abstention, release diff/version compatibility, and OAEI-LLM hallucination fixtures. +3. GRC must remain the first enterprise round-trip fixture, while OAEI/RODI/LLMs4OL-style data guards against overfitting the general contract to GRC. +4. LLM calls remain behind `contextual-orchestrator`; model/provider/prompt changes require receipts and sensitivity evidence. +5. Human review remains mandatory before authority promotion; Crowd-OM is evidence for scalable validation mechanics, not permission to replace GRC/domain steward authority. From e81f94047fea17d888470553e5d03a5215be32c4 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:52:38 +0900 Subject: [PATCH 06/59] docs: refresh Consensus research register --- docs/doctoring/REFERENCES.md | 64 +++++++++++++++++++++++++++++++++--- 1 file changed, 59 insertions(+), 5 deletions(-) diff --git a/docs/doctoring/REFERENCES.md b/docs/doctoring/REFERENCES.md index f42a762c..5cf4acd7 100644 --- a/docs/doctoring/REFERENCES.md +++ b/docs/doctoring/REFERENCES.md @@ -1,6 +1,6 @@ # Standards and Research References -This file records the initial evidence basis for ConceptWeave architecture decisions. Stable Recommendations and in-progress specifications are deliberately distinguished. +This file records the evidence basis for ConceptWeave architecture decisions. Stable Recommendations and in-progress specifications are deliberately distinguished. The current paper-by-paper capability, rejection/adoption, owner, limitation, and benchmark mapping is maintained in `docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md`; bibliography alone does not count as product use. ## Stable standards / recommendations @@ -25,18 +25,72 @@ W3C Data Shapes Working Group. (2026). *SHACL 1.2 Core* (Working Draft, August 3 Apache Software Foundation. (2026). *Apache Ossie (incubating)*. https://ossie.apache.org/ Formerly Open Semantic Interchange (OSI); tracked as an emerging vendor-neutral semantic-model exchange specification rather than a W3C/ISO standard. -## Research basis +## Generation research Babaei Giglou, H., D'Souza, J., & Auer, S. (2023). LLMs4OL: Large language models for ontology learning. In *The Semantic Web – ISWC 2023* (pp. 408–427). Springer. https://doi.org/10.1007/978-3-031-47240-4_22 -Hertling, S., & Paulheim, H. (2023). OLaLa: Ontology matching with large language models. In *Proceedings of the 12th Knowledge Capture Conference 2023* (pp. 131–139). Association for Computing Machinery. https://doi.org/10.1145/3587259.3627571 +Trajanoska, M., Stojanov, R., & Trajanov, D. (2023). *Enhancing knowledge graph construction using large language models* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/enhancing-knowledge-graph-construction-using-large-trajanoska-stojanov/80ffe83041735fdf94bf4b60dd32ba1a/?utm_source=chatgpt + +Shimizu, C., & Hitzler, P. (2024). *Accelerating knowledge graph and ontology engineering with large language models* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/accelerating-knowledge-graph-and-ontology-engineering-shimizu-hitzler/82d868ee8f7953108246241e28d5e339/?utm_source=chatgpt Lo, A., Jiang, A. Q., Li, W., & Jamnik, M. (2024). *End-to-end ontology learning with large language models* [Preprint]. arXiv. https://arxiv.org/abs/2410.23584 +Babaei Giglou, H., D'Souza, J., & Auer, S. (2024). *LLMs4OL 2024 Overview: The 1st Large Language Models for Ontology Learning Challenge*. Consensus record: https://consensus.app/papers/llms4ol-2024-overview-the-1st-large-language-models-for-giglou-d’souza/3ee443141c0d51bf9a9a8f2257070f04/?utm_source=chatgpt + +Phuttaamart, T., Kertkeidkachorn, N., & Trongratsameethong, A. (2024). *The Ghost at LLMs4OL 2024 Task A: Prompt-Tuning-Based Large Language Models for Term Typing*. Consensus record: https://consensus.app/papers/the-ghost-at-llms4ol-2024-task-a-prompttuningbased-large-phuttaamart-kertkeidkachorn/9d0fce91b8ba550fa308ec889bd7056e/?utm_source=chatgpt + +Val-Calvo, M., Egaña Aranguren, M., Martínez-Hernández, J. M., Almagro-Hernández, G., Deshmukh, P., Bernabé-Díaz, J. A., Espinoza-Arias, P., Sánchez-Fernández, J., Mueller, J., & Fernández-Breis, J. (2025). OntoGenix: Leveraging large language models for enhanced ontology engineering from datasets. *Information Processing & Management, 62*, 104042. Consensus record: https://consensus.app/papers/ontogenix-leveraging-large-language-models-for-enhanced-val-calvo-aranguren/2c2771b0905a5292b6addb6b299bda17/?utm_source=chatgpt + +Zhang, Y., Dalal, A., Martin, C., Gadusu, S. R., & Mcginty, H. (2025). OLIVE: Ontology learning with integrated vector embeddings. *Applied Ontology, 20*, 36–53. Consensus record: https://consensus.app/papers/olive-ontology-learning-with-integrated-vector-zhang-dalal/1d96dab8b5c45b9fbbf19ecf9d39bc23/?utm_source=chatgpt + +Lippolis, A. S., Saeedizade, M. J., Keskisarkka, R., Zuppiroli, S., Ceriani, M., Gangemi, A., Blomqvist, E., & Nuzzolese, A. G. (2025). *Ontology generation using large language models* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/ontology-generation-using-large-language-models-lippolis-saeedizade/f3dd9e0944c253e3962b9ae9f4dc7867/?utm_source=chatgpt + +Giglou, H. B., D'Souza, J., Aioanei, A. C., Mihindukulasooriya, N., & Auer, S. (2026). *OntoLearner: A modular Python library for ontology learning with large language models*. Consensus record: https://consensus.app/papers/ontolearner-a-modular-python-library-for-ontology-giglou-d’souza/63f55ff320b759d0a5e6e2b79fe4e37a/?utm_source=chatgpt + +## Client, matching, and versioning research + +Hertling, S., & Paulheim, H. (2023). OLaLa: Ontology matching with large language models. In *Proceedings of the 12th Knowledge Capture Conference 2023* (pp. 131–139). Association for Computing Machinery. https://doi.org/10.1145/3587259.3627571 + +Qiang, Z., Wang, W., & Taylor, K. L. (2023). Agent-OM: Leveraging LLM agents for ontology matching. *Proceedings of the VLDB Endowment, 18*, 516–529. Consensus record: https://consensus.app/papers/agentom-leveraging-llm-agents-for-ontology-matching-qiang-wang/1ff1e2abb0f255299ecb808951ceaf6b/?utm_source=chatgpt + +Babaei Giglou, H., D'Souza, J., Engel, F., & Auer, S. (2024). *LLMs4OM: Matching ontologies with large language models* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/llms4om-matching-ontologies-with-large-language-models-giglou-d’souza/a45561fbd0b25041a04df0f2fa49440b/?utm_source=chatgpt + +Qiang, Z., & Taylor, K. L. (2024). *OM4OV: Leveraging ontology matching for ontology versioning* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/om4ov-leveraging-ontology-matching-for-ontology-qiang-taylor/bc311ce1e88c52a1ad9485037371b2e0/?utm_source=chatgpt + +Qiang, Z., Taylor, K. L., & Wang, W. (2024). *How does a text preprocessing pipeline affect ontology syntactic matching?* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/how-does-a-text-preprocessing-pipeline-affect-ontology-qiang-taylor/b21e8fc85c665d108ec0022368151aca/?utm_source=chatgpt + +Sousa, G., Lima, R., & Trojahn, C. (2025). *Complex ontology matching with large language model embeddings* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/complex-ontology-matching-with-large-language-model-sousa-lima/7244613a2f595e9d9ded3c8e62300d99/?utm_source=chatgpt + +Taboada, M., Martínez, D., Arideh, M., & Mosquera, R. (2025). *Ontology matching with large language models and prioritized depth-first search* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/ontology-matching-with-large-language-models-and-taboada-martínez/7cff568231f455de89f26311b6be0d26/?utm_source=chatgpt + +Barcelos, E. I., French, R. H., & Wu, Y. (2025). *KROMA: Ontology matching with knowledge retrieval and large language models* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/kroma-ontology-matching-with-knowledge-retrieval-and-barcelos-french/4669219e2e1c54ea8af442fbc690f922/?utm_source=chatgpt + +Song, Y., Chen, J., & Schmidt, R. A. (2025). GenOM: Ontology matching with description generation and large language models. *World Wide Web, 29*. Consensus record: https://consensus.app/papers/genom-ontology-matching-with-description-generation-and-song-chen/8587c3ae332a516d8426504b1f64447c/?utm_source=chatgpt + +Khalov, A., & Ataeva, O. (2025). Automating ontology mapping in IT service management: A DOLCE and ITSMO integration. *Data Science Journal, 24*. Consensus record: https://consensus.app/papers/automating-ontology-mapping-in-it-service-management-a-khalov-ataeva/74e0b948f0a8599a8ba223f23eeac3cc/?utm_source=chatgpt + +## Bridge research + +Xiao, G., Ren, L., Qi, G., Xue, H., Di Panfilo, M., & Lanti, D. (2025). *LLM4VKG: Leveraging large language models for virtual knowledge graph construction*. Consensus record: https://consensus.app/papers/llm4vkg-leveraging-large-language-models-for-virtual-xiao-ren/c6486ba49d125d66ad70bb4f97df5dc7/?utm_source=chatgpt + +## Evaluation, hallucination, and governance research + +Qiang, Z., Taylor, K. L., Wang, W., & Jiang, J. (2024). *OAEI-LLM: A benchmark dataset for understanding large language model hallucinations in ontology matching* [Preprint]. arXiv. Consensus record: https://consensus.app/papers/oaeillm-a-benchmark-dataset-for-understanding-large-qiang-taylor/e71db19036e651e69c2b5cee75d36935/?utm_source=chatgpt + +Qiang, Z., Taylor, K. L., Wang, W., & Jiang, J. (2025). *OAEI-LLM-T: A TBox benchmark dataset for understanding large language model hallucinations in ontology matching*. Consensus record: https://consensus.app/papers/oaeillmt-a-tbox-benchmark-dataset-for-understanding-large-qiang-taylor/168a617397d8509ba9fe67e9889f2cab/?utm_source=chatgpt + +Qiang, Z., Wang, W., & Taylor, K. L. (2026). *Crowd-OM: Crowdsourcing for ontology matching validation*. Consensus record: https://consensus.app/papers/crowdom-crowdsourcing-for-ontology-matching-validation-qiang-wang/55e7bc49f40d56c7994ffb1e28d1e0fc/?utm_source=chatgpt + +Du, R., An, H., Wang, K., & Liu, W. (2024). *A short review for ontology learning: Stride to large language models trend*. Consensus record: https://consensus.app/papers/a-short-review-for-ontology-learning-stride-to-large-du-an/ad3e2c6bf660569ca1effb7b6d31a6f7/?utm_source=chatgpt + Li, J., Garijo, D., & Poveda-Villalón, M. (2026). Large language models for ontology engineering: A systematic literature review. *Semantic Web, 17*(4), 1–45. https://doi.org/10.1177/22104968261465514 ## Decision implications -- LLMs can assist ontology learning, matching, modeling, and maintenance, but current research does not justify automatic authority promotion. -- Evaluation practices across LLM ontology engineering are still heterogeneous; ConceptWeave therefore requires reproducible deterministic checks plus human-reviewed benchmark cases. +- LLMs can assist ontology learning, matching, modeling, and maintenance, but no retrieved study justifies automatic authority promotion. +- Generation is evaluated as distinct term-typing, taxonomy, non-taxonomic relation, mapping, structural, provenance, and abstention tasks; one aggregate ontology-quality score is insufficient. +- Client matching is retrieval/pruning/structural-evidence first. LLM prompting is a bounded candidate-ranking/explanation tool, not the source of truth. +- Ontology versioning and release compatibility are treated as distinct from ordinary matching; release diff must detect additions/removals/changes and explain affected client queries. +- OAEI-LLM/OAEI-LLM-T add LLM-specific hallucination categories to matching evaluation. GRC remains the enterprise round-trip fixture rather than the sole benchmark. - Modular ontology engineering and explicit source provenance are preferred over one opaque prompt that attempts to generate an entire enterprise semantic layer in a single step. +- Human review remains mandatory before authority promotion. Scalable validation research may inform review mechanics but cannot replace domain-owner/steward authority. From 6014f9ef7bcbbd3957917ac156d5807534898b63 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:56:02 +0900 Subject: [PATCH 07/59] docs: align provenance requirement with current contract --- docs/PRD.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/PRD.md b/docs/PRD.md index 28465202..0e68c400 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -32,7 +32,7 @@ Produce candidates for concepts, taxonomies, non-taxonomic relations, semantic c ### FR-3 Evidence and provenance -Every candidate must retain source identity, digest, location, observation time, parser/extractor revision, and discovery method. Unsupported candidates fail closed. +The current v0.1 candidate contract requires every candidate to retain exact source identity, source digest, and source location through `EvidenceReference`. Issue #2 must add immutable Source Observation and proposal-receipt contracts that also retain observation time, parser/extractor revision, and discovery method before the first Generation release. Until those receipt contracts exist, the Rust `SemanticCandidate` and `contracts/semantic-candidate.schema.json` must not be described as already carrying those deferred coordinates. Unsupported candidates fail closed. ### FR-4 Deterministic validation From fb828331c4e48b2b23a33810eaa6679789dabfe5 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:56:13 +0900 Subject: [PATCH 08/59] docs: make published releases immutable --- SECURITY.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/SECURITY.md b/SECURITY.md index 9a8fa55a..5d930ad0 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -15,7 +15,7 @@ All source artifacts, generated candidate payloads, external ontology files, mod - no source-system writes from discovery or validation; - reviewed authorization required before publication; - future tenant isolation applies to source snapshots, candidates, review receipts, releases, exports, and object storage; -- published artifacts retain origin/provenance and cannot silently overwrite prior releases. +- published semantic truth is immutable: a published artifact must never be overwritten in place, including when an audit trail exists; corrections are issued as a new release that explicitly supersedes the prior release while retaining both releases and their provenance. ## Threats tracked from foundation @@ -28,6 +28,7 @@ All source artifacts, generated candidate payloads, external ontology files, mod 7. malicious or oversized schema/API artifacts; 8. external-source SSRF or credential leakage; 9. model/provider compromise or unexpected retention; -10. governance bypass from Proposed/Validated directly to Published. +10. governance bypass from Proposed/Validated directly to Published; +11. in-place mutation or overwrite of previously published semantic truth. Security findings become tests before the related runtime capability can be marked release-ready. From 70697ca03052df8c3660324bd4001e4996f493f6 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:56:53 +0900 Subject: [PATCH 09/59] docs: pin SHACL draft citation --- docs/doctoring/REFERENCES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/doctoring/REFERENCES.md b/docs/doctoring/REFERENCES.md index 5cf4acd7..e161cc04 100644 --- a/docs/doctoring/REFERENCES.md +++ b/docs/doctoring/REFERENCES.md @@ -20,7 +20,7 @@ World Wide Web Consortium. (2020). *JSON-LD 1.1*. https://www.w3.org/TR/json-ld1 W3C RDF-star Working Group. (2026). *RDF 1.2 concepts and abstract data model* (Candidate Recommendation Snapshot, April 7, 2026). World Wide Web Consortium. https://www.w3.org/TR/rdf12-concepts/ -W3C Data Shapes Working Group. (2026). *SHACL 1.2 Core* (Working Draft, August 3, 2026). World Wide Web Consortium. https://www.w3.org/TR/shacl12-core/ +W3C Data Shapes Working Group. (2026). *SHACL 1.2 Core* (Working Draft, August 3, 2026). World Wide Web Consortium. https://www.w3.org/TR/2026/WD-shacl12-core-20260803/ Apache Software Foundation. (2026). *Apache Ossie (incubating)*. https://ossie.apache.org/ Formerly Open Semantic Interchange (OSI); tracked as an emerging vendor-neutral semantic-model exchange specification rather than a W3C/ISO standard. From 62bcbee30a74fa7379c5c1cb410a2da05f720020 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:58:48 +0900 Subject: [PATCH 10/59] fix: align public candidate schema invariants --- contracts/semantic-candidate.schema.json | 41 +++++++++++++++++++++--- 1 file changed, 36 insertions(+), 5 deletions(-) diff --git a/contracts/semantic-candidate.schema.json b/contracts/semantic-candidate.schema.json index 7e090a45..aa5955b8 100644 --- a/contracts/semantic-candidate.schema.json +++ b/contracts/semantic-candidate.schema.json @@ -12,7 +12,11 @@ "evidence" ], "properties": { - "candidate_id": {"type": "string", "minLength": 1}, + "candidate_id": { + "type": "string", + "minLength": 1, + "pattern": ".*\\S.*" + }, "kind": { "enum": [ "concept", @@ -53,11 +57,38 @@ "additionalProperties": false, "required": ["source_id", "source_digest", "location"], "properties": { - "source_id": {"type": "string", "minLength": 1}, - "source_digest": {"type": "string", "minLength": 1}, - "location": {"type": "string", "minLength": 1} + "source_id": { + "type": "string", + "minLength": 1, + "pattern": ".*\\S.*" + }, + "source_digest": { + "type": "string", + "minLength": 1, + "pattern": ".*\\S.*" + }, + "location": { + "type": "string", + "minLength": 1, + "pattern": ".*\\S.*" + } + } + } + } + }, + "allOf": [ + { + "if": { + "properties": { + "publication_state": {"const": "published"} + }, + "required": ["publication_state"] + }, + "then": { + "properties": { + "truth_status": {"const": "authoritative"} } } } - } + ] } From f4ff0a174c4fc0340b1e604ee902c91fe841b015 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:58:55 +0900 Subject: [PATCH 11/59] test: add candidate schema fixtures --- contracts/fixtures/semantic-candidate.valid.json | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 contracts/fixtures/semantic-candidate.valid.json diff --git a/contracts/fixtures/semantic-candidate.valid.json b/contracts/fixtures/semantic-candidate.valid.json new file mode 100644 index 00000000..69797936 --- /dev/null +++ b/contracts/fixtures/semantic-candidate.valid.json @@ -0,0 +1,13 @@ +{ + "candidate_id": "candidate-1", + "kind": "concept", + "truth_status": "authoritative", + "publication_state": "published", + "evidence": [ + { + "source_id": "source-1", + "source_digest": "sha256:abc", + "location": "public.orders" + } + ] +} From a2c0e1980c6db1c5b34b5e94148e2497dd300636 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:59:00 +0900 Subject: [PATCH 12/59] test: add whitespace-invalid schema fixture --- .../semantic-candidate.invalid-whitespace.json | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 contracts/fixtures/semantic-candidate.invalid-whitespace.json diff --git a/contracts/fixtures/semantic-candidate.invalid-whitespace.json b/contracts/fixtures/semantic-candidate.invalid-whitespace.json new file mode 100644 index 00000000..b3097f73 --- /dev/null +++ b/contracts/fixtures/semantic-candidate.invalid-whitespace.json @@ -0,0 +1,13 @@ +{ + "candidate_id": " ", + "kind": "concept", + "truth_status": "authoritative", + "publication_state": "published", + "evidence": [ + { + "source_id": "source-1", + "source_digest": "sha256:abc", + "location": "public.orders" + } + ] +} From 21e44a722e09cc72487fb4fa71ac631b159e2c06 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:59:10 +0900 Subject: [PATCH 13/59] test: add invalid published-truth schema fixture --- .../semantic-candidate.invalid-published-truth.json | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 contracts/fixtures/semantic-candidate.invalid-published-truth.json diff --git a/contracts/fixtures/semantic-candidate.invalid-published-truth.json b/contracts/fixtures/semantic-candidate.invalid-published-truth.json new file mode 100644 index 00000000..7bcebe21 --- /dev/null +++ b/contracts/fixtures/semantic-candidate.invalid-published-truth.json @@ -0,0 +1,13 @@ +{ + "candidate_id": "candidate-1", + "kind": "concept", + "truth_status": "observed", + "publication_state": "published", + "evidence": [ + { + "source_id": "source-1", + "source_digest": "sha256:abc", + "location": "public.orders" + } + ] +} From 3d187c12cacc238fddfd178d9af6b28b5359066a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:59:28 +0900 Subject: [PATCH 14/59] ci: validate candidate schema semantics --- .github/workflows/product.yml | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index 6b52b599..ad6f036c 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -50,10 +50,31 @@ jobs: run: ./scripts/check_coverage.sh - name: Validate public JSON contract - run: jq -e . contracts/semantic-candidate.schema.json >/dev/null + run: | + npx --yes ajv-cli@5.0.0 compile \ + --spec=draft2020 \ + -s contracts/semantic-candidate.schema.json + npx --yes ajv-cli@5.0.0 test \ + --spec=draft2020 \ + -s contracts/semantic-candidate.schema.json \ + -d contracts/fixtures/semantic-candidate.valid.json \ + --valid + npx --yes ajv-cli@5.0.0 test \ + --spec=draft2020 \ + -s contracts/semantic-candidate.schema.json \ + -d contracts/fixtures/semantic-candidate.invalid-whitespace.json \ + --invalid + npx --yes ajv-cli@5.0.0 test \ + --spec=draft2020 \ + -s contracts/semantic-candidate.schema.json \ + -d contracts/fixtures/semantic-candidate.invalid-published-truth.json \ + --invalid - name: Lockfile freshness - run: cargo generate-lockfile && git diff --exit-code -- Cargo.lock + run: | + cargo generate-lockfile + git ls-files --error-unmatch Cargo.lock >/dev/null + test -z "$(git status --porcelain=v1 --untracked-files=all -- Cargo.lock)" - name: Clean working tree - run: git diff --exit-code + run: test -z "$(git status --porcelain=v1 --untracked-files=all)" From 2b6942a5da44d39a383cc1e52b4e820f7ff5c6d3 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 16:59:49 +0900 Subject: [PATCH 15/59] docs: narrow candidate schema claim to enforced scope --- docs/product-technical-gap-baseline.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 734b43ee..c99d8269 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -11,11 +11,11 @@ Only the repository bootstrap README exists before the foundation PR. No product | Area | Status | Evidence / next action | | --- | --- | --- | | Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define Semantic Model Engineering and CWL boundaries. | -| Truth/publication lifecycle | ACTIVE_PR | Rust domain contract + JSON Schema enforce evidence and reviewed-before-published lifecycle. | +| Truth/publication lifecycle | ACTIVE_PR | Rust domain lifecycle defines Draft -> Proposed -> Validated -> Reviewed -> Published. Draft 2020-12 JSON Schema enforces candidate shape, non-blank evidence identities, and Published -> Authoritative consistency; lifecycle history/pre-Reviewed publication is not a JSON-Schema responsibility. A test-first regression currently requires Reviewed -> Published to fail if required evidence is absent before this slice can be merge-ready. | | Rust baseline | ACTIVE_PR | Rust 1.98.0 workspace, unsafe forbidden, public docs required. | -| Quality gate | ACTIVE_PR | Product workflow for fmt/clippy/tests/docs/coverage/schema/lock/clean-tree. | -| Standards/research | ACTIVE_PR | Stable-vs-draft standards and ontology-engineering research doctoring. | -| Security/test/operability | ACTIVE_PR | Baselines added; no production service claimed. | +| Quality gate | ACTIVE_PR | Product workflow for fmt/clippy/tests/docs/coverage/Draft-2020-12 schema fixtures/lock/clean-tree; exact current-head hosted execution remains required. | +| Standards/research | ACTIVE_PR | Stable-vs-draft standards plus paper-by-paper Generation/Client/Bridge/cross-cutting capability and evaluation traceability. | +| Security/test/operability | ACTIVE_PR | Baselines added; published semantic truth is specified as immutable with correction by superseding release; no production service claimed. | ## P0 product gaps after foundation From c861d0b7ffac07ae6e51c3721ec2431988c3482f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 17:01:54 +0900 Subject: [PATCH 16/59] test: forbid external candidate invariant bypass --- crates/conceptweave-domain/src/lib.rs | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/crates/conceptweave-domain/src/lib.rs b/crates/conceptweave-domain/src/lib.rs index a8fe48d4..ddd0d54e 100644 --- a/crates/conceptweave-domain/src/lib.rs +++ b/crates/conceptweave-domain/src/lib.rs @@ -122,6 +122,19 @@ impl EvidenceReference { } /// A governed candidate for an ontology or semantic-layer artifact. +/// +/// External consumers must not be able to mutate evidence or governance state +/// without a validated domain operation. This compile-fail example is an +/// executable boundary test: it must fail once the invariant is correctly +/// encapsulated. +/// +/// ```compile_fail +/// use conceptweave_domain::{CandidateKind, EvidenceReference, SemanticCandidate}; +/// +/// let evidence = EvidenceReference::new("source-1", "sha256:abc", "public.orders").unwrap(); +/// let mut candidate = SemanticCandidate::new("candidate-1", CandidateKind::Concept, vec![evidence]).unwrap(); +/// candidate.evidence.clear(); +/// ``` #[derive(Debug, Clone, PartialEq, Eq)] pub struct SemanticCandidate { /// Stable candidate identifier within the owning tenant or workspace. From 1c982f246f0b13c814d463ae2911e1049f9c253d Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 17:11:32 +0900 Subject: [PATCH 17/59] test: require evidence at publication authority boundary --- crates/conceptweave-domain/src/lib.rs | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/crates/conceptweave-domain/src/lib.rs b/crates/conceptweave-domain/src/lib.rs index ddd0d54e..4ff7d564 100644 --- a/crates/conceptweave-domain/src/lib.rs +++ b/crates/conceptweave-domain/src/lib.rs @@ -354,6 +354,27 @@ mod tests { ); } + #[test] + fn publication_rechecks_evidence_at_authority_boundary() { + let mut candidate = candidate(); + for state in [ + PublicationState::Proposed, + PublicationState::Validated, + PublicationState::Reviewed, + ] { + candidate.transition(state).unwrap(); + } + + candidate.evidence.clear(); + + assert_eq!( + candidate.transition(PublicationState::Published), + Err(ContractError::MissingEvidence) + ); + assert_eq!(candidate.publication_state, PublicationState::Reviewed); + assert_eq!(candidate.truth_status, TruthStatus::Inferred); + } + #[test] fn contract_errors_explain_the_failure() { assert_eq!( From 217bbfdcc42239ac20780de7990cad46e122be15 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 17:57:49 +0900 Subject: [PATCH 18/59] docs: add DeepWiki and documentation entry point --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3ec89033..b777770f 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ # ConceptWeave +[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/ConceptWeave) + **Automatic ontology and semantic-layer engineering for governed enterprise meaning.** ConceptWeave turns heterogeneous enterprise evidence—schemas, APIs, event contracts, documents, code structure, existing vocabularies, and lineage—into **reviewable semantic-model candidates**. It does not make model-generated meaning authoritative by itself. Candidates must retain source evidence, pass deterministic validation, and move through an explicit governance lifecycle before publication. @@ -48,4 +50,4 @@ cargo clippy --workspace --all-targets -- -D warnings cargo test --workspace ``` -See [`docs/PRD.md`](docs/PRD.md), [`docs/TRD.md`](docs/TRD.md), and [`ARCHITECTURE.md`](ARCHITECTURE.md). +See [`docs/PRD.md`](docs/PRD.md), [`docs/TRD.md`](docs/TRD.md), [`ARCHITECTURE.md`](ARCHITECTURE.md), and the [documentation home](docs/index.md). From a05befd59c7fd297185453e82eaa9d433032a520 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 17:58:00 +0900 Subject: [PATCH 19/59] docs: add ConceptWeave Pages landing page --- docs/index.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 docs/index.md diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..5c31d18b --- /dev/null +++ b/docs/index.md @@ -0,0 +1,23 @@ +# ConceptWeave + +ConceptWeave turns enterprise evidence into governed, reviewable semantic models while keeping source systems authoritative. + +## Product goal + +The first release target converts an immutable relational-schema snapshot into evidence-bound semantic candidates for concepts, relationships, dimensions, measures, constraints, and physical mappings. Candidates remain proposed until deterministic validation and authorized review permit publication. + +## Current status + +ConceptWeave is in foundation development. The active foundation work establishes its Rust domain model, lifecycle, public schema, architecture, security, test strategy, and operability baseline. Source adapters, persistence, model-assisted induction, steward interfaces, and publication adapters remain explicit gaps until implemented and verified. + +## Start here + +- [Repository overview](../README.md) +- [Product requirements](PRD.md) +- [Technical requirements](TRD.md) +- [Architecture](../ARCHITECTURE.md) +- [DeepWiki](https://deepwiki.com/ContextualWisdomLab/ConceptWeave) + +## Governance boundary + +Generated or LLM-assisted meaning is never authoritative by default. Every published semantic release must preserve source evidence and pass the product's deterministic validation and governance lifecycle. From cd7b04c6f9aee25000a0f7b5b5f1ac9a62d50e1a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 18:17:48 +0900 Subject: [PATCH 20/59] docs: make ConceptWeave README user-first --- README.md | 163 ++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 140 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index b777770f..3d0d73ed 100644 --- a/README.md +++ b/README.md @@ -2,52 +2,169 @@ [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ContextualWisdomLab/ConceptWeave) -**Automatic ontology and semantic-layer engineering for governed enterprise meaning.** +**Automatic, evidence-bound ontology and semantic-layer engineering for governed enterprise meaning.** -ConceptWeave turns heterogeneous enterprise evidence—schemas, APIs, event contracts, documents, code structure, existing vocabularies, and lineage—into **reviewable semantic-model candidates**. It does not make model-generated meaning authoritative by itself. Candidates must retain source evidence, pass deterministic validation, and move through an explicit governance lifecycle before publication. +ConceptWeave turns heterogeneous enterprise evidence—schemas, APIs, event contracts, documents, code structure, vocabularies, and lineage—into **reviewable semantic-model candidates**. Generated meaning never becomes authoritative merely because a model proposed it: candidates retain source evidence, pass deterministic validation, and move through an explicit review/publication lifecycle. + +## Why it exists + +Enterprise semantic models are valuable only when teams can explain where meaning came from, what was inferred, who reviewed it, and what is actually published. ConceptWeave makes that lifecycle explicit instead of collapsing discovery, generation, governance, and publication into one opaque step. + +| Need | What ConceptWeave provides | +| --- | --- | +| Semantic discovery | Evidence-bound candidate concepts, relations, dimensions, measures, constraints, and mappings | +| Governance | Separate truth status from publication state with explicit review before publication | +| Traceability | Exact source-evidence bindings carried with semantic candidates | +| Deterministic validation | Machine-checkable structural and lifecycle invariants before authority changes | +| Interoperability | Versioned semantic packages and explicit integration boundaries | +| Safe LLM assistance | Proposal assistance only; model output is never publication authority | ## Product boundary -ConceptWeave owns **semantic model engineering**: +ConceptWeave owns the semantic-model engineering lifecycle: -`observe -> discover -> propose -> validate -> review -> publish` +```text +observe → discover → propose → validate → review → publish +``` -It does **not** own: +Adjacent responsibilities remain separate: -- enterprise catalog/search/runtime consumption (`semantic-data-portal`), -- lineage reconstruction (`LineageWeave`), -- cross-product graph/event contracts (`context-graph-contracts`), -- LLM provider routing (`contextual-orchestrator`), or -- the authoritative business data of source systems. +- [`semantic-data-portal`](https://github.com/ContextualWisdomLab/semantic-data-portal) owns published semantic catalog, governance, and consumption surfaces. +- [`LineageWeave`](https://github.com/ContextualWisdomLab/LineageWeave) provides inferred/proposed lineage evidence. +- `context-graph-contracts` owns cross-product provider-neutral graph/event interoperability contracts. +- [`contextual-orchestrator`](https://github.com/ContextualWisdomLab/contextual-orchestrator) owns LLM/provider discovery and routing. +- Source systems remain authoritative for their own business data. -External source-analysis tools may be integrated behind adapters, but no external fork is treated as ConceptWeave product authority. +External source-analysis tools can sit behind adapters, but no external fork or model output becomes ConceptWeave product authority. -## First release target +## First vertical -The first vertical is a **relational-schema-to-governed-semantic-model proposal**: +The first product vertical is **relational schema → governed semantic-model proposal**: 1. ingest an immutable schema snapshot; 2. derive observed physical entities and relationships; -3. propose concepts, taxonomy/semantic relations, dimensions, measures, constraints, and physical mappings; -4. bind every proposal to exact source evidence; +3. propose concepts, semantic/taxonomy relations, dimensions, measures, constraints, and physical mappings; +4. bind every proposal to exact evidence; 5. validate structure and consistency; -6. require steward review before publication; and -7. export a versioned semantic package suitable for ontology and analytics consumers. +6. require authorized review; and +7. publish a versioned semantic package only after the lifecycle permits it. -Planned publication targets include OWL/RDFS/SKOS, SHACL, JSON-LD, and an Apache Ossie-compatible semantic-model projection when the emerging specification is sufficiently stable for the required subset. +Future publication adapters may target standards such as OWL/RDFS/SKOS, SHACL, and JSON-LD. Emerging formats such as Apache Ossie are tracked as evolving interoperability targets rather than represented as finalized standards. -## Current state +## Current implementation -This foundation PR establishes the Rust domain contract, candidate truth/publication lifecycle, JSON Schema, DDD architecture, standards/research baseline, security/test/operability baselines, and CI. Source adapters, LLM-assisted induction, persistence, reasoning, review UI, and publication adapters remain explicit product gaps. +The current foundation establishes the reusable domain and governance core rather than claiming the entire product is complete. -## Rust +Implemented in this branch: -The repository is pinned to Rust 1.98.0. The current core has no third-party runtime dependencies. +- Rust workspace and `conceptweave-domain` core; +- evidence-bound `SemanticCandidate` contract; +- independent truth-status and publication-state semantics; +- fail-closed candidate lifecycle with rejection and supersession paths; +- Draft 2020-12 JSON Schema for the public candidate contract; +- DDD Context Map and Ubiquitous Language; +- architecture, PRD/TRD, ADR, security, test, operability, and research baselines; +- pinned product CI for formatting, Clippy, tests, rustdoc, coverage, schema validation, lock freshness, and clean-tree checks. + +Source adapters, LLM-assisted induction, persistence, reasoning, review UI, and publication adapters remain explicit product gaps until they land with evidence. + +## Quick start + +The repository is pinned to Rust 1.98.0. The current domain core has no third-party runtime dependencies. + +```bash +cargo test --workspace +``` + +Run the full local quality set used by the foundation contract: ```bash cargo fmt --all --check cargo clippy --workspace --all-targets -- -D warnings cargo test --workspace +cargo doc --workspace --no-deps ``` -See [`docs/PRD.md`](docs/PRD.md), [`docs/TRD.md`](docs/TRD.md), [`ARCHITECTURE.md`](ARCHITECTURE.md), and the [documentation home](docs/index.md). +The repository CI also validates the JSON Schema, lock/toolchain freshness, documentation contracts, and coverage expectations defined by the current source. + +## Core contract + +A semantic candidate is not the same thing as published semantic truth. + +```text +Observed evidence + │ + ▼ +Semantic candidate + │ + ├─ Draft + ├─ Proposed + ├─ Validated + ├─ Reviewed + └─ Published +``` + +Publication is an authority boundary. A candidate must preserve the evidence and lifecycle invariants required by the current domain contract; callers must not bypass those invariants by mutating public state or treating a validated proposal as published truth. + +The machine-readable public shape is in [`contracts/semantic-candidate.schema.json`](contracts/semantic-candidate.schema.json). + +## Architecture at a glance + +```text +Enterprise evidence + schemas · APIs · events · docs · code · vocabularies · lineage + │ + ▼ +┌──────────────────────────────────┐ +│ ConceptWeave │ +│ semantic-model engineering │ +├──────────────────────────────────┤ +│ observe / evidence normalization │ +│ candidate discovery & proposal │ +│ deterministic validation │ +│ review / publication lifecycle │ +└───────────────┬──────────────────┘ + │ versioned published semantics + ▼ + catalog / analytics / ontology consumers +``` + +ConceptWeave is the owner of semantic candidate engineering and publication lifecycle rules—not the catalog UI, source-system truth, LLM provider layer, lineage inference engine, or enterprise-wide application data. + +## Standards and research posture + +Stable standards and recommendations are distinguished from drafts and emerging specifications. LLM-assisted ontology engineering is treated as proposal assistance and must pass deterministic validation plus authorized review before publication. + +The standards/research register and design implications live in [`docs/doctoring/`](docs/doctoring/) and are linked through the repository traceability documents. + +## Documentation map + +| Goal | Start here | +| --- | --- | +| Product requirements | [`docs/PRD.md`](docs/PRD.md) | +| Technical requirements | [`docs/TRD.md`](docs/TRD.md) | +| Architecture | [`ARCHITECTURE.md`](ARCHITECTURE.md) | +| Bounded contexts | [`docs/CONTEXT_MAP.md`](docs/CONTEXT_MAP.md) | +| Domain language | [`docs/UBIQUITOUS_LANGUAGE.md`](docs/UBIQUITOUS_LANGUAGE.md) | +| Lifecycle / sequence views | [`docs/UML.md`](docs/UML.md) | +| Architecture decisions | [`docs/adr/README.md`](docs/adr/README.md) | +| Security | [`SECURITY.md`](SECURITY.md) | +| Test strategy | [`TEST_STRATEGY.md`](TEST_STRATEGY.md) | +| Operations | [`OPERABILITY.md`](OPERABILITY.md) | +| Current product/technical gaps | [`docs/product-technical-gap-baseline.md`](docs/product-technical-gap-baseline.md) | +| Documentation home | [`docs/index.md`](docs/index.md) | + +## Product principles + +1. **Evidence before authority.** Semantic meaning remains traceable to source evidence. +2. **Proposal is not publication.** Discovery and LLM assistance cannot self-authorize semantic truth. +3. **Deterministic gates matter.** Lifecycle and structural invariants are executable contracts. +4. **Product boundaries stay explicit.** Integrations use contracts rather than copying adjacent product responsibilities. +5. **Standards claims stay precise.** Drafts and emerging specifications are never presented as stable standards. +6. **Current source is the truth boundary.** Planned adapters and open-PR behavior are not described as already shipped. + +## Contributing + +Before changing the domain contract or lifecycle, read [`AGENTS.md`](AGENTS.md), the PRD/TRD, architecture, applicable ADRs, and the current product-gap baseline. Behavioral changes should preserve the repository's test-first and evidence-bound publication discipline and update the matching public contracts/documentation in the same change. + +No license claim is made here unless and until the repository contains an explicit license grant. \ No newline at end of file From 39b5b04cae0990bc1aa63412791218b661385f67 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 18:18:41 +0900 Subject: [PATCH 21/59] fix(ci): keep lockfile check immutable --- .github/workflows/product.yml | 42 ++++++++++++----------------------- 1 file changed, 14 insertions(+), 28 deletions(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index ad6f036c..f1f02a4c 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -1,4 +1,4 @@ -name: Product +name: Product CI on: pull_request: @@ -9,23 +9,17 @@ permissions: contents: read jobs: - rust-quality: + product: runs-on: ubuntu-latest steps: - - name: Checkout exact source revision - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/checkout@v4 with: - ref: ${{ github.event.pull_request.head.sha || github.sha }} - fetch-depth: 0 persist-credentials: false - - name: Verify exact source revision - env: - EXPECTED_SHA: ${{ github.event.pull_request.head.sha || github.sha }} - run: test "$(git rev-parse HEAD)" = "$EXPECTED_SHA" - - - name: Show pinned Rust toolchain - run: rustc --version && cargo --version + - uses: dtolnay/rust-toolchain@master + with: + toolchain: 1.98.0 + components: rustfmt, clippy - name: Format run: cargo fmt --all --check @@ -36,29 +30,21 @@ jobs: - name: Test run: cargo test --workspace - - name: Public documentation - env: - RUSTDOCFLAGS: -D warnings + - name: Rustdoc run: cargo doc --workspace --no-deps - - name: Install cargo-llvm-cov - uses: taiki-e/install-action@1ed6d7be6168f6c9046541087ff549b6bc581fdf # v2.87.2 - with: - tool: cargo-llvm-cov + - name: Coverage + run: bash scripts/check_coverage.sh - - name: Exact owned coverage - run: ./scripts/check_coverage.sh - - - name: Validate public JSON contract + - name: Validate semantic candidate schema run: | npx --yes ajv-cli@5.0.0 compile \ --spec=draft2020 \ -s contracts/semantic-candidate.schema.json - npx --yes ajv-cli@5.0.0 test \ + npx --yes ajv-cli@5.0.0 validate \ --spec=draft2020 \ -s contracts/semantic-candidate.schema.json \ - -d contracts/fixtures/semantic-candidate.valid.json \ - --valid + -d contracts/fixtures/semantic-candidate.valid.json npx --yes ajv-cli@5.0.0 test \ --spec=draft2020 \ -s contracts/semantic-candidate.schema.json \ @@ -72,7 +58,7 @@ jobs: - name: Lockfile freshness run: | - cargo generate-lockfile + cargo generate-lockfile --locked git ls-files --error-unmatch Cargo.lock >/dev/null test -z "$(git status --porcelain=v1 --untracked-files=all -- Cargo.lock)" From a70e08541f6783ca5c6e3405065a6e70e3f2d43c Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 18:19:08 +0900 Subject: [PATCH 22/59] fix(ci): preserve exact-head product workflow --- .github/workflows/product.yml | 40 +++++++++++++++++++++++------------ 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index f1f02a4c..fd39d719 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -1,4 +1,4 @@ -name: Product CI +name: Product on: pull_request: @@ -9,17 +9,23 @@ permissions: contents: read jobs: - product: + rust-quality: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - name: Checkout exact source revision + uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + fetch-depth: 0 persist-credentials: false - - uses: dtolnay/rust-toolchain@master - with: - toolchain: 1.98.0 - components: rustfmt, clippy + - name: Verify exact source revision + env: + EXPECTED_SHA: ${{ github.event.pull_request.head.sha || github.sha }} + run: test "$(git rev-parse HEAD)" = "$EXPECTED_SHA" + + - name: Show pinned Rust toolchain + run: rustc --version && cargo --version - name: Format run: cargo fmt --all --check @@ -30,21 +36,29 @@ jobs: - name: Test run: cargo test --workspace - - name: Rustdoc + - name: Public documentation + env: + RUSTDOCFLAGS: -D warnings run: cargo doc --workspace --no-deps - - name: Coverage - run: bash scripts/check_coverage.sh + - name: Install cargo-llvm-cov + uses: taiki-e/install-action@1ed6d7be6168f6c9046541087ff549b6bc581fdf # v2.87.2 + with: + tool: cargo-llvm-cov - - name: Validate semantic candidate schema + - name: Exact owned coverage + run: ./scripts/check_coverage.sh + + - name: Validate public JSON contract run: | npx --yes ajv-cli@5.0.0 compile \ --spec=draft2020 \ -s contracts/semantic-candidate.schema.json - npx --yes ajv-cli@5.0.0 validate \ + npx --yes ajv-cli@5.0.0 test \ --spec=draft2020 \ -s contracts/semantic-candidate.schema.json \ - -d contracts/fixtures/semantic-candidate.valid.json + -d contracts/fixtures/semantic-candidate.valid.json \ + --valid npx --yes ajv-cli@5.0.0 test \ --spec=draft2020 \ -s contracts/semantic-candidate.schema.json \ From 9ac9de961d1729c7536b8280d020b28af9954dfb Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 18:19:47 +0900 Subject: [PATCH 23/59] fix(domain): encapsulate semantic candidate invariants --- crates/conceptweave-domain/src/lib.rs | 77 +++++++++++++++++++-------- 1 file changed, 54 insertions(+), 23 deletions(-) diff --git a/crates/conceptweave-domain/src/lib.rs b/crates/conceptweave-domain/src/lib.rs index 4ff7d564..903429c3 100644 --- a/crates/conceptweave-domain/src/lib.rs +++ b/crates/conceptweave-domain/src/lib.rs @@ -123,10 +123,8 @@ impl EvidenceReference { /// A governed candidate for an ontology or semantic-layer artifact. /// -/// External consumers must not be able to mutate evidence or governance state -/// without a validated domain operation. This compile-fail example is an -/// executable boundary test: it must fail once the invariant is correctly -/// encapsulated. +/// External consumers cannot mutate evidence or governance state without a +/// validated domain operation. /// /// ```compile_fail /// use conceptweave_domain::{CandidateKind, EvidenceReference, SemanticCandidate}; @@ -137,16 +135,11 @@ impl EvidenceReference { /// ``` #[derive(Debug, Clone, PartialEq, Eq)] pub struct SemanticCandidate { - /// Stable candidate identifier within the owning tenant or workspace. - pub candidate_id: String, - /// Kind of semantic artifact proposed by the candidate. - pub kind: CandidateKind, - /// Current epistemic status. - pub truth_status: TruthStatus, - /// Current governance/publication state. - pub publication_state: PublicationState, - /// Exact source evidence supporting the candidate. - pub evidence: Vec, + candidate_id: String, + kind: CandidateKind, + truth_status: TruthStatus, + publication_state: PublicationState, + evidence: Vec, } impl SemanticCandidate { @@ -173,12 +166,40 @@ impl SemanticCandidate { }) } + /// Returns the stable candidate identifier. + pub fn candidate_id(&self) -> &str { + &self.candidate_id + } + + /// Returns the semantic artifact kind proposed by this candidate. + pub fn kind(&self) -> CandidateKind { + self.kind + } + + /// Returns the candidate's current epistemic status. + pub fn truth_status(&self) -> TruthStatus { + self.truth_status + } + + /// Returns the candidate's current governance/publication state. + pub fn publication_state(&self) -> PublicationState { + self.publication_state + } + + /// Returns the immutable evidence references supporting this candidate. + pub fn evidence(&self) -> &[EvidenceReference] { + &self.evidence + } + /// Moves the candidate through the fail-closed governance lifecycle. pub fn transition(&mut self, target: PublicationState) -> Result<(), ContractError> { let from = self.publication_state; if !ALLOWED_TRANSITIONS.contains(&(from, target)) { return Err(ContractError::InvalidTransition { from, to: target }); } + if target == PublicationState::Published && self.evidence.is_empty() { + return Err(ContractError::MissingEvidence); + } self.publication_state = target; self.truth_status = truth_for_state(target); Ok(()) @@ -293,21 +314,31 @@ mod tests { ); } + #[test] + fn candidate_accessors_expose_read_only_domain_state() { + let candidate = candidate(); + assert_eq!(candidate.candidate_id(), "candidate-1"); + assert_eq!(candidate.kind(), CandidateKind::Concept); + assert_eq!(candidate.truth_status(), TruthStatus::Inferred); + assert_eq!(candidate.publication_state(), PublicationState::Draft); + assert_eq!(candidate.evidence().len(), 1); + } + #[test] fn reviewed_candidate_can_be_published() { let mut candidate = candidate(); - assert_eq!(candidate.truth_status, TruthStatus::Inferred); + assert_eq!(candidate.truth_status(), TruthStatus::Inferred); assert!(!candidate.is_publishable()); candidate.transition(PublicationState::Proposed).unwrap(); - assert_eq!(candidate.truth_status, TruthStatus::Proposed); + assert_eq!(candidate.truth_status(), TruthStatus::Proposed); candidate.transition(PublicationState::Validated).unwrap(); - assert_eq!(candidate.truth_status, TruthStatus::Inferred); + assert_eq!(candidate.truth_status(), TruthStatus::Inferred); candidate.transition(PublicationState::Reviewed).unwrap(); - assert_eq!(candidate.truth_status, TruthStatus::Inferred); + assert_eq!(candidate.truth_status(), TruthStatus::Inferred); assert!(candidate.is_publishable()); candidate.transition(PublicationState::Published).unwrap(); - assert_eq!(candidate.truth_status, TruthStatus::Authoritative); + assert_eq!(candidate.truth_status(), TruthStatus::Authoritative); assert!(!candidate.is_publishable()); } @@ -322,7 +353,7 @@ mod tests { }) ); candidate.transition(PublicationState::Rejected).unwrap(); - assert_eq!(candidate.truth_status, TruthStatus::Rejected); + assert_eq!(candidate.truth_status(), TruthStatus::Rejected); assert_eq!( candidate.transition(PublicationState::Proposed), Err(ContractError::InvalidTransition { @@ -344,7 +375,7 @@ mod tests { candidate.transition(state).unwrap(); } candidate.transition(PublicationState::Superseded).unwrap(); - assert_eq!(candidate.truth_status, TruthStatus::Superseded); + assert_eq!(candidate.truth_status(), TruthStatus::Superseded); assert_eq!( candidate.transition(PublicationState::Rejected), Err(ContractError::InvalidTransition { @@ -371,8 +402,8 @@ mod tests { candidate.transition(PublicationState::Published), Err(ContractError::MissingEvidence) ); - assert_eq!(candidate.publication_state, PublicationState::Reviewed); - assert_eq!(candidate.truth_status, TruthStatus::Inferred); + assert_eq!(candidate.publication_state(), PublicationState::Reviewed); + assert_eq!(candidate.truth_status(), TruthStatus::Inferred); } #[test] From ec9b8125f86ab9b7545ba501f7b7c7c7d31c6783 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 18:20:05 +0900 Subject: [PATCH 24/59] test(domain): cover read-only publication invariants --- .../tests/publication_invariants.rs | 25 ++++++++----------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/crates/conceptweave-domain/tests/publication_invariants.rs b/crates/conceptweave-domain/tests/publication_invariants.rs index 8e9c0eb6..213a7d56 100644 --- a/crates/conceptweave-domain/tests/publication_invariants.rs +++ b/crates/conceptweave-domain/tests/publication_invariants.rs @@ -1,6 +1,4 @@ -use conceptweave_domain::{ - CandidateKind, ContractError, EvidenceReference, PublicationState, SemanticCandidate, -}; +use conceptweave_domain::{CandidateKind, EvidenceReference, PublicationState, SemanticCandidate}; fn reviewed_candidate() -> SemanticCandidate { let evidence = EvidenceReference::new("source-1", "sha256:abc", "public.orders").unwrap(); @@ -18,21 +16,20 @@ fn reviewed_candidate() -> SemanticCandidate { } #[test] -fn reviewed_candidate_without_evidence_is_not_publishable() { - let mut candidate = reviewed_candidate(); - candidate.evidence.clear(); +fn reviewed_candidate_exposes_evidence_read_only() { + let candidate = reviewed_candidate(); - assert!(!candidate.is_publishable()); + assert_eq!(candidate.publication_state(), PublicationState::Reviewed); + assert_eq!(candidate.evidence().len(), 1); + assert!(candidate.is_publishable()); } #[test] -fn reviewed_candidate_without_evidence_cannot_transition_to_published() { +fn reviewed_candidate_can_publish_only_through_validated_transition() { let mut candidate = reviewed_candidate(); - candidate.evidence.clear(); - assert_eq!( - candidate.transition(PublicationState::Published), - Err(ContractError::MissingEvidence) - ); - assert_eq!(candidate.publication_state, PublicationState::Reviewed); + candidate.transition(PublicationState::Published).unwrap(); + + assert_eq!(candidate.publication_state(), PublicationState::Published); + assert!(!candidate.is_publishable()); } From 51ca6f511c3742e53d2e7b8a8b6bb1f68d3809d7 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 20:07:58 +0900 Subject: [PATCH 25/59] docs: license ConceptWeave under Apache-2.0 --- LICENSE | 201 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 201 insertions(+) create mode 100644 LICENSE diff --git a/LICENSE b/LICENSE new file mode 100644 index 00000000..261eeb9e --- /dev/null +++ b/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. From 37d30a28399c2a8fc64dc92a91b89fb4eb7a74bf Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 20:08:08 +0900 Subject: [PATCH 26/59] docs: declare Apache-2.0 package license --- Cargo.toml | 1 + 1 file changed, 1 insertion(+) diff --git a/Cargo.toml b/Cargo.toml index b5e817a6..0eec8e8c 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -7,3 +7,4 @@ version = "0.1.0" edition = "2024" rust-version = "1.98" repository = "https://github.com/ContextualWisdomLab/ConceptWeave" +license = "Apache-2.0" From 3449cad2d5c8ee5314d3b3b9fd54b783f7fd5d5d Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 20:08:19 +0900 Subject: [PATCH 27/59] docs: align crate license metadata --- crates/conceptweave-domain/Cargo.toml | 1 + 1 file changed, 1 insertion(+) diff --git a/crates/conceptweave-domain/Cargo.toml b/crates/conceptweave-domain/Cargo.toml index 2a3f12e3..8f9ce7b2 100644 --- a/crates/conceptweave-domain/Cargo.toml +++ b/crates/conceptweave-domain/Cargo.toml @@ -4,6 +4,7 @@ version.workspace = true edition.workspace = true rust-version.workspace = true repository.workspace = true +license.workspace = true description = "Core domain contracts for governed ontology and semantic-layer engineering" [lib] From 34d5740a8b0cc33e45e4618cf67bfb6f70c97de0 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 20:08:45 +0900 Subject: [PATCH 28/59] docs: publish ConceptWeave license --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3d0d73ed..afb42260 100644 --- a/README.md +++ b/README.md @@ -167,4 +167,6 @@ The standards/research register and design implications live in [`docs/doctoring Before changing the domain contract or lifecycle, read [`AGENTS.md`](AGENTS.md), the PRD/TRD, architecture, applicable ADRs, and the current product-gap baseline. Behavioral changes should preserve the repository's test-first and evidence-bound publication discipline and update the matching public contracts/documentation in the same change. -No license claim is made here unless and until the repository contains an explicit license grant. \ No newline at end of file +## License + +ConceptWeave is licensed under the [Apache License 2.0](LICENSE). Third-party tools and future adapters retain their own licenses and must satisfy the repository's commercial-use and attribution policy before incorporation. From 7cc16b7eef6a6ac6465f39e8795388758031907d Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 20:27:19 +0900 Subject: [PATCH 29/59] fix: encapsulate semantic evidence references --- crates/conceptweave-domain/src/lib.rs | 115 ++++++++++++++++++++------ 1 file changed, 88 insertions(+), 27 deletions(-) diff --git a/crates/conceptweave-domain/src/lib.rs b/crates/conceptweave-domain/src/lib.rs index 903429c3..9d8a8e7d 100644 --- a/crates/conceptweave-domain/src/lib.rs +++ b/crates/conceptweave-domain/src/lib.rs @@ -84,14 +84,25 @@ pub enum PublicationState { } /// A stable reference to the evidence supporting a semantic candidate. +/// +/// Evidence identity is immutable outside this crate. Callers must construct a +/// reference through [`EvidenceReference::new`], which rejects blank identity +/// fields, and can inspect values only through read-only accessors. +/// +/// ```compile_fail +/// use conceptweave_domain::EvidenceReference; +/// +/// let reference = EvidenceReference { +/// source_id: "source-1".into(), +/// source_digest: "sha256:abc".into(), +/// location: "public.orders".into(), +/// }; +/// ``` #[derive(Debug, Clone, PartialEq, Eq)] pub struct EvidenceReference { - /// Stable identifier of the observed source snapshot or artifact. - pub source_id: String, - /// Content digest of the exact source revision used as evidence. - pub source_digest: String, - /// Human- and machine-readable location within the source artifact. - pub location: String, + source_id: String, + source_digest: String, + location: String, } impl EvidenceReference { @@ -101,23 +112,41 @@ impl EvidenceReference { source_digest: impl Into, location: impl Into, ) -> Result { - let source_id = source_id.into(); - let source_digest = source_digest.into(); - let location = location.into(); - if source_id.trim().is_empty() { + let reference = Self { + source_id: source_id.into(), + source_digest: source_digest.into(), + location: location.into(), + }; + reference.validate()?; + Ok(reference) + } + + /// Returns the stable identifier of the observed source snapshot or artifact. + pub fn source_id(&self) -> &str { + &self.source_id + } + + /// Returns the content digest of the exact source revision used as evidence. + pub fn source_digest(&self) -> &str { + &self.source_digest + } + + /// Returns the human- and machine-readable location within the source artifact. + pub fn location(&self) -> &str { + &self.location + } + + fn validate(&self) -> Result<(), ContractError> { + if self.source_id.trim().is_empty() { return Err(ContractError::EmptyField("source_id")); } - if source_digest.trim().is_empty() { + if self.source_digest.trim().is_empty() { return Err(ContractError::EmptyField("source_digest")); } - if location.trim().is_empty() { + if self.location.trim().is_empty() { return Err(ContractError::EmptyField("location")); } - Ok(Self { - source_id, - source_digest, - location, - }) + Ok(()) } } @@ -143,7 +172,7 @@ pub struct SemanticCandidate { } impl SemanticCandidate { - /// Creates an inferred draft candidate with at least one evidence reference. + /// Creates an inferred draft candidate with at least one valid evidence reference. pub fn new( candidate_id: impl Into, kind: CandidateKind, @@ -153,9 +182,7 @@ impl SemanticCandidate { if candidate_id.trim().is_empty() { return Err(ContractError::EmptyField("candidate_id")); } - if evidence.is_empty() { - return Err(ContractError::MissingEvidence); - } + validate_evidence(&evidence)?; let publication_state = PublicationState::Draft; Ok(Self { candidate_id, @@ -197,8 +224,8 @@ impl SemanticCandidate { if !ALLOWED_TRANSITIONS.contains(&(from, target)) { return Err(ContractError::InvalidTransition { from, to: target }); } - if target == PublicationState::Published && self.evidence.is_empty() { - return Err(ContractError::MissingEvidence); + if target == PublicationState::Published { + validate_evidence(&self.evidence)?; } self.publication_state = target; self.truth_status = truth_for_state(target); @@ -207,8 +234,19 @@ impl SemanticCandidate { /// Returns whether the candidate is immediately eligible for publication. pub fn is_publishable(&self) -> bool { - self.publication_state == PublicationState::Reviewed && !self.evidence.is_empty() + self.publication_state == PublicationState::Reviewed + && validate_evidence(&self.evidence).is_ok() + } +} + +fn validate_evidence(evidence: &[EvidenceReference]) -> Result<(), ContractError> { + if evidence.is_empty() { + return Err(ContractError::MissingEvidence); + } + for reference in evidence { + reference.validate()?; } + Ok(()) } const ALLOWED_TRANSITIONS: &[(PublicationState, PublicationState)] = &[ @@ -281,9 +319,9 @@ mod tests { #[test] fn evidence_reference_accepts_valid_values() { let reference = evidence(); - assert_eq!(reference.source_id, "source-1"); - assert_eq!(reference.source_digest, "sha256:abc"); - assert_eq!(reference.location, "schema.orders.total"); + assert_eq!(reference.source_id(), "source-1"); + assert_eq!(reference.source_digest(), "sha256:abc"); + assert_eq!(reference.location(), "schema.orders.total"); } #[test] @@ -322,6 +360,7 @@ mod tests { assert_eq!(candidate.truth_status(), TruthStatus::Inferred); assert_eq!(candidate.publication_state(), PublicationState::Draft); assert_eq!(candidate.evidence().len(), 1); + assert_eq!(candidate.evidence()[0].source_id(), "source-1"); } #[test] @@ -406,6 +445,28 @@ mod tests { assert_eq!(candidate.truth_status(), TruthStatus::Inferred); } + #[test] + fn publication_revalidates_each_evidence_reference() { + let mut candidate = candidate(); + for state in [ + PublicationState::Proposed, + PublicationState::Validated, + PublicationState::Reviewed, + ] { + candidate.transition(state).unwrap(); + } + + candidate.evidence[0].source_id = " ".into(); + + assert!(!candidate.is_publishable()); + assert_eq!( + candidate.transition(PublicationState::Published), + Err(ContractError::EmptyField("source_id")) + ); + assert_eq!(candidate.publication_state(), PublicationState::Reviewed); + assert_eq!(candidate.truth_status(), TruthStatus::Inferred); + } + #[test] fn contract_errors_explain_the_failure() { assert_eq!( From 6520478a0a85637e729ce5f713dc64f802f6695f Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 21:50:29 +0900 Subject: [PATCH 30/59] fix(ci): repair exact-head format and bound Product queue --- .github/workflows/product.yml | 4 ++++ crates/conceptweave-domain/src/lib.rs | 4 +++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index fd39d719..d359330a 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -5,6 +5,10 @@ on: push: branches: [main] +concurrency: + group: conceptweave-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + permissions: contents: read diff --git a/crates/conceptweave-domain/src/lib.rs b/crates/conceptweave-domain/src/lib.rs index 9d8a8e7d..383985dd 100644 --- a/crates/conceptweave-domain/src/lib.rs +++ b/crates/conceptweave-domain/src/lib.rs @@ -293,7 +293,9 @@ impl fmt::Display for ContractError { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { match self { Self::EmptyField(field) => write!(formatter, "required field `{field}` is blank"), - Self::MissingEvidence => write!(formatter, "semantic candidates require source evidence"), + Self::MissingEvidence => { + write!(formatter, "semantic candidates require source evidence") + } Self::InvalidTransition { from, to } => write!( formatter, "publication transition from {from:?} to {to:?} is not permitted" From 971ddcbd27328e9f6d53c61bb16f7d5a05c97435 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 21:53:57 +0900 Subject: [PATCH 31/59] fix(ci): move checkout to Node 24 runtime --- .github/workflows/product.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index d359330a..48bed2e7 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -17,7 +17,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout exact source revision - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 with: ref: ${{ github.event.pull_request.head.sha || github.sha }} fetch-depth: 0 From 84e205da2e099c4b4a36eb8da014a0ddf769f970 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 22:49:37 +0900 Subject: [PATCH 32/59] test(ci): specify explicit runner admission contract --- scripts/check_ci_contract.py | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 scripts/check_ci_contract.py diff --git a/scripts/check_ci_contract.py b/scripts/check_ci_contract.py new file mode 100644 index 00000000..eae64820 --- /dev/null +++ b/scripts/check_ci_contract.py @@ -0,0 +1,36 @@ +"""Fail closed when the Product workflow regresses to a starved runner selector.""" + +from __future__ import annotations + +from pathlib import Path + + +WORKFLOW_PATH = Path(".github/workflows/product.yml") + + +def main() -> int: + """Validate queue-admission and supersession invariants for Product CI.""" + workflow = WORKFLOW_PATH.read_text(encoding="utf-8") + + required_fragments = ( + "runs-on: ubuntu-24.04", + "cancel-in-progress: true", + "actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0", + ) + missing = [fragment for fragment in required_fragments if fragment not in workflow] + if missing: + raise SystemExit( + "Product CI contract missing required fragment(s): " + ", ".join(missing) + ) + + if "runs-on: ubuntu-latest" in workflow: + raise SystemExit( + "Product CI must not use ubuntu-latest while the organization runner " + "incident demonstrates selective floating-image starvation" + ) + + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From 65ad7df2f611b951c4e60bd4654488f7da64dcbb Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 22:50:05 +0900 Subject: [PATCH 33/59] fix(ci): pin Product runner to Ubuntu 24.04 --- .github/workflows/product.yml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index 48bed2e7..b2e4b6e3 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -14,7 +14,7 @@ permissions: jobs: rust-quality: - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 steps: - name: Checkout exact source revision uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 @@ -28,6 +28,9 @@ jobs: EXPECTED_SHA: ${{ github.event.pull_request.head.sha || github.sha }} run: test "$(git rev-parse HEAD)" = "$EXPECTED_SHA" + - name: Validate Product CI contract + run: python3 scripts/check_ci_contract.py + - name: Show pinned Rust toolchain run: rustc --version && cargo --version From 625dae5d9f01810148c3329dbab381ed02b34d0c Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 22:53:37 +0900 Subject: [PATCH 34/59] fix(ci): use pinned nightly for branch coverage --- .github/workflows/product.yml | 5 +++++ scripts/check_ci_contract.py | 10 ++++++---- scripts/check_coverage.sh | 8 +++++++- 3 files changed, 18 insertions(+), 5 deletions(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index b2e4b6e3..d49caaf1 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -15,6 +15,8 @@ permissions: jobs: rust-quality: runs-on: ubuntu-24.04 + env: + COVERAGE_TOOLCHAIN: nightly-2026-08-20 steps: - name: Checkout exact source revision uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 @@ -53,6 +55,9 @@ jobs: with: tool: cargo-llvm-cov + - name: Install pinned branch-coverage toolchain + run: rustup toolchain install "$COVERAGE_TOOLCHAIN" --profile minimal --component llvm-tools-preview + - name: Exact owned coverage run: ./scripts/check_coverage.sh diff --git a/scripts/check_ci_contract.py b/scripts/check_ci_contract.py index eae64820..e0e7130e 100644 --- a/scripts/check_ci_contract.py +++ b/scripts/check_ci_contract.py @@ -1,4 +1,4 @@ -"""Fail closed when the Product workflow regresses to a starved runner selector.""" +"""Fail closed when Product CI regresses on runner or coverage toolchain identity.""" from __future__ import annotations @@ -9,13 +9,15 @@ def main() -> int: - """Validate queue-admission and supersession invariants for Product CI.""" + """Validate queue-admission, supersession, and branch-coverage invariants.""" workflow = WORKFLOW_PATH.read_text(encoding="utf-8") required_fragments = ( "runs-on: ubuntu-24.04", "cancel-in-progress: true", "actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0", + "COVERAGE_TOOLCHAIN: nightly-2026-08-20", + 'rustup toolchain install "$COVERAGE_TOOLCHAIN" --profile minimal --component llvm-tools-preview', ) missing = [fragment for fragment in required_fragments if fragment not in workflow] if missing: @@ -25,8 +27,8 @@ def main() -> int: if "runs-on: ubuntu-latest" in workflow: raise SystemExit( - "Product CI must not use ubuntu-latest while the organization runner " - "incident demonstrates selective floating-image starvation" + "Product CI must not use ubuntu-latest while current organization " + "evidence demonstrates selective floating-image starvation" ) return 0 diff --git a/scripts/check_coverage.sh b/scripts/check_coverage.sh index 8bba62e0..ed9d1511 100755 --- a/scripts/check_coverage.sh +++ b/scripts/check_coverage.sh @@ -1,7 +1,13 @@ #!/usr/bin/env bash set -euo pipefail -cargo llvm-cov --workspace --branch --json --output-path coverage.json +coverage_toolchain="${COVERAGE_TOOLCHAIN:-nightly-2026-08-20}" + +cargo "+${coverage_toolchain}" llvm-cov \ + --workspace \ + --branch \ + --json \ + --output-path coverage.json jq -e ' .data[0].totals.lines.percent == 100 and From 616db913cb9a18b7660577b3b739b9d01b649dca Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 22:59:11 +0900 Subject: [PATCH 35/59] test(governance): require authority before steward review --- .../tests/publication_invariants.rs | 50 ++++++++++++------- 1 file changed, 32 insertions(+), 18 deletions(-) diff --git a/crates/conceptweave-domain/tests/publication_invariants.rs b/crates/conceptweave-domain/tests/publication_invariants.rs index 213a7d56..f9571a61 100644 --- a/crates/conceptweave-domain/tests/publication_invariants.rs +++ b/crates/conceptweave-domain/tests/publication_invariants.rs @@ -1,35 +1,49 @@ -use conceptweave_domain::{CandidateKind, EvidenceReference, PublicationState, SemanticCandidate}; +use conceptweave_domain::{ + CandidateKind, ContractError, EvidenceReference, PublicationState, SemanticCandidate, +}; -fn reviewed_candidate() -> SemanticCandidate { +fn validated_candidate() -> SemanticCandidate { let evidence = EvidenceReference::new("source-1", "sha256:abc", "public.orders").unwrap(); let mut candidate = SemanticCandidate::new("candidate-1", CandidateKind::Concept, vec![evidence]).unwrap(); - for state in [ - PublicationState::Proposed, - PublicationState::Validated, - PublicationState::Reviewed, - ] { - candidate.transition(state).unwrap(); - } + candidate.transition(PublicationState::Proposed).unwrap(); + candidate.transition(PublicationState::Validated).unwrap(); candidate } #[test] -fn reviewed_candidate_exposes_evidence_read_only() { - let candidate = reviewed_candidate(); +fn evidence_remains_read_only_before_governance_review() { + let candidate = validated_candidate(); - assert_eq!(candidate.publication_state(), PublicationState::Reviewed); + assert_eq!(candidate.publication_state(), PublicationState::Validated); assert_eq!(candidate.evidence().len(), 1); - assert!(candidate.is_publishable()); + assert!(!candidate.is_publishable()); } #[test] -fn reviewed_candidate_can_publish_only_through_validated_transition() { - let mut candidate = reviewed_candidate(); +fn external_callers_cannot_enter_reviewed_state_without_governance_authority() { + let mut candidate = validated_candidate(); - candidate.transition(PublicationState::Published).unwrap(); + assert_eq!( + candidate.transition(PublicationState::Reviewed), + Err(ContractError::GovernanceAuthorizationRequired { + target: PublicationState::Reviewed, + }) + ); + assert_eq!(candidate.publication_state(), PublicationState::Validated); +} - assert_eq!(candidate.publication_state(), PublicationState::Published); - assert!(!candidate.is_publishable()); +#[test] +fn external_callers_cannot_publish_by_skipping_governance() { + let mut candidate = validated_candidate(); + + assert_eq!( + candidate.transition(PublicationState::Published), + Err(ContractError::InvalidTransition { + from: PublicationState::Validated, + to: PublicationState::Published, + }) + ); + assert_eq!(candidate.publication_state(), PublicationState::Validated); } From d5b8120fb7f188efa86119b4c3312b99af173082 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 23:02:11 +0900 Subject: [PATCH 36/59] fix(governance): close unauthorized publication state escalation --- crates/conceptweave-domain/src/lib.rs | 159 ++++++++++++++++---------- 1 file changed, 98 insertions(+), 61 deletions(-) diff --git a/crates/conceptweave-domain/src/lib.rs b/crates/conceptweave-domain/src/lib.rs index 383985dd..d994dbb5 100644 --- a/crates/conceptweave-domain/src/lib.rs +++ b/crates/conceptweave-domain/src/lib.rs @@ -153,7 +153,10 @@ impl EvidenceReference { /// A governed candidate for an ontology or semantic-layer artifact. /// /// External consumers cannot mutate evidence or governance state without a -/// validated domain operation. +/// validated domain operation. The public transition API deliberately stops at +/// the semantic-steward boundary: entering `Reviewed`, entering `Published`, or +/// superseding/rejecting an already reviewed artifact requires the separate +/// Governance & Publication context to establish authority first. /// /// ```compile_fail /// use conceptweave_domain::{CandidateKind, EvidenceReference, SemanticCandidate}; @@ -218,21 +221,29 @@ impl SemanticCandidate { &self.evidence } - /// Moves the candidate through the fail-closed governance lifecycle. + /// Moves a candidate through transitions that do not require steward authority. + /// + /// Deterministic discovery and validation code may propose, validate, or + /// reject a candidate. Crossing into a steward-reviewed or published state, + /// or changing an already reviewed/published artifact, fails closed until + /// the Governance & Publication bounded context supplies an authorized path. pub fn transition(&mut self, target: PublicationState) -> Result<(), ContractError> { let from = self.publication_state; if !ALLOWED_TRANSITIONS.contains(&(from, target)) { return Err(ContractError::InvalidTransition { from, to: target }); } - if target == PublicationState::Published { - validate_evidence(&self.evidence)?; + if requires_governance_authorization(from, target) { + return Err(ContractError::GovernanceAuthorizationRequired { target }); } self.publication_state = target; self.truth_status = truth_for_state(target); Ok(()) } - /// Returns whether the candidate is immediately eligible for publication. + /// Returns whether the current reviewed candidate has valid publication evidence. + /// + /// This is an eligibility check only. It does not grant steward authority or + /// publish the candidate. pub fn is_publishable(&self) -> bool { self.publication_state == PublicationState::Reviewed && validate_evidence(&self.evidence).is_ok() @@ -249,6 +260,19 @@ fn validate_evidence(evidence: &[EvidenceReference]) -> Result<(), ContractError Ok(()) } +fn requires_governance_authorization( + from: PublicationState, + target: PublicationState, +) -> bool { + matches!( + from, + PublicationState::Reviewed | PublicationState::Published + ) || matches!( + target, + PublicationState::Reviewed | PublicationState::Published | PublicationState::Superseded + ) +} + const ALLOWED_TRANSITIONS: &[(PublicationState, PublicationState)] = &[ (PublicationState::Draft, PublicationState::Proposed), (PublicationState::Draft, PublicationState::Rejected), @@ -280,6 +304,11 @@ pub enum ContractError { EmptyField(&'static str), /// A candidate was created without any supporting evidence. MissingEvidence, + /// A caller attempted to cross a steward-governed lifecycle boundary. + GovernanceAuthorizationRequired { + /// Requested state that requires the Governance & Publication context. + target: PublicationState, + }, /// A governance state transition attempted to skip or reverse required review. InvalidTransition { /// State before the rejected transition. @@ -296,6 +325,10 @@ impl fmt::Display for ContractError { Self::MissingEvidence => { write!(formatter, "semantic candidates require source evidence") } + Self::GovernanceAuthorizationRequired { target } => write!( + formatter, + "publication state {target:?} requires authorized governance" + ), Self::InvalidTransition { from, to } => write!( formatter, "publication transition from {from:?} to {to:?} is not permitted" @@ -318,6 +351,13 @@ mod tests { SemanticCandidate::new("candidate-1", CandidateKind::Concept, vec![evidence()]).unwrap() } + fn reviewed_candidate_for_governance_test() -> SemanticCandidate { + let mut candidate = candidate(); + candidate.publication_state = PublicationState::Reviewed; + candidate.truth_status = truth_for_state(PublicationState::Reviewed); + candidate + } + #[test] fn evidence_reference_accepts_valid_values() { let reference = evidence(); @@ -366,21 +406,19 @@ mod tests { } #[test] - fn reviewed_candidate_can_be_published() { + fn deterministic_lifecycle_stops_at_governance_boundary() { let mut candidate = candidate(); - assert_eq!(candidate.truth_status(), TruthStatus::Inferred); - assert!(!candidate.is_publishable()); - candidate.transition(PublicationState::Proposed).unwrap(); - assert_eq!(candidate.truth_status(), TruthStatus::Proposed); candidate.transition(PublicationState::Validated).unwrap(); + + assert_eq!( + candidate.transition(PublicationState::Reviewed), + Err(ContractError::GovernanceAuthorizationRequired { + target: PublicationState::Reviewed, + }) + ); + assert_eq!(candidate.publication_state(), PublicationState::Validated); assert_eq!(candidate.truth_status(), TruthStatus::Inferred); - candidate.transition(PublicationState::Reviewed).unwrap(); - assert_eq!(candidate.truth_status(), TruthStatus::Inferred); - assert!(candidate.is_publishable()); - candidate.transition(PublicationState::Published).unwrap(); - assert_eq!(candidate.truth_status(), TruthStatus::Authoritative); - assert!(!candidate.is_publishable()); } #[test] @@ -405,68 +443,60 @@ mod tests { } #[test] - fn published_candidate_can_only_be_superseded() { - let mut candidate = candidate(); - for state in [ - PublicationState::Proposed, - PublicationState::Validated, - PublicationState::Reviewed, - PublicationState::Published, - ] { - candidate.transition(state).unwrap(); - } - candidate.transition(PublicationState::Superseded).unwrap(); - assert_eq!(candidate.truth_status(), TruthStatus::Superseded); + fn reviewed_and_published_state_changes_require_governance() { + let mut candidate = reviewed_candidate_for_governance_test(); + assert!(candidate.is_publishable()); + assert_eq!( + candidate.transition(PublicationState::Published), + Err(ContractError::GovernanceAuthorizationRequired { + target: PublicationState::Published, + }) + ); assert_eq!( candidate.transition(PublicationState::Rejected), - Err(ContractError::InvalidTransition { - from: PublicationState::Superseded, - to: PublicationState::Rejected, + Err(ContractError::GovernanceAuthorizationRequired { + target: PublicationState::Rejected, + }) + ); + + candidate.publication_state = PublicationState::Published; + candidate.truth_status = truth_for_state(PublicationState::Published); + assert_eq!(candidate.truth_status(), TruthStatus::Authoritative); + assert_eq!( + candidate.transition(PublicationState::Superseded), + Err(ContractError::GovernanceAuthorizationRequired { + target: PublicationState::Superseded, }) ); } #[test] - fn publication_rechecks_evidence_at_authority_boundary() { - let mut candidate = candidate(); - for state in [ - PublicationState::Proposed, - PublicationState::Validated, - PublicationState::Reviewed, - ] { - candidate.transition(state).unwrap(); - } + fn publication_eligibility_rechecks_evidence() { + let mut candidate = reviewed_candidate_for_governance_test(); + assert!(candidate.is_publishable()); candidate.evidence.clear(); - - assert_eq!( - candidate.transition(PublicationState::Published), - Err(ContractError::MissingEvidence) - ); - assert_eq!(candidate.publication_state(), PublicationState::Reviewed); - assert_eq!(candidate.truth_status(), TruthStatus::Inferred); + assert!(!candidate.is_publishable()); } #[test] - fn publication_revalidates_each_evidence_reference() { - let mut candidate = candidate(); - for state in [ - PublicationState::Proposed, - PublicationState::Validated, - PublicationState::Reviewed, - ] { - candidate.transition(state).unwrap(); - } - + fn publication_eligibility_revalidates_each_evidence_reference() { + let mut candidate = reviewed_candidate_for_governance_test(); candidate.evidence[0].source_id = " ".into(); assert!(!candidate.is_publishable()); + } + + #[test] + fn truth_mapping_preserves_published_and_superseded_semantics() { assert_eq!( - candidate.transition(PublicationState::Published), - Err(ContractError::EmptyField("source_id")) + truth_for_state(PublicationState::Published), + TruthStatus::Authoritative + ); + assert_eq!( + truth_for_state(PublicationState::Superseded), + TruthStatus::Superseded ); - assert_eq!(candidate.publication_state(), PublicationState::Reviewed); - assert_eq!(candidate.truth_status(), TruthStatus::Inferred); } #[test] @@ -479,6 +509,13 @@ mod tests { ContractError::MissingEvidence.to_string(), "semantic candidates require source evidence" ); + assert_eq!( + ContractError::GovernanceAuthorizationRequired { + target: PublicationState::Reviewed, + } + .to_string(), + "publication state Reviewed requires authorized governance" + ); assert_eq!( ContractError::InvalidTransition { from: PublicationState::Draft, From e704ce5fe1f489db1c83f29b385671bd18794d0b Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 23:05:45 +0900 Subject: [PATCH 37/59] style(rust): apply governance boundary formatting --- crates/conceptweave-domain/src/lib.rs | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/crates/conceptweave-domain/src/lib.rs b/crates/conceptweave-domain/src/lib.rs index d994dbb5..968bccb7 100644 --- a/crates/conceptweave-domain/src/lib.rs +++ b/crates/conceptweave-domain/src/lib.rs @@ -260,10 +260,7 @@ fn validate_evidence(evidence: &[EvidenceReference]) -> Result<(), ContractError Ok(()) } -fn requires_governance_authorization( - from: PublicationState, - target: PublicationState, -) -> bool { +fn requires_governance_authorization(from: PublicationState, target: PublicationState) -> bool { matches!( from, PublicationState::Reviewed | PublicationState::Published From 17a0662675da7c4c3f6df2bff8021d9dcd4f0016 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 23:08:31 +0900 Subject: [PATCH 38/59] chore(ci): expose exact coverage gaps --- scripts/check_coverage.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/scripts/check_coverage.sh b/scripts/check_coverage.sh index ed9d1511..ef549f97 100755 --- a/scripts/check_coverage.sh +++ b/scripts/check_coverage.sh @@ -9,6 +9,18 @@ cargo "+${coverage_toolchain}" llvm-cov \ --json \ --output-path coverage.json +jq '.data[0].totals' coverage.json +jq -r ' + .data[0].files[] + | select( + .summary.lines.percent != 100 + or .summary.functions.percent != 100 + or .summary.regions.percent != 100 + or .summary.branches.percent != 100 + ) + | "COVERAGE_GAP file=\(.filename) lines=\(.summary.lines.percent) functions=\(.summary.functions.percent) regions=\(.summary.regions.percent) branches=\(.summary.branches.percent)" +' coverage.json + jq -e ' .data[0].totals.lines.percent == 100 and .data[0].totals.functions.percent == 100 and From f9876c6f24122d78627dac7771b76b307c1bb95c Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 23:15:57 +0900 Subject: [PATCH 39/59] test: cover draft publication eligibility branch --- .../tests/publication_eligibility.rs | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 crates/conceptweave-domain/tests/publication_eligibility.rs diff --git a/crates/conceptweave-domain/tests/publication_eligibility.rs b/crates/conceptweave-domain/tests/publication_eligibility.rs new file mode 100644 index 00000000..a3c16956 --- /dev/null +++ b/crates/conceptweave-domain/tests/publication_eligibility.rs @@ -0,0 +1,19 @@ +use conceptweave_domain::{CandidateKind, EvidenceReference, SemanticCandidate}; + +#[test] +fn draft_candidate_is_not_publishable_before_governance_review() { + let evidence = EvidenceReference::new( + "source-1", + "sha256:abc", + "schema.orders.total", + ) + .expect("valid evidence fixture"); + let candidate = SemanticCandidate::new( + "candidate-1", + CandidateKind::Concept, + vec![evidence], + ) + .expect("valid semantic candidate fixture"); + + assert!(!candidate.is_publishable()); +} From bf9c07c386cc0ef7bb6a3b51b45b57bd63a53926 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 23:18:53 +0900 Subject: [PATCH 40/59] fix: format publication eligibility regression --- .../tests/publication_eligibility.rs | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/crates/conceptweave-domain/tests/publication_eligibility.rs b/crates/conceptweave-domain/tests/publication_eligibility.rs index a3c16956..73cdd0a2 100644 --- a/crates/conceptweave-domain/tests/publication_eligibility.rs +++ b/crates/conceptweave-domain/tests/publication_eligibility.rs @@ -2,18 +2,10 @@ use conceptweave_domain::{CandidateKind, EvidenceReference, SemanticCandidate}; #[test] fn draft_candidate_is_not_publishable_before_governance_review() { - let evidence = EvidenceReference::new( - "source-1", - "sha256:abc", - "schema.orders.total", - ) - .expect("valid evidence fixture"); - let candidate = SemanticCandidate::new( - "candidate-1", - CandidateKind::Concept, - vec![evidence], - ) - .expect("valid semantic candidate fixture"); + let evidence = EvidenceReference::new("source-1", "sha256:abc", "schema.orders.total") + .expect("valid evidence fixture"); + let candidate = SemanticCandidate::new("candidate-1", CandidateKind::Concept, vec![evidence]) + .expect("valid semantic candidate fixture"); assert!(!candidate.is_publishable()); } From 682d351796d113652ec0918e40b54078ea4e420e Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 23:22:52 +0900 Subject: [PATCH 41/59] chore(ci): report exact uncovered branch coordinates --- scripts/check_coverage.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/scripts/check_coverage.sh b/scripts/check_coverage.sh index ef549f97..63f50fec 100755 --- a/scripts/check_coverage.sh +++ b/scripts/check_coverage.sh @@ -21,6 +21,14 @@ jq -r ' | "COVERAGE_GAP file=\(.filename) lines=\(.summary.lines.percent) functions=\(.summary.functions.percent) regions=\(.summary.regions.percent) branches=\(.summary.branches.percent)" ' coverage.json +jq -r ' + .data[0].files[] + | .filename as $file + | (.branches // [])[] + | select((.[4] // 0) == 0 or (.[5] // 0) == 0) + | "BRANCH_GAP file=\($file) start=\(.[0]):\(.[1]) end=\(.[2]):\(.[3]) true_count=\(.[4]) false_count=\(.[5])" +' coverage.json + jq -e ' .data[0].totals.lines.percent == 100 and .data[0].totals.functions.percent == 100 and From 3510c17cb416f24340bf210c2466ff84d24b8e61 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 23:28:52 +0900 Subject: [PATCH 42/59] fix(ci): aggregate branch outcomes by source condition --- scripts/check_coverage.sh | 55 ++++++++++++++++++++++++++++++++------- 1 file changed, 45 insertions(+), 10 deletions(-) diff --git a/scripts/check_coverage.sh b/scripts/check_coverage.sh index 63f50fec..79fe6549 100755 --- a/scripts/check_coverage.sh +++ b/scripts/check_coverage.sh @@ -16,22 +16,57 @@ jq -r ' .summary.lines.percent != 100 or .summary.functions.percent != 100 or .summary.regions.percent != 100 - or .summary.branches.percent != 100 ) - | "COVERAGE_GAP file=\(.filename) lines=\(.summary.lines.percent) functions=\(.summary.functions.percent) regions=\(.summary.regions.percent) branches=\(.summary.branches.percent)" + | "COVERAGE_GAP file=\(.filename) lines=\(.summary.lines.percent) functions=\(.summary.functions.percent) regions=\(.summary.regions.percent)" ' coverage.json +jq ' + [ + .data[0].files[] + | .filename as $file + | (.branches // [])[] + | { + file: $file, + line_start: .[0], + column_start: .[1], + line_end: .[2], + column_end: .[3], + true_count: .[4], + false_count: .[5] + } + ] + | sort_by(.file, .line_start, .column_start, .line_end, .column_end) + | group_by([.file, .line_start, .column_start, .line_end, .column_end]) + | map({ + file: .[0].file, + line_start: .[0].line_start, + column_start: .[0].column_start, + line_end: .[0].line_end, + column_end: .[0].column_end, + true_count: (map(.true_count) | add), + false_count: (map(.false_count) | add) + }) +' coverage.json > source-branches.json + +jq ' + { + count: (length * 2), + covered: ([.[] | (.true_count > 0), (.false_count > 0) | select(.)] | length), + notcovered: ([.[] | (.true_count == 0), (.false_count == 0) | select(.)] | length) + } + | .percent = (if .count == 0 then 100 else (.covered * 100 / .count) end) +' source-branches.json + jq -r ' - .data[0].files[] - | .filename as $file - | (.branches // [])[] - | select((.[4] // 0) == 0 or (.[5] // 0) == 0) - | "BRANCH_GAP file=\($file) start=\(.[0]):\(.[1]) end=\(.[2]):\(.[3]) true_count=\(.[4]) false_count=\(.[5])" -' coverage.json + .[] + | select(.true_count == 0 or .false_count == 0) + | "BRANCH_GAP file=\(.file) start=\(.line_start):\(.column_start) end=\(.line_end):\(.column_end) true_count=\(.true_count) false_count=\(.false_count)" +' source-branches.json jq -e ' .data[0].totals.lines.percent == 100 and .data[0].totals.functions.percent == 100 and - .data[0].totals.regions.percent == 100 and - .data[0].totals.branches.percent == 100 + .data[0].totals.regions.percent == 100 ' coverage.json >/dev/null + +jq -e 'all(.[]; .true_count > 0 and .false_count > 0)' source-branches.json >/dev/null From 0305f1c474a0d49d9764120239eacefc4c2b7d87 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 23:30:12 +0900 Subject: [PATCH 43/59] fix: make governance coverage branches reachable --- crates/conceptweave-domain/src/lib.rs | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/crates/conceptweave-domain/src/lib.rs b/crates/conceptweave-domain/src/lib.rs index 968bccb7..b4a8aa6e 100644 --- a/crates/conceptweave-domain/src/lib.rs +++ b/crates/conceptweave-domain/src/lib.rs @@ -262,11 +262,11 @@ fn validate_evidence(evidence: &[EvidenceReference]) -> Result<(), ContractError fn requires_governance_authorization(from: PublicationState, target: PublicationState) -> bool { matches!( - from, - PublicationState::Reviewed | PublicationState::Published - ) || matches!( - target, - PublicationState::Reviewed | PublicationState::Published | PublicationState::Superseded + (from, target), + (PublicationState::Validated, PublicationState::Reviewed) + | (PublicationState::Reviewed, PublicationState::Published) + | (PublicationState::Reviewed, PublicationState::Rejected) + | (PublicationState::Published, PublicationState::Superseded) ) } From 5cd7d1de742fe34aa99900641cc8b124e7c65f9e Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 23:31:21 +0900 Subject: [PATCH 44/59] fix(ci): clean coverage evidence before tree check --- scripts/check_coverage.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/check_coverage.sh b/scripts/check_coverage.sh index 79fe6549..2f691898 100755 --- a/scripts/check_coverage.sh +++ b/scripts/check_coverage.sh @@ -2,6 +2,7 @@ set -euo pipefail coverage_toolchain="${COVERAGE_TOOLCHAIN:-nightly-2026-08-20}" +trap 'rm -f coverage.json source-branches.json' EXIT cargo "+${coverage_toolchain}" llvm-cov \ --workspace \ From bba351b77bf5f1ab5cfd55979fbb2bd158f78b81 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 2 Sep 2026 00:37:50 +0900 Subject: [PATCH 45/59] docs(gap): record merged central runner repair --- docs/product-technical-gap-baseline.md | 35 +++++++++++++++++--------- 1 file changed, 23 insertions(+), 12 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index c99d8269..cbc42906 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1,21 +1,29 @@ # Product / Technical Gap Baseline -**Snapshot:** 2026-09-01 +**Snapshot:** 2026-09-02 ## Shipped on protected `main` Only the repository bootstrap README exists before the foundation PR. No production capability is claimed. -## Active foundation slice +## Active foundation slice — PR #1 -| Area | Status | Evidence / next action | +The exact PR head is the live GitHub branch head; check evidence is valid only for that unchanged SHA. The predecessor head `5cd7d1de742fe34aa99900641cc8b124e7c65f9e` reached terminal repository-owned Product success (exact checkout, fmt, Clippy, tests, rustdoc, exact owned coverage, JSON contract, lock freshness, clean tree). This baseline update intentionally creates a newer documentation-only head so all checks must be re-established rather than transferred. + +| Area | Status | Evidence / action / next verification | | --- | --- | --- | -| Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define Semantic Model Engineering and CWL boundaries. | -| Truth/publication lifecycle | ACTIVE_PR | Rust domain lifecycle defines Draft -> Proposed -> Validated -> Reviewed -> Published. Draft 2020-12 JSON Schema enforces candidate shape, non-blank evidence identities, and Published -> Authoritative consistency; lifecycle history/pre-Reviewed publication is not a JSON-Schema responsibility. A test-first regression currently requires Reviewed -> Published to fail if required evidence is absent before this slice can be merge-ready. | +| Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define Semantic Model Engineering and CWL boundaries. Revalidate on the new exact head. | +| Truth/publication lifecycle | ACTIVE_PR | Rust domain lifecycle defines Draft -> Proposed -> Validated -> Reviewed -> Published with explicit governance authorization at steward/publication boundaries; candidate JSON Schema enforces public structural shape and Published -> Authoritative consistency. The earlier missing-evidence and branch-coverage defects were repaired and proven by predecessor exact-head Product success. | | Rust baseline | ACTIVE_PR | Rust 1.98.0 workspace, unsafe forbidden, public docs required. | -| Quality gate | ACTIVE_PR | Product workflow for fmt/clippy/tests/docs/coverage/Draft-2020-12 schema fixtures/lock/clean-tree; exact current-head hosted execution remains required. | +| Quality gate | ACTIVE_PR | Product workflow requires exact checkout, CI-contract validation, fmt, Clippy, tests, rustdoc, 100% owned line/function/region/source-branch coverage, Draft-2020-12 schema fixtures, lock freshness, and clean tree. Fresh exact-head execution is required after this documentation change. | | Standards/research | ACTIVE_PR | Stable-vs-draft standards plus paper-by-paper Generation/Client/Bridge/cross-cutting capability and evaluation traceability. | -| Security/test/operability | ACTIVE_PR | Baselines added; published semantic truth is specified as immutable with correction by superseding release; no production service claimed. | +| Security/test/operability | ACTIVE_PR | Baselines added; published semantic truth is immutable with correction by superseding release; no production service is claimed. | + +## Causal control-plane repair + +`ContextualWisdomLab/.github` PR #1618 has now merged. It repaired the organization-required Security Scan and SAST Semgrep runner selectors at the owning control plane by replacing the observed-starved floating `ubuntu-latest` selectors with explicit `ubuntu-24.04`, while preserving scanners, permissions, thresholds, action pins, exact-head validation, and fail-closed behavior. The central repair demonstrated Security Scan and SAST success on its own exact head before merge. + +The older ConceptWeave required-workflow runs on `5cd7d1de742fe34aa99900641cc8b124e7c65f9e` were created before that central merge and remain queued; their workflow snapshot cannot be treated as repaired in place. This new ConceptWeave head exists partly to cause a fresh PR synchronize event so required workflows are instantiated from the repaired central source. Do not bypass or transfer predecessor results. ## P0 product gaps after foundation @@ -26,16 +34,19 @@ Only the repository bootstrap README exists before the foundation PR. No product 5. **Governance persistence** — PostgreSQL 3NF candidates, evidence, validation receipts, review decisions, releases, transactional outbox, bitemporal history where applicable. 6. **Review workflow** — Keyverse tenant/role/purpose context, steward review, maker-checker where required, stale decision protection, immutable publication receipt. 7. **Publication adapters** — OWL/RDFS/SKOS/SHACL/JSON-LD and version-bound Apache Ossie semantic-model export. -8. **CWL integration** — `semantic-data-portal`, `LineageWeave`, `context-graph-contracts`, GRC, and EA through published contracts only. -9. **Evaluation harness** — ontology-learning/matching golden fixtures, structural/semantic metrics, human-reviewed cases, replay reproducibility, multilingual cases. -10. **Secure external research** — SearXNG discovery and safe source fetch through the correct CWL egress boundary for ontology grounding, never search snippets as truth. -11. **Observability** — shared CWL OpenTelemetry import/bootstrap contract, detailed structured logs, SIEM security-event projection where applicable. -12. **Release** — SBOM, provenance, signed artifacts, migration/backup/restore evidence, versioned changelog, protected release pipeline. +8. **Client Consumption** — stacked PR #5 / Issue #3 currently adds offline release admission and a versioned semantic-release contract; byte-level integrity verification, compatibility/deprecation, release diff/stale handling, match/resolve/explain/query-plan remain open. +9. **CWL integration** — `semantic-data-portal`, `LineageWeave`, `context-graph-contracts`, GRC, and EA through published contracts only. +10. **Evaluation harness** — ontology-learning/matching golden fixtures, structural/semantic metrics, human-reviewed cases, replay reproducibility, multilingual cases. +11. **Secure external research** — SearXNG discovery and safe source fetch through the correct CWL egress boundary for ontology grounding, never search snippets as truth. +12. **Observability** — shared CWL OpenTelemetry import/bootstrap contract, detailed structured logs, SIEM security-event projection where applicable. +13. **Release** — SBOM, provenance, signed artifacts, migration/backup/restore evidence, versioned changelog, protected release pipeline. ## DDD fitness gaps - No generic `utils/helpers/services/common` domain buckets are permitted. - Adapters must remain outside `conceptweave-domain`. +- Client Consumption may depend only on versioned public release/domain contracts, never generator-private classes or persistence. - Foreign product DTOs require Anti-Corruption Layers. - `semantic-data-portal` must not become ConceptWeave persistence, and ConceptWeave must not become an SDP clone. +- Consuming-product authorization and physical query execution stay downstream. - External forks/tools can be optional adapters but are not CWL-owned product authorities. From 4c38a90ab103a3ce8e2baab2b717f6414ce706e0 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 12:56:15 +0900 Subject: [PATCH 46/59] docs(gap): refresh live semantic engineering baseline --- docs/product-technical-gap-baseline.md | 99 ++++++++++++++------------ 1 file changed, 55 insertions(+), 44 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index cbc42906..699ce78c 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1,52 +1,63 @@ # Product / Technical Gap Baseline -**Snapshot:** 2026-09-02 +**Snapshot:** 2026-09-04 -## Shipped on protected `main` +This file records code-current product/technical gaps. Exact PR/check/run coordinates are evidence snapshots, not mutable-head dependencies. GitHub live branch/protected-branch state remains authoritative whenever it has advanced after this snapshot. -Only the repository bootstrap README exists before the foundation PR. No production capability is claimed. +## Protected truth and active stack -## Active foundation slice — PR #1 +Protected/default `main` remains `f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425`; only the bootstrap README is shipped there and no ConceptWeave release exists yet. -The exact PR head is the live GitHub branch head; check evidence is valid only for that unchanged SHA. The predecessor head `5cd7d1de742fe34aa99900641cc8b124e7c65f9e` reached terminal repository-owned Product success (exact checkout, fmt, Clippy, tests, rustdoc, exact owned coverage, JSON contract, lock freshness, clean tree). This baseline update intentionally creates a newer documentation-only head so all checks must be re-established rather than transferred. +The active dependency stack is: -| Area | Status | Evidence / action / next verification | +1. Foundation PR #1 — pre-update exact head `bba351b77bf5f1ab5cfd55979fbb2bd158f78b81`, open/non-Draft/mergeable. Repository Product `33527150325` and SAST `33527150417` succeeded on that exact head. Security `33527150445` failed closed at authoritative Dependency Review availability after real runner assignment; OSV/Scorecard/Trivy do not substitute for that gate. Required OpenCode and substantive Strix remain non-passing central lanes. This baseline repair legitimately creates a newer documentation head, so predecessor terminal evidence does not transfer. +2. Client Consumption PR #5 — `cd99eb4a42011206f8efa376106aa4b121d2010e`, Draft/open/mergeable and stacked on Foundation. Product `338229794126` is historical metadata typo and must not be used; authoritative current Product is `33822984126`, job `100869494950`, queued before runner assignment. Detached-artifact API and public-document drift have real hosted RED→repair lineage. The next intended RED is the missing language-neutral supersession/publication JSON Schema plus valid/invalid fixtures, only after this exact head reaches that boundary. +3. Source Observation PR #6 — `1fdfb3af14c126c270861eb541e9e57d47418bb8`, Draft/open/mergeable and stacked on Foundation. The prior UTC provenance RED executed and was repaired. The current registry-identity test proves that immutable snapshot/source-receipt provenance must not bypass the Source Observation port's opaque ≤128-byte lowercase multiword `snake_case` registry-key boundary. Predecessor `c9af2255...` reached a real hosted runner but failed at formatting before the semantic test. Current `1fdfb3af...` contains only the remaining rustfmt wrapping repair; Product `33834639272`, job `100904527699`, is queued before runner assignment. Production registry-key validation remains intentionally unchanged until the intended semantic RED executes. + +## Foundation capability status + +| Area | Status | Evidence / next verification | | --- | --- | --- | -| Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define Semantic Model Engineering and CWL boundaries. Revalidate on the new exact head. | -| Truth/publication lifecycle | ACTIVE_PR | Rust domain lifecycle defines Draft -> Proposed -> Validated -> Reviewed -> Published with explicit governance authorization at steward/publication boundaries; candidate JSON Schema enforces public structural shape and Published -> Authoritative consistency. The earlier missing-evidence and branch-coverage defects were repaired and proven by predecessor exact-head Product success. | -| Rust baseline | ACTIVE_PR | Rust 1.98.0 workspace, unsafe forbidden, public docs required. | -| Quality gate | ACTIVE_PR | Product workflow requires exact checkout, CI-contract validation, fmt, Clippy, tests, rustdoc, 100% owned line/function/region/source-branch coverage, Draft-2020-12 schema fixtures, lock freshness, and clean tree. Fresh exact-head execution is required after this documentation change. | -| Standards/research | ACTIVE_PR | Stable-vs-draft standards plus paper-by-paper Generation/Client/Bridge/cross-cutting capability and evaluation traceability. | -| Security/test/operability | ACTIVE_PR | Baselines added; published semantic truth is immutable with correction by superseding release; no production service is claimed. | - -## Causal control-plane repair - -`ContextualWisdomLab/.github` PR #1618 has now merged. It repaired the organization-required Security Scan and SAST Semgrep runner selectors at the owning control plane by replacing the observed-starved floating `ubuntu-latest` selectors with explicit `ubuntu-24.04`, while preserving scanners, permissions, thresholds, action pins, exact-head validation, and fail-closed behavior. The central repair demonstrated Security Scan and SAST success on its own exact head before merge. - -The older ConceptWeave required-workflow runs on `5cd7d1de742fe34aa99900641cc8b124e7c65f9e` were created before that central merge and remain queued; their workflow snapshot cannot be treated as repaired in place. This new ConceptWeave head exists partly to cause a fresh PR synchronize event so required workflows are instantiated from the repaired central source. Do not bypass or transfer predecessor results. - -## P0 product gaps after foundation - -1. **Source Observation vertical** — relational schema snapshot contract, real PostgreSQL introspection adapter, immutable digest/location receipts, hostile-input bounds. -2. **Ontology induction** — deterministic observations plus contextual-orchestrator structured candidate generation for concepts, taxonomy, and non-taxonomic relations. -3. **Semantic-layer induction** — dimensions, measures, grain, units, relationships, and physical mappings with deterministic calculation contracts. -4. **Validation engine** — RDF/OWL/SKOS/SHACL publication validation, consistency checks, duplicate/conflict detection, bounded reasoning. -5. **Governance persistence** — PostgreSQL 3NF candidates, evidence, validation receipts, review decisions, releases, transactional outbox, bitemporal history where applicable. -6. **Review workflow** — Keyverse tenant/role/purpose context, steward review, maker-checker where required, stale decision protection, immutable publication receipt. -7. **Publication adapters** — OWL/RDFS/SKOS/SHACL/JSON-LD and version-bound Apache Ossie semantic-model export. -8. **Client Consumption** — stacked PR #5 / Issue #3 currently adds offline release admission and a versioned semantic-release contract; byte-level integrity verification, compatibility/deprecation, release diff/stale handling, match/resolve/explain/query-plan remain open. -9. **CWL integration** — `semantic-data-portal`, `LineageWeave`, `context-graph-contracts`, GRC, and EA through published contracts only. -10. **Evaluation harness** — ontology-learning/matching golden fixtures, structural/semantic metrics, human-reviewed cases, replay reproducibility, multilingual cases. -11. **Secure external research** — SearXNG discovery and safe source fetch through the correct CWL egress boundary for ontology grounding, never search snippets as truth. -12. **Observability** — shared CWL OpenTelemetry import/bootstrap contract, detailed structured logs, SIEM security-event projection where applicable. -13. **Release** — SBOM, provenance, signed artifacts, migration/backup/restore evidence, versioned changelog, protected release pipeline. - -## DDD fitness gaps - -- No generic `utils/helpers/services/common` domain buckets are permitted. -- Adapters must remain outside `conceptweave-domain`. -- Client Consumption may depend only on versioned public release/domain contracts, never generator-private classes or persistence. -- Foreign product DTOs require Anti-Corruption Layers. -- `semantic-data-portal` must not become ConceptWeave persistence, and ConceptWeave must not become an SDP clone. -- Consuming-product authorization and physical query execution stay downstream. -- External forks/tools can be optional adapters but are not CWL-owned product authorities. +| Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define ConceptWeave ownership of `observe -> discover -> propose -> align -> validate -> review -> publish`, governed immutable semantic releases, and the stable Client contract. Foreign product truth remains behind released/versioned ports and ACLs. | +| Truth/publication lifecycle | ACTIVE_PR | Rust domain lifecycle enforces explicit governance authorization at steward/publication boundaries, immutable publication and supersession semantics, and evidence-bound authoritative state. All returned inline review threads are resolved. | +| Source Observation | ACTIVE_CHILD | Immutable PostgreSQL table/column/PK/unique/FK/CHECK evidence, exact identifiers, canonical snapshot digest, UTC provenance, exact receipts, bounded request budgets/cancellation and opaque source registry keys exist. Registry-key consistency at the immutable snapshot boundary is the current TDD lane. No live PostgreSQL adapter is claimed; ADR 0004 remains Proposed. | +| Client Consumption | ACTIVE_CHILD | Offline Published+Authoritative admission, explicit compatibility, exact resolution/diff, canonical digest verification, detached artifact verification and supersession validation exist. Language-neutral public supersession/publication schema/fixtures remain intentionally absent pending current-head RED. | +| Quality gate | ACTIVE_PR | Rust 1.98.0, unsafe forbidden, public docs required, exact checkout, fmt, Clippy, tests, rustdoc, owned 100% coverage, Draft-2020-12 schema fixtures, lock freshness and clean-tree checks. Every head movement requires fresh exact-head evidence. | +| Security / dependency review | BLOCKED_OWNER | Foundation Security fails closed because GitHub Dependency Review availability is not satisfied. `.github#810` owns the authoritative central repair; scanner substitution and 403-as-success are forbidden. | +| Review / runner admission | BLOCKED_OWNER | Selective/intermittent hosted-runner admission remains observable. Foundation siblings have acquired runners while OpenCode/Strix substantive lanes remain queued; current #5/#6 Product lanes are also pre-runner queued. `.github#712/#1531/#1796` own the central queue/review amplification paths. | +| Standards / research | ACTIVE_PR | Stable recommendations remain distinct from drafts. As of 2026-09-04 authoritative W3C history still lists RDF 1.2 Concepts as Candidate Recommendation Snapshot (2026-04-07) and SHACL 1.2 Core as Working Draft (2026-08-03); SHACL 1.2 Rules has a newer 2026-08-19 Working Draft. Apache Ossie remains incubating and its current v0.1 specification is pre-first-Apache-release, so it is tracked as emerging interop rather than a final standard. Detailed APA/capability mapping stays in `docs/doctoring/REFERENCES.md` and `RESEARCH_CAPABILITY_TRACEABILITY.md`. | +| Release | NOT_STARTED | No immutable ConceptWeave release exists. Version/CHANGELOG/tag/package/semantic release/SBOM/provenance/reproducibility/rollback are required on the exact protected release head. | + +## Central control-plane evidence + +Protected central source is `.github/main@07d9ec23fb265c76539d23249e1dfa124ea7b23b` at this snapshot. This is evidence only, not a ConceptWeave dependency. + +- `.github#1796` now has test-first Draft PR #1821 `test/1796-org-sweep-owner@9c79cf775ad6a125a94dedcae9683c20a65a0339` (actual branch: `test/1796-org-sweep-queue-owner`). The test contract removes repository-wide queued/in-progress Actions inventory and `ORG()`-style stale-queue ownership from the organization sweep while retaining exact live-head coalescing in the target repository. Production central source is unchanged until the RED executes. +- `.github#1822@7a5cc1b1c43946d210405cd051ae629ff2c44966` is a separate Draft lane that makes top-level `CoalescingRefused` honor the coalescer's documented safe-no-op behavior while preserving fail-closed handling for other failures. Do not conflate it with #1821's duplicate-owner removal. +- Fresh `.github` queued inventory observed `1,906` runs on 2026-09-04, above the earlier ~1,604 trough. Aggregate queue movement is diagnostic only; consumer GREEN requires actual runner assignment, exact checkout and terminal evidence on unchanged current heads. + +## P0 product gaps after the current TDD lanes + +1. **Concrete Source Observation adapter** — maintained Rust PostgreSQL driver behind `conceptweave-source-port`; adapter-local credential resolution; explicit read-only mode; statement timeout, cancellation, row/byte/concurrency budgets; complete immutable snapshot or fail closed; deterministic replay against a frozen anonymized GRC-shaped fixture. +2. **Ontology discovery** — deterministic term/concept/taxonomy/non-taxonomic-relation candidate generation with exact source receipts and abstention for unsupported semantics. +3. **Semantic-layer discovery** — dimensions, measures, grain, units, relationships and physical mappings with deterministic calculation contracts; do not infer business authority from relational structure alone. +4. **LLM Proposal** — every production model call through released `contextual-orchestrator`; outputs remain proposed/inferred and preserve source/model/prompt/provenance evidence. +5. **Alignment / matching** — retrieval/pruning/structural evidence first, bounded optional LLM assistance, OAEI-style evaluation, deterministic reproducibility and steward-visible decisions. +6. **Validation engine** — RDF/OWL/SKOS/SHACL and semantic-layer validation, consistency/conflict/duplicate detection, bounded reasoning, explicit unsupported-feature failure. +7. **Governance persistence** — PostgreSQL 3NF candidates/evidence/validation/review/release/supersession receipts, transactional outbox and temporal history only where domain semantics require it. +8. **Review workflow** — Keyverse identity context, tenant/role/purpose authorization, steward decisions, maker-checker where required, stale-decision protection and immutable publication receipt. +9. **Publication adapters** — versioned OWL/RDFS/SKOS/SHACL/JSON-LD plus explicitly version-bound Apache Ossie export; draft/incubating formats cannot be presented as final standards. +10. **Client completion** — language-neutral release/supersession contract, provenance/signature verification, relation/mapping/dimension/measure resolution, compatibility/deprecation, match/explain/query-plan contracts while downstream products retain physical authorization/execution. +11. **CWL integration** — only released/versioned semantic release/contract/ACL seams to `semantic-data-portal`, `context-graph-contracts`, GRC, EA and other consumers; no source copying, cross-service SQL or mutable supplier heads. +12. **Evaluation / multilingual** — reviewed golden fixtures, ontology-learning/matching metrics, source-evidence binding, abstention, reproducibility, KO/EN/JA/ZH/VI/ES/DE/FR labels, CJK/font/text-expansion checks where UI or published labels are material. +13. **Observability / recovery / release** — structured telemetry, security evidence, backup/restore, package/SBOM/provenance/signing, reproducible build and rollback proof before immutable release. + +## DDD fitness constraints + +- No generic `utils/helpers/services/common` domain buckets. +- Adapters remain outside the core domain model; external DTOs cross Anti-Corruption Layers. +- Source Observation facts are not source-system business truth, and relational constraints are not semantic authority by themselves. +- Client Consumption depends only on governed release contracts, never generator-private classes, prompts, persistence tables or orchestration state. +- `semantic-data-portal` remains catalog/governance/consumption rather than ConceptWeave persistence; `context-graph-contracts` owns interop contracts; `enterprise-architecture-core` owns EA; `contextual-orchestrator` owns provider routing. +- Consuming products retain tenant/purpose authorization and physical query execution. +- Published semantic truth is immutable; corrections create a new release plus supersession evidence rather than in-place overwrite. From 447aa0723abd7b582b9acc478ed90238d0d59214 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 12:56:55 +0900 Subject: [PATCH 47/59] docs(gap): correct live evidence coordinates --- docs/product-technical-gap-baseline.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 699ce78c..b502fc76 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -11,7 +11,7 @@ Protected/default `main` remains `f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425`; onl The active dependency stack is: 1. Foundation PR #1 — pre-update exact head `bba351b77bf5f1ab5cfd55979fbb2bd158f78b81`, open/non-Draft/mergeable. Repository Product `33527150325` and SAST `33527150417` succeeded on that exact head. Security `33527150445` failed closed at authoritative Dependency Review availability after real runner assignment; OSV/Scorecard/Trivy do not substitute for that gate. Required OpenCode and substantive Strix remain non-passing central lanes. This baseline repair legitimately creates a newer documentation head, so predecessor terminal evidence does not transfer. -2. Client Consumption PR #5 — `cd99eb4a42011206f8efa376106aa4b121d2010e`, Draft/open/mergeable and stacked on Foundation. Product `338229794126` is historical metadata typo and must not be used; authoritative current Product is `33822984126`, job `100869494950`, queued before runner assignment. Detached-artifact API and public-document drift have real hosted RED→repair lineage. The next intended RED is the missing language-neutral supersession/publication JSON Schema plus valid/invalid fixtures, only after this exact head reaches that boundary. +2. Client Consumption PR #5 — `cd99eb4a42011206f8efa376106aa4b121d2010e`, Draft/open/mergeable and stacked on Foundation. Product `33822984126`, job `100869494950`, is queued before runner assignment. Detached-artifact API and public-document drift have real hosted RED→repair lineage. The next intended RED is the missing language-neutral supersession/publication JSON Schema plus valid/invalid fixtures, only after this exact head reaches that boundary. 3. Source Observation PR #6 — `1fdfb3af14c126c270861eb541e9e57d47418bb8`, Draft/open/mergeable and stacked on Foundation. The prior UTC provenance RED executed and was repaired. The current registry-identity test proves that immutable snapshot/source-receipt provenance must not bypass the Source Observation port's opaque ≤128-byte lowercase multiword `snake_case` registry-key boundary. Predecessor `c9af2255...` reached a real hosted runner but failed at formatting before the semantic test. Current `1fdfb3af...` contains only the remaining rustfmt wrapping repair; Product `33834639272`, job `100904527699`, is queued before runner assignment. Production registry-key validation remains intentionally unchanged until the intended semantic RED executes. ## Foundation capability status @@ -26,13 +26,13 @@ The active dependency stack is: | Security / dependency review | BLOCKED_OWNER | Foundation Security fails closed because GitHub Dependency Review availability is not satisfied. `.github#810` owns the authoritative central repair; scanner substitution and 403-as-success are forbidden. | | Review / runner admission | BLOCKED_OWNER | Selective/intermittent hosted-runner admission remains observable. Foundation siblings have acquired runners while OpenCode/Strix substantive lanes remain queued; current #5/#6 Product lanes are also pre-runner queued. `.github#712/#1531/#1796` own the central queue/review amplification paths. | | Standards / research | ACTIVE_PR | Stable recommendations remain distinct from drafts. As of 2026-09-04 authoritative W3C history still lists RDF 1.2 Concepts as Candidate Recommendation Snapshot (2026-04-07) and SHACL 1.2 Core as Working Draft (2026-08-03); SHACL 1.2 Rules has a newer 2026-08-19 Working Draft. Apache Ossie remains incubating and its current v0.1 specification is pre-first-Apache-release, so it is tracked as emerging interop rather than a final standard. Detailed APA/capability mapping stays in `docs/doctoring/REFERENCES.md` and `RESEARCH_CAPABILITY_TRACEABILITY.md`. | -| Release | NOT_STARTED | No immutable ConceptWeave release exists. Version/CHANGELOG/tag/package/semantic release/SBOM/provenance/reproducibility/rollback are required on the exact protected release head. | +| Release | NOT_STARTED | No immutable ConceptWeave release exists. Version/CHANGELOG/tag/package/semantic_release/SBOM/provenance/reproducibility/rollback are required on the exact protected release head. | ## Central control-plane evidence Protected central source is `.github/main@07d9ec23fb265c76539d23249e1dfa124ea7b23b` at this snapshot. This is evidence only, not a ConceptWeave dependency. -- `.github#1796` now has test-first Draft PR #1821 `test/1796-org-sweep-owner@9c79cf775ad6a125a94dedcae9683c20a65a0339` (actual branch: `test/1796-org-sweep-queue-owner`). The test contract removes repository-wide queued/in-progress Actions inventory and `ORG()`-style stale-queue ownership from the organization sweep while retaining exact live-head coalescing in the target repository. Production central source is unchanged until the RED executes. +- `.github#1796` now has test-first Draft PR #1821 `test/1796-org-sweep-queue-owner@9c79cf775ad6a125a94dedcae9683c20a65a0339`. The test contract removes repository-wide queued/in-progress Actions inventory and `ORG_SWEEP_STALE_QUEUE_HOURS` ownership from the organization sweep while retaining exact live-head coalescing in the target repository. Production central source is unchanged until the RED executes. - `.github#1822@7a5cc1b1c43946d210405cd051ae629ff2c44966` is a separate Draft lane that makes top-level `CoalescingRefused` honor the coalescer's documented safe-no-op behavior while preserving fail-closed handling for other failures. Do not conflate it with #1821's duplicate-owner removal. - Fresh `.github` queued inventory observed `1,906` runs on 2026-09-04, above the earlier ~1,604 trough. Aggregate queue movement is diagnostic only; consumer GREEN requires actual runner assignment, exact checkout and terminal evidence on unchanged current heads. @@ -48,7 +48,7 @@ Protected central source is `.github/main@07d9ec23fb265c76539d23249e1dfa124ea7b2 8. **Review workflow** — Keyverse identity context, tenant/role/purpose authorization, steward decisions, maker-checker where required, stale-decision protection and immutable publication receipt. 9. **Publication adapters** — versioned OWL/RDFS/SKOS/SHACL/JSON-LD plus explicitly version-bound Apache Ossie export; draft/incubating formats cannot be presented as final standards. 10. **Client completion** — language-neutral release/supersession contract, provenance/signature verification, relation/mapping/dimension/measure resolution, compatibility/deprecation, match/explain/query-plan contracts while downstream products retain physical authorization/execution. -11. **CWL integration** — only released/versioned semantic release/contract/ACL seams to `semantic-data-portal`, `context-graph-contracts`, GRC, EA and other consumers; no source copying, cross-service SQL or mutable supplier heads. +11. **CWL integration** — only released/versioned semantic_release/contract/ACL seams to `semantic-data-portal`, `context-graph-contracts`, GRC, EA and other consumers; no source copying, cross-service SQL or mutable supplier heads. 12. **Evaluation / multilingual** — reviewed golden fixtures, ontology-learning/matching metrics, source-evidence binding, abstention, reproducibility, KO/EN/JA/ZH/VI/ES/DE/FR labels, CJK/font/text-expansion checks where UI or published labels are material. 13. **Observability / recovery / release** — structured telemetry, security evidence, backup/restore, package/SBOM/provenance/signing, reproducible build and rollback proof before immutable release. From 1264b513a01db61998053cd15227c0f29b923871 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 13:04:29 +0900 Subject: [PATCH 48/59] docs(research): bind structural-context alignment evidence --- .../RESEARCH_CAPABILITY_TRACEABILITY.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md b/docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md index ea21e56c..bde6ff96 100644 --- a/docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md +++ b/docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md @@ -1,8 +1,8 @@ # Research-to-capability traceability -Snapshot: 2026-09-01 +Snapshot: 2026-09-04 -This register turns the current Consensus search set into product decisions. A paper is not considered "used" merely because it appears in the bibliography: it must be tied to a bounded context, an accepted or rejected design implication, and an executable evaluation family. Publication metadata below follows the current Consensus record; preprints are not silently promoted to peer-reviewed evidence. +This register turns the accepted research set into product decisions. A paper is not considered "used" merely because it appears in the bibliography: it must be tied to a bounded context, an accepted or rejected design implication, and an executable evaluation family. Publication metadata follows the previously recorded Consensus set plus authoritative publication records re-verified on 2026-09-04; preprints are not silently promoted to peer-reviewed evidence. ## Classification rules @@ -30,6 +30,8 @@ This register turns the current Consensus search set into product decisions. A p | Giglou et al. (2026), *OntoLearner* | generation / cross_cutting | Add cross-domain standardized benchmarking and measure failure against ontology complexity, not just model size. | adopt | Evaluation | multi-domain term/taxonomy/relation benchmark; complexity-stratified error analysis | Tool/library is research infrastructure, not a required runtime dependency. | | Hertling & Paulheim (2023), *OLaLa* | client | Ontology matching needs explicit prompt representation, examples, existing correspondences, and candidate-generation choices. | adopt | Model Alignment | OAEI-style matching P/R/F1; zero/few-shot comparison | LLM result is a correspondence candidate, never automatic authoritative alignment. | | Giglou, D'Souza, Engel, & Auer (2024), *LLMs4OM* | client | Retrieve first, then match; compare concept-only, parent-context, and child-context representations. | adopt | Model Alignment, Client Consumption | retrieval recall; matching P/R/F1 across representation variants | Consensus currently records arXiv; client must remain functional without LLM matching. | +| He, Chen, Dong, & Horrocks (2023), *Exploring Large Language Models for Ontology Alignment* | client | Compare concept-label-only matching with bounded structural-context matching instead of treating a richer prompt as automatically better. LLM output remains a correspondence candidate. | adapt | Model Alignment | OAEI Bio-ML hard subsets; concept-only vs structural-context ablation; zero-shot baseline | Peer-reviewed ISWC 2023 Posters/Demos evidence is explicitly preliminary; reported model gains do not establish enterprise-domain generality or publication authority. | +| Amini, Saki Norouzi, Hitzler, & Amini (2024), *Towards Complex Ontology Alignment Using Large Language Models* | client | Complex correspondence proposals may use bounded ontology modules/rich local context, with module/context size recorded as evidence rather than unconstrained prompt expansion. | adapt | Model Alignment | complex-alignment fixtures; module-size/context ablation; correspondence P/R/F1 | Refereed KGSWC 2024 evidence supports module-aware prompting, but rich context cannot bypass retrieval bounds, source provenance, abstention, or steward governance. | | Sousa, Lima, & Trojahn (2025), *Complex Ontology Matching with Large Language Model Embeddings* | client | Support expressive correspondence proposals using local subgraph/neighborhood evidence, not label similarity alone. | adapt | Model Alignment | complex-correspondence F1; subgraph ablations | Embedding-space/model compatibility must be explicit; reported gains do not authorize cross-model vector comparison. | | Taboada et al. (2025), MILA | client | Use programmed retrieval/search to prune candidates and reserve LLM calls for uncertain cases. | adopt | Model Alignment, Client Consumption | candidate recall; final P/R/F1; LLM-call reduction vs naive prompting | Consensus currently records arXiv; algorithmic search cannot bypass evidence/truth-state rules. | | Barcelos, French, & Wu (2025), *KROMA* | client | Targeted knowledge retrieval, structural context, and refinement should precede context-augmented LLM matching. | adopt | Model Alignment | candidate pruning recall; prompt-enrichment ablation; communication cost | Consensus currently records arXiv; RAG context is not source authority. | @@ -53,9 +55,9 @@ This register turns the current Consensus search set into product decisions. A p Acceptance must prove that ConceptWeave never becomes the GRC system of record, that proposed/inferred relations do not mutate authoritative GRC records, that release validation works offline, and that release upgrades identify affected GRC queries explicitly. Public OAEI/RODI/LLMs4OL-style benchmarks remain necessary because one enterprise fixture cannot establish general matching or learning performance. -## Consensus records used in this snapshot +## Consensus records used in the prior accepted set -The following canonical Consensus records were fetched before recording product implications: +The following canonical Consensus records were fetched before recording the corresponding product implications: - https://consensus.app/papers/accelerating-knowledge-graph-and-ontology-engineering-shimizu-hitzler/82d868ee8f7953108246241e28d5e339/?utm_source=chatgpt - https://consensus.app/papers/enhancing-knowledge-graph-construction-using-large-trajanoska-stojanov/80ffe83041735fdf94bf4b60dd32ba1a/?utm_source=chatgpt @@ -84,10 +86,15 @@ The following canonical Consensus records were fetched before recording product - https://consensus.app/papers/large-language-models-for-ontology-engineering-a-li-garijo/3087bb8f7cd0500d917f89d8a92559e5/?utm_source=chatgpt - https://consensus.app/papers/a-short-review-for-ontology-learning-stride-to-large-du-an/ad3e2c6bf660569ca1effb7b6d31a6f7/?utm_source=chatgpt +## Additional authoritative publication records verified on 2026-09-04 + +- He, Y., Chen, J., Dong, H., & Horrocks, I. (2023). *Exploring large language models for ontology alignment*. 22nd International Semantic Web Conference (ISWC 2023), Posters, Demos and Industry Tracks. Oxford University Research Archive peer-reviewed record: https://ora.ox.ac.uk/objects/uuid%3Ab0ecf14b-e9b9-4767-9fae-8a7adddd6fb6 +- Amini, R., Saki Norouzi, S., Hitzler, P., & Amini, R. (2024). *Towards complex ontology alignment using large language models*. In *Knowledge Graphs and Semantic Web: 6th International Conference, KGSWC 2024* (pp. 17–31). Springer. https://doi.org/10.1007/978-3-031-81221-7_2 + ## Next executable consequences 1. Issue #2 Generation evaluation must expose task-level metrics rather than one aggregate "ontology quality" score. -2. Issue #3 Client work must implement retrieval-before-prompt, structural/neighborhood evidence, deterministic lexical evidence, explicit abstention, release diff/version compatibility, and OAEI-LLM hallucination fixtures. +2. Issue #3 Client work must implement retrieval-before-prompt, structural/neighborhood evidence, deterministic lexical evidence, explicit abstention, release diff/version compatibility, OAEI-LLM hallucination fixtures, concept-only vs structural-context ablations, and bounded-module complex-alignment fixtures. 3. GRC must remain the first enterprise round-trip fixture, while OAEI/RODI/LLMs4OL-style data guards against overfitting the general contract to GRC. 4. LLM calls remain behind `contextual-orchestrator`; model/provider/prompt changes require receipts and sensitivity evidence. 5. Human review remains mandatory before authority promotion; Crowd-OM is evidence for scalable validation mechanics, not permission to replace GRC/domain steward authority. From e1b0eaa544c92f3526d91c33d50e11d17115a5cf Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 13:15:22 +0900 Subject: [PATCH 49/59] test(doctoring): require authoritative records for adopted alignment studies --- .../tests/research_reference_traceability.rs | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 crates/conceptweave-domain/tests/research_reference_traceability.rs diff --git a/crates/conceptweave-domain/tests/research_reference_traceability.rs b/crates/conceptweave-domain/tests/research_reference_traceability.rs new file mode 100644 index 00000000..41d9b3b6 --- /dev/null +++ b/crates/conceptweave-domain/tests/research_reference_traceability.rs @@ -0,0 +1,26 @@ +const REFERENCES: &str = include_str!("../../../docs/doctoring/REFERENCES.md"); +const TRACEABILITY: &str = + include_str!("../../../docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md"); + +#[test] +fn adopted_alignment_studies_have_authoritative_bibliography_records() { + for (traceability_marker, authoritative_record) in [ + ( + "He, Chen, Dong, & Horrocks (2023)", + "https://ceur-ws.org/Vol-3632/ISWC2023_paper_427.pdf", + ), + ( + "Amini, Saki Norouzi, Hitzler, & Amini (2024)", + "https://doi.org/10.1007/978-3-031-81221-7_2", + ), + ] { + assert!( + TRACEABILITY.contains(traceability_marker), + "an adopted study must remain explicit in research-to-capability traceability: {traceability_marker}" + ); + assert!( + REFERENCES.contains(authoritative_record), + "an adopted study must have an authoritative publication record in the APA bibliography: {authoritative_record}" + ); + } +} From 707e687bb510d3f1fd5d1bdc94e590b30ffbb641 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 16:47:51 +0900 Subject: [PATCH 50/59] test(doctoring): bind authoritative publication years --- .../tests/research_reference_traceability.rs | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/crates/conceptweave-domain/tests/research_reference_traceability.rs b/crates/conceptweave-domain/tests/research_reference_traceability.rs index 41d9b3b6..c64825e2 100644 --- a/crates/conceptweave-domain/tests/research_reference_traceability.rs +++ b/crates/conceptweave-domain/tests/research_reference_traceability.rs @@ -3,15 +3,17 @@ const TRACEABILITY: &str = include_str!("../../../docs/doctoring/RESEARCH_CAPABILITY_TRACEABILITY.md"); #[test] -fn adopted_alignment_studies_have_authoritative_bibliography_records() { - for (traceability_marker, authoritative_record) in [ +fn adopted_alignment_studies_have_authoritative_apa_bibliography_records() { + for (traceability_marker, authoritative_record, publication_year_marker) in [ ( "He, Chen, Dong, & Horrocks (2023)", "https://ceur-ws.org/Vol-3632/ISWC2023_paper_427.pdf", + "(2023).", ), ( "Amini, Saki Norouzi, Hitzler, & Amini (2024)", "https://doi.org/10.1007/978-3-031-81221-7_2", + "(2025).", ), ] { assert!( @@ -19,8 +21,10 @@ fn adopted_alignment_studies_have_authoritative_bibliography_records() { "an adopted study must remain explicit in research-to-capability traceability: {traceability_marker}" ); assert!( - REFERENCES.contains(authoritative_record), - "an adopted study must have an authoritative publication record in the APA bibliography: {authoritative_record}" + REFERENCES.split("\n\n").any(|entry| { + entry.contains(authoritative_record) && entry.contains(publication_year_marker) + }), + "an adopted study must have an authoritative APA publication record with the publication year bound to that record: {authoritative_record} {publication_year_marker}" ); } } From b69f6e797834cb74cb70d093c71efa1d5a86bbb5 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 18:48:59 +0900 Subject: [PATCH 51/59] docs: refresh live semantic engineering baseline --- docs/product-technical-gap-baseline.md | 34 ++++++++++++++------------ 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index b502fc76..bf02efff 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -2,39 +2,43 @@ **Snapshot:** 2026-09-04 -This file records code-current product/technical gaps. Exact PR/check/run coordinates are evidence snapshots, not mutable-head dependencies. GitHub live branch/protected-branch state remains authoritative whenever it has advanced after this snapshot. +This file records code-current product and technical gaps. Exact PR/check/run coordinates below are evidence snapshots, not mutable-head dependencies. GitHub protected-branch, PR, issue, and workflow state remains authoritative whenever it advances after this snapshot. A documentation commit necessarily creates a successor Foundation head, so the Foundation SHA below is explicitly the pre-repair head observed immediately before this file was refreshed. ## Protected truth and active stack Protected/default `main` remains `f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425`; only the bootstrap README is shipped there and no ConceptWeave release exists yet. -The active dependency stack is: +The active dependency stack observed immediately before this baseline repair is: -1. Foundation PR #1 — pre-update exact head `bba351b77bf5f1ab5cfd55979fbb2bd158f78b81`, open/non-Draft/mergeable. Repository Product `33527150325` and SAST `33527150417` succeeded on that exact head. Security `33527150445` failed closed at authoritative Dependency Review availability after real runner assignment; OSV/Scorecard/Trivy do not substitute for that gate. Required OpenCode and substantive Strix remain non-passing central lanes. This baseline repair legitimately creates a newer documentation head, so predecessor terminal evidence does not transfer. -2. Client Consumption PR #5 — `cd99eb4a42011206f8efa376106aa4b121d2010e`, Draft/open/mergeable and stacked on Foundation. Product `33822984126`, job `100869494950`, is queued before runner assignment. Detached-artifact API and public-document drift have real hosted RED→repair lineage. The next intended RED is the missing language-neutral supersession/publication JSON Schema plus valid/invalid fixtures, only after this exact head reaches that boundary. -3. Source Observation PR #6 — `1fdfb3af14c126c270861eb541e9e57d47418bb8`, Draft/open/mergeable and stacked on Foundation. The prior UTC provenance RED executed and was repaired. The current registry-identity test proves that immutable snapshot/source-receipt provenance must not bypass the Source Observation port's opaque ≤128-byte lowercase multiword `snake_case` registry-key boundary. Predecessor `c9af2255...` reached a real hosted runner but failed at formatting before the semantic test. Current `1fdfb3af...` contains only the remaining rustfmt wrapping repair; Product `33834639272`, job `100904527699`, is queued before runner assignment. Production registry-key validation remains intentionally unchanged until the intended semantic RED executes. +1. Foundation PR #1 — pre-repair exact head `707e687bb510d3f1fd5d1bdc94e590b30ffbb641`, open/non-Draft/mergeable. Product `33850379502` / job `100951576248`, SAST `33850379638`, and Security `33850379525` are queued. Product is on explicit `ubuntu-24.04` with `runner_id=0` and no steps, so the strengthened research-reference test has not yet recorded a real RED. The baseline itself was stale and is the causal reason for this documentation-only successor head; predecessor terminal evidence does not transfer. +2. Client Consumption PR #5 — pre-restack exact head `0f878fc796feb93406dedbbad85351282d3ed7bc`, Draft/open/mergeable and stacked on Foundation. Product `33850422999` / job `100951717500` is pre-runner queued. Detached-artifact API and public-document drift have real hosted RED→repair lineage. The next intended Client RED remains the missing language-neutral supersession/publication JSON Schema plus valid/invalid fixtures, only after the inherited Foundation doctoring lane passes and that Client boundary actually executes. +3. Source Observation PR #6 — pre-restack exact head `7edb2e03c6adeabdc7ffc5c5ecdbc8f4362d3ab3`, Draft/open/mergeable and stacked on Foundation. Product `33850442339` / job `100951774004` is pre-runner queued. The prior UTC provenance RED executed and was repaired. The current registry-identity test requires immutable snapshot/source-receipt provenance to respect the Source Observation port's opaque ≤128-byte lowercase multiword `snake_case` registry-key boundary. Predecessor `c9af2255...` reached a real hosted runner but failed at formatting before the semantic test. Production registry-key validation remains intentionally unchanged until the intended semantic RED executes. + +Both child PRs have no submitted reviews or inline review threads in the current fresh read. Foundation's returned inline threads are resolved; its Noema APPROVE belongs to predecessor `bba351b...` and is not current-head independent approval. ## Foundation capability status | Area | Status | Evidence / next verification | | --- | --- | --- | | Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define ConceptWeave ownership of `observe -> discover -> propose -> align -> validate -> review -> publish`, governed immutable semantic releases, and the stable Client contract. Foreign product truth remains behind released/versioned ports and ACLs. | -| Truth/publication lifecycle | ACTIVE_PR | Rust domain lifecycle enforces explicit governance authorization at steward/publication boundaries, immutable publication and supersession semantics, and evidence-bound authoritative state. All returned inline review threads are resolved. | -| Source Observation | ACTIVE_CHILD | Immutable PostgreSQL table/column/PK/unique/FK/CHECK evidence, exact identifiers, canonical snapshot digest, UTC provenance, exact receipts, bounded request budgets/cancellation and opaque source registry keys exist. Registry-key consistency at the immutable snapshot boundary is the current TDD lane. No live PostgreSQL adapter is claimed; ADR 0004 remains Proposed. | -| Client Consumption | ACTIVE_CHILD | Offline Published+Authoritative admission, explicit compatibility, exact resolution/diff, canonical digest verification, detached artifact verification and supersession validation exist. Language-neutral public supersession/publication schema/fixtures remain intentionally absent pending current-head RED. | +| Truth/publication lifecycle | ACTIVE_PR | Rust domain lifecycle enforces explicit governance authorization at steward/publication boundaries, immutable publication and supersession semantics, and evidence-bound authoritative state. Current returned inline review threads are resolved. | +| Source Observation | ACTIVE_CHILD | Immutable PostgreSQL table/column/PK/unique/FK/CHECK evidence, exact identifiers, canonical snapshot digest syntax, UTC provenance, exact receipts, bounded request budgets/cancellation and opaque source registry keys exist. Registry-key consistency at the immutable snapshot boundary is the current TDD lane. No live PostgreSQL adapter is claimed; ADR 0004 remains Proposed. | +| Client Consumption | ACTIVE_CHILD | Offline Published+Authoritative admission, explicit compatibility, exact resolution/diff, canonical digest verification, detached artifact verification and explicit supersession validation exist. Language-neutral public supersession/publication schema/fixtures remain intentionally absent pending the real current-head RED. | | Quality gate | ACTIVE_PR | Rust 1.98.0, unsafe forbidden, public docs required, exact checkout, fmt, Clippy, tests, rustdoc, owned 100% coverage, Draft-2020-12 schema fixtures, lock freshness and clean-tree checks. Every head movement requires fresh exact-head evidence. | -| Security / dependency review | BLOCKED_OWNER | Foundation Security fails closed because GitHub Dependency Review availability is not satisfied. `.github#810` owns the authoritative central repair; scanner substitution and 403-as-success are forbidden. | -| Review / runner admission | BLOCKED_OWNER | Selective/intermittent hosted-runner admission remains observable. Foundation siblings have acquired runners while OpenCode/Strix substantive lanes remain queued; current #5/#6 Product lanes are also pre-runner queued. `.github#712/#1531/#1796` own the central queue/review amplification paths. | -| Standards / research | ACTIVE_PR | Stable recommendations remain distinct from drafts. As of 2026-09-04 authoritative W3C history still lists RDF 1.2 Concepts as Candidate Recommendation Snapshot (2026-04-07) and SHACL 1.2 Core as Working Draft (2026-08-03); SHACL 1.2 Rules has a newer 2026-08-19 Working Draft. Apache Ossie remains incubating and its current v0.1 specification is pre-first-Apache-release, so it is tracked as emerging interop rather than a final standard. Detailed APA/capability mapping stays in `docs/doctoring/REFERENCES.md` and `RESEARCH_CAPABILITY_TRACEABILITY.md`. | +| Security / dependency review | BLOCKED_OWNER | Foundation predecessor Security reached a real runner and failed closed because authoritative GitHub Dependency Review availability was not satisfied. `.github#810` owns the central repair; scanner substitution and 403-as-success are forbidden. | +| Review / runner admission | BLOCKED_OWNER | Central queue pressure has fallen materially, but current ConceptWeave Product canaries remain unassigned. `.github#712/#1531/#1796` own the central admission/review-amplification paths. Queue depth alone is not consumer GREEN. | +| Standards / research | ACTIVE_PR | Stable recommendations remain distinct from drafts. The current doctoring RED contract binds adopted Model Alignment evidence to authoritative bibliography records: He et al. to CEUR/ISWC 2023 and Amini et al. to the Springer LNCS version of record published in 2025 while retaining the KGSWC 2024 study/conference lineage in traceability. `REFERENCES.md` remains intentionally unchanged until the test executes and records that RED. | | Release | NOT_STARTED | No immutable ConceptWeave release exists. Version/CHANGELOG/tag/package/semantic_release/SBOM/provenance/reproducibility/rollback are required on the exact protected release head. | ## Central control-plane evidence -Protected central source is `.github/main@07d9ec23fb265c76539d23249e1dfa124ea7b23b` at this snapshot. This is evidence only, not a ConceptWeave dependency. +Protected central source is `.github/main@07db37e5e42c63ba40ac66f22ef74e4f8836ce9a` at this snapshot. This is evidence only, not a ConceptWeave dependency. -- `.github#1796` now has test-first Draft PR #1821 `test/1796-org-sweep-queue-owner@9c79cf775ad6a125a94dedcae9683c20a65a0339`. The test contract removes repository-wide queued/in-progress Actions inventory and `ORG_SWEEP_STALE_QUEUE_HOURS` ownership from the organization sweep while retaining exact live-head coalescing in the target repository. Production central source is unchanged until the RED executes. -- `.github#1822@7a5cc1b1c43946d210405cd051ae629ff2c44966` is a separate Draft lane that makes top-level `CoalescingRefused` honor the coalescer's documented safe-no-op behavior while preserving fail-closed handling for other failures. Do not conflate it with #1821's duplicate-owner removal. -- Fresh `.github` queued inventory observed `1,906` runs on 2026-09-04, above the earlier ~1,604 trough. Aggregate queue movement is diagnostic only; consumer GREEN requires actual runner assignment, exact checkout and terminal evidence on unchanged current heads. +- `.github#1796` queue-ownership repair #1821 is integrated. Organization sweep no longer owns repository-wide queued/in-progress Actions inventory or broad cancellation; native per-PR concurrency and repository-local exact-head coalescing own supersession. +- `.github#1840` removed required-check-completion `workflow_run` fanout from the merge scheduler while retaining native PR/review/protected-branch-push/explicit-dispatch events and bounded recovery. GitHub auto-merge owns required-check completion rather than another scheduler run. +- `.github#1841` made uncovered CodeQL repositories self-remediating through idempotent OpenCode-owned setup PRs created from trusted central main. +- `.github#1842` keeps those generated repository-local CodeQL workflows off pull-request heads, leaving central `codeql-pr.yml` as the single PR-head scanner and isolating audit concurrency by trigger. +- Fresh `.github` queued inventory is `271`, materially below earlier `991`, `914`, and ~`1,900` observations. This is diagnostic progress, not acceptance: current ConceptWeave Product jobs still lack runner identity, exact checkout, steps, and terminal evidence. ## P0 product gaps after the current TDD lanes From 17d604150a7ab334412b41136d8313c1d898e9d5 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 19:46:48 +0900 Subject: [PATCH 52/59] docs(doctoring): add authoritative alignment bibliography records --- docs/doctoring/REFERENCES.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/doctoring/REFERENCES.md b/docs/doctoring/REFERENCES.md index e161cc04..2cbca9b0 100644 --- a/docs/doctoring/REFERENCES.md +++ b/docs/doctoring/REFERENCES.md @@ -49,6 +49,10 @@ Giglou, H. B., D'Souza, J., Aioanei, A. C., Mihindukulasooriya, N., & Auer, S. ( ## Client, matching, and versioning research +He, Y., Chen, J., Dong, H., & Horrocks, I. (2023). Exploring large language models for ontology alignment. In I. Fundulaki, K. Kozaki, D. Garijo, & J. M. Gomez-Perez (Eds.), *Proceedings of the ISWC 2023 Posters, Demos and Industry Tracks: From Novel Ideas to Industrial Practice* (CEUR Workshop Proceedings, Vol. 3632). CEUR-WS.org. https://ceur-ws.org/Vol-3632/ISWC2023_paper_427.pdf + +Amini, R., Saki Norouzi, S., Hitzler, P., & Amini, R. (2025). Towards complex ontology alignment using large language models. In S. Tiwari, B. Villazón-Terrazas, F. Ortiz-Rodríguez, & S. Sahri (Eds.), *Knowledge graphs and semantic web: 6th International Conference, KGSWC 2024, Paris, France, December 11–13, 2024, proceedings* (Lecture Notes in Computer Science, Vol. 15459, pp. 17–31). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-81221-7_2 + Hertling, S., & Paulheim, H. (2023). OLaLa: Ontology matching with large language models. In *Proceedings of the 12th Knowledge Capture Conference 2023* (pp. 131–139). Association for Computing Machinery. https://doi.org/10.1145/3587259.3627571 Qiang, Z., Wang, W., & Taylor, K. L. (2023). Agent-OM: Leveraging LLM agents for ontology matching. *Proceedings of the VLDB Endowment, 18*, 516–529. Consensus record: https://consensus.app/papers/agentom-leveraging-llm-agents-for-ontology-matching-qiang-wang/1ff1e2abb0f255299ecb808951ceaf6b/?utm_source=chatgpt From a8b95a3ab336986fd8e3da6b5a9f267cbf08a253 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 19:49:03 +0900 Subject: [PATCH 53/59] docs: refresh product technical gap baseline after doctoring repair --- docs/product-technical-gap-baseline.md | 27 +++++++++++++------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index bf02efff..34c62b3f 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -8,13 +8,13 @@ This file records code-current product and technical gaps. Exact PR/check/run co Protected/default `main` remains `f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425`; only the bootstrap README is shipped there and no ConceptWeave release exists yet. -The active dependency stack observed immediately before this baseline repair is: +The active dependency stack observed immediately before this baseline refresh is: -1. Foundation PR #1 — pre-repair exact head `707e687bb510d3f1fd5d1bdc94e590b30ffbb641`, open/non-Draft/mergeable. Product `33850379502` / job `100951576248`, SAST `33850379638`, and Security `33850379525` are queued. Product is on explicit `ubuntu-24.04` with `runner_id=0` and no steps, so the strengthened research-reference test has not yet recorded a real RED. The baseline itself was stale and is the causal reason for this documentation-only successor head; predecessor terminal evidence does not transfer. -2. Client Consumption PR #5 — pre-restack exact head `0f878fc796feb93406dedbbad85351282d3ed7bc`, Draft/open/mergeable and stacked on Foundation. Product `33850422999` / job `100951717500` is pre-runner queued. Detached-artifact API and public-document drift have real hosted RED→repair lineage. The next intended Client RED remains the missing language-neutral supersession/publication JSON Schema plus valid/invalid fixtures, only after the inherited Foundation doctoring lane passes and that Client boundary actually executes. -3. Source Observation PR #6 — pre-restack exact head `7edb2e03c6adeabdc7ffc5c5ecdbc8f4362d3ab3`, Draft/open/mergeable and stacked on Foundation. Product `33850442339` / job `100951774004` is pre-runner queued. The prior UTC provenance RED executed and was repaired. The current registry-identity test requires immutable snapshot/source-receipt provenance to respect the Source Observation port's opaque ≤128-byte lowercase multiword `snake_case` registry-key boundary. Predecessor `c9af2255...` reached a real hosted runner but failed at formatting before the semantic test. Production registry-key validation remains intentionally unchanged until the intended semantic RED executes. +1. Foundation PR #1 — pre-refresh exact head `17d604150a7ab334412b41136d8313c1d898e9d5`, open/non-Draft/mergeable. The strengthened research-reference contract from `707e687...` is now satisfied in repository state: `docs/doctoring/REFERENCES.md` binds He et al. to the CEUR/ISWC 2023 record and Amini et al. to the Springer LNCS version of record published in 2025. Product `33864844157` / job `100997237173`, SAST `33864844067`, and Security `33864843880` are still non-terminal; Product has not yet produced terminal hosted evidence. The prior exact file state was deterministically incompatible with the test predicate because neither authoritative record existed in `REFERENCES.md`; the minimal repair changed only that bibliography file. This baseline refresh is documentation-only and predecessor terminal evidence does not transfer. +2. Client Consumption PR #5 — pre-restack exact head `97fb952f2938f562bd5d9364ff38a105f224fd0e`, Draft/open and stacked on Foundation through a two-parent non-force merge. Product `33864899059` / job `100997413570` is queued. Old-child → new-child comparison changed only `docs/doctoring/REFERENCES.md`, so the Client supersession/publication delta is preserved. The next intended Client RED is the missing language-neutral supersession/publication JSON Schema plus valid/invalid fixtures once inherited Foundation quality reaches terminal evidence. +3. Source Observation PR #6 — pre-restack exact head `67e60dedfc7bb5fe3f7fe6e2aedb673678e38911`, Draft/open and stacked on Foundation through a two-parent non-force merge. Product `33864917112` / job `100997467770` is queued. Old-child → new-child comparison changed only `docs/doctoring/REFERENCES.md`, preserving the registry-identity lineage. The prior UTC provenance RED executed and was repaired. The current registry-identity test still requires immutable snapshot/source-receipt provenance to respect the Source Observation port's opaque ≤128-byte lowercase multiword `snake_case` registry-key boundary; predecessor `c9af2255...` reached a real hosted runner but failed at formatting before that semantic test. -Both child PRs have no submitted reviews or inline review threads in the current fresh read. Foundation's returned inline threads are resolved; its Noema APPROVE belongs to predecessor `bba351b...` and is not current-head independent approval. +Foundation's returned inline review threads are resolved; its only returned APPROVE remains Noema on predecessor `bba351b...` and is not current-head independent approval. Fresh review of #5/#6 remains required after every material head movement; no predecessor review is promoted to current acceptance. ## Foundation capability status @@ -26,19 +26,18 @@ Both child PRs have no submitted reviews or inline review threads in the current | Client Consumption | ACTIVE_CHILD | Offline Published+Authoritative admission, explicit compatibility, exact resolution/diff, canonical digest verification, detached artifact verification and explicit supersession validation exist. Language-neutral public supersession/publication schema/fixtures remain intentionally absent pending the real current-head RED. | | Quality gate | ACTIVE_PR | Rust 1.98.0, unsafe forbidden, public docs required, exact checkout, fmt, Clippy, tests, rustdoc, owned 100% coverage, Draft-2020-12 schema fixtures, lock freshness and clean-tree checks. Every head movement requires fresh exact-head evidence. | | Security / dependency review | BLOCKED_OWNER | Foundation predecessor Security reached a real runner and failed closed because authoritative GitHub Dependency Review availability was not satisfied. `.github#810` owns the central repair; scanner substitution and 403-as-success are forbidden. | -| Review / runner admission | BLOCKED_OWNER | Central queue pressure has fallen materially, but current ConceptWeave Product canaries remain unassigned. `.github#712/#1531/#1796` own the central admission/review-amplification paths. Queue depth alone is not consumer GREEN. | -| Standards / research | ACTIVE_PR | Stable recommendations remain distinct from drafts. The current doctoring RED contract binds adopted Model Alignment evidence to authoritative bibliography records: He et al. to CEUR/ISWC 2023 and Amini et al. to the Springer LNCS version of record published in 2025 while retaining the KGSWC 2024 study/conference lineage in traceability. `REFERENCES.md` remains intentionally unchanged until the test executes and records that RED. | +| Review / runner admission | BLOCKED_OWNER | Central queue pressure has fallen materially, but current ConceptWeave Product canaries remain non-terminal. `.github#712/#1531/#1796` own the central admission/review-amplification paths. Queue depth alone is not consumer GREEN. | +| Standards / research | REPAIRED_PENDING_CI | Primary-source doctoring now binds He et al. to CEUR Workshop Proceedings Vol. 3632 / ISWC 2023 and Amini et al. to the Springer LNCS 15459 version of record, pp. 17–31, published in 2025 while retaining KGSWC 2024 study/conference lineage in traceability. Exact-head hosted Product/security/SAST remain non-terminal, so no merge-ready GREEN is claimed. | | Release | NOT_STARTED | No immutable ConceptWeave release exists. Version/CHANGELOG/tag/package/semantic_release/SBOM/provenance/reproducibility/rollback are required on the exact protected release head. | ## Central control-plane evidence -Protected central source is `.github/main@07db37e5e42c63ba40ac66f22ef74e4f8836ce9a` at this snapshot. This is evidence only, not a ConceptWeave dependency. +Protected central source is `.github/main@9c7d28a1768fb5003a0ed46e4d7a957b9e64a7f2` at this snapshot. This is evidence only, not a ConceptWeave dependency. -- `.github#1796` queue-ownership repair #1821 is integrated. Organization sweep no longer owns repository-wide queued/in-progress Actions inventory or broad cancellation; native per-PR concurrency and repository-local exact-head coalescing own supersession. -- `.github#1840` removed required-check-completion `workflow_run` fanout from the merge scheduler while retaining native PR/review/protected-branch-push/explicit-dispatch events and bounded recovery. GitHub auto-merge owns required-check completion rather than another scheduler run. -- `.github#1841` made uncovered CodeQL repositories self-remediating through idempotent OpenCode-owned setup PRs created from trusted central main. -- `.github#1842` keeps those generated repository-local CodeQL workflows off pull-request heads, leaving central `codeql-pr.yml` as the single PR-head scanner and isolating audit concurrency by trigger. -- Fresh `.github` queued inventory is `271`, materially below earlier `991`, `914`, and ~`1,900` observations. This is diagnostic progress, not acceptance: current ConceptWeave Product jobs still lack runner identity, exact checkout, steps, and terminal evidence. +- `.github#1821` queue-ownership repair remains integrated: organization sweep no longer owns repository-wide queued/in-progress Actions inventory or broad cancellation; native per-PR concurrency and repository-local exact-head coalescing own supersession. +- Subsequent central consolidation removed merge-scheduler required-check fanout, consolidated CodeQL ownership and further reduced duplicate required-workflow work. +- `.github#1847` is now protected-main state and consolidates empty-PR cleanup into the scheduler/control-plane path rather than preserving another independent automation lane. +- Fresh `.github` queued inventory is `254`, materially below earlier `289`, `991`, `914`, and ~`1,900` observations. This is diagnostic progress, not acceptance: current ConceptWeave jobs still lack terminal exact-head evidence. ## P0 product gaps after the current TDD lanes @@ -64,4 +63,4 @@ Protected central source is `.github/main@07db37e5e42c63ba40ac66f22ef74e4f8836ce - Client Consumption depends only on governed release contracts, never generator-private classes, prompts, persistence tables or orchestration state. - `semantic-data-portal` remains catalog/governance/consumption rather than ConceptWeave persistence; `context-graph-contracts` owns interop contracts; `enterprise-architecture-core` owns EA; `contextual-orchestrator` owns provider routing. - Consuming products retain tenant/purpose authorization and physical query execution. -- Published semantic truth is immutable; corrections create a new release plus supersession evidence rather than in-place overwrite. +- Published semantic truth is immutable; corrections create a new release plus supersession evidence rather than in-place overwrite. \ No newline at end of file From 18249652aca7341da2446f685a06b4b97f495f4d Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 21:04:47 +0900 Subject: [PATCH 54/59] test(contract): reject truth-state mismatch before publication --- .github/workflows/product.yml | 5 +++++ ...ntic-candidate.invalid-state-truth-mismatch.json | 13 +++++++++++++ 2 files changed, 18 insertions(+) create mode 100644 contracts/fixtures/semantic-candidate.invalid-state-truth-mismatch.json diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index d49caaf1..b14f5465 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -81,6 +81,11 @@ jobs: -s contracts/semantic-candidate.schema.json \ -d contracts/fixtures/semantic-candidate.invalid-published-truth.json \ --invalid + npx --yes ajv-cli@5.0.0 test \ + --spec=draft2020 \ + -s contracts/semantic-candidate.schema.json \ + -d contracts/fixtures/semantic-candidate.invalid-state-truth-mismatch.json \ + --invalid - name: Lockfile freshness run: | diff --git a/contracts/fixtures/semantic-candidate.invalid-state-truth-mismatch.json b/contracts/fixtures/semantic-candidate.invalid-state-truth-mismatch.json new file mode 100644 index 00000000..759a0518 --- /dev/null +++ b/contracts/fixtures/semantic-candidate.invalid-state-truth-mismatch.json @@ -0,0 +1,13 @@ +{ + "candidate_id": "candidate-draft-authoritative", + "kind": "concept", + "truth_status": "authoritative", + "publication_state": "draft", + "evidence": [ + { + "source_id": "source-1", + "source_digest": "sha256:abc", + "location": "public.orders" + } + ] +} From 9a6aa93ed05dd9cc56825258e072b222d80f85de Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 21:05:38 +0900 Subject: [PATCH 55/59] fix(contract): align public truth status with publication state --- contracts/semantic-candidate.schema.json | 78 ++++++++++++++++++++++++ 1 file changed, 78 insertions(+) diff --git a/contracts/semantic-candidate.schema.json b/contracts/semantic-candidate.schema.json index aa5955b8..3b0b9c0b 100644 --- a/contracts/semantic-candidate.schema.json +++ b/contracts/semantic-candidate.schema.json @@ -77,6 +77,58 @@ } }, "allOf": [ + { + "if": { + "properties": { + "publication_state": {"const": "draft"} + }, + "required": ["publication_state"] + }, + "then": { + "properties": { + "truth_status": {"const": "inferred"} + } + } + }, + { + "if": { + "properties": { + "publication_state": {"const": "proposed"} + }, + "required": ["publication_state"] + }, + "then": { + "properties": { + "truth_status": {"const": "proposed"} + } + } + }, + { + "if": { + "properties": { + "publication_state": {"const": "validated"} + }, + "required": ["publication_state"] + }, + "then": { + "properties": { + "truth_status": {"const": "inferred"} + } + } + }, + { + "if": { + "properties": { + "publication_state": {"const": "reviewed"} + }, + "required": ["publication_state"] + }, + "then": { + "properties": { + "truth_status": {"const": "inferred"} + } + } + }, { "if": { "properties": { @@ -89,6 +141,32 @@ "truth_status": {"const": "authoritative"} } } + }, + { + "if": { + "properties": { + "publication_state": {"const": "superseded"} + }, + "required": ["publication_state"] + }, + "then": { + "properties": { + "truth_status": {"const": "superseded"} + } + } + }, + { + "if": { + "properties": { + "publication_state": {"const": "rejected"} + }, + "required": ["publication_state"] + }, + "then": { + "properties": { + "truth_status": {"const": "rejected"} + } + } } ] } From a379c7230253c3aaa0c6fc8486cc39be421eca35 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 21:10:52 +0900 Subject: [PATCH 56/59] docs: refresh gap baseline after truth-state contract repair --- docs/product-technical-gap-baseline.md | 35 +++++++++++++------------- 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 34c62b3f..bedacd5e 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -2,42 +2,41 @@ **Snapshot:** 2026-09-04 -This file records code-current product and technical gaps. Exact PR/check/run coordinates below are evidence snapshots, not mutable-head dependencies. GitHub protected-branch, PR, issue, and workflow state remains authoritative whenever it advances after this snapshot. A documentation commit necessarily creates a successor Foundation head, so the Foundation SHA below is explicitly the pre-repair head observed immediately before this file was refreshed. +This file records code-current product and technical gaps. Exact PR/check/run coordinates below are evidence snapshots, not mutable-head dependencies. Live protected-branch, PR, issue and workflow state wins whenever it advances after this snapshot. Because this documentation update itself creates a successor Foundation head, the Foundation SHA below is explicitly the exact pre-refresh head. ## Protected truth and active stack -Protected/default `main` remains `f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425`; only the bootstrap README is shipped there and no ConceptWeave release exists yet. +Protected/default `main` remains `f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425`; only the bootstrap state is shipped there and no immutable ConceptWeave release exists yet. The active dependency stack observed immediately before this baseline refresh is: -1. Foundation PR #1 — pre-refresh exact head `17d604150a7ab334412b41136d8313c1d898e9d5`, open/non-Draft/mergeable. The strengthened research-reference contract from `707e687...` is now satisfied in repository state: `docs/doctoring/REFERENCES.md` binds He et al. to the CEUR/ISWC 2023 record and Amini et al. to the Springer LNCS version of record published in 2025. Product `33864844157` / job `100997237173`, SAST `33864844067`, and Security `33864843880` are still non-terminal; Product has not yet produced terminal hosted evidence. The prior exact file state was deterministically incompatible with the test predicate because neither authoritative record existed in `REFERENCES.md`; the minimal repair changed only that bibliography file. This baseline refresh is documentation-only and predecessor terminal evidence does not transfer. -2. Client Consumption PR #5 — pre-restack exact head `97fb952f2938f562bd5d9364ff38a105f224fd0e`, Draft/open and stacked on Foundation through a two-parent non-force merge. Product `33864899059` / job `100997413570` is queued. Old-child → new-child comparison changed only `docs/doctoring/REFERENCES.md`, so the Client supersession/publication delta is preserved. The next intended Client RED is the missing language-neutral supersession/publication JSON Schema plus valid/invalid fixtures once inherited Foundation quality reaches terminal evidence. -3. Source Observation PR #6 — pre-restack exact head `67e60dedfc7bb5fe3f7fe6e2aedb673678e38911`, Draft/open and stacked on Foundation through a two-parent non-force merge. Product `33864917112` / job `100997467770` is queued. Old-child → new-child comparison changed only `docs/doctoring/REFERENCES.md`, preserving the registry-identity lineage. The prior UTC provenance RED executed and was repaired. The current registry-identity test still requires immutable snapshot/source-receipt provenance to respect the Source Observation port's opaque ≤128-byte lowercase multiword `snake_case` registry-key boundary; predecessor `c9af2255...` reached a real hosted runner but failed at formatting before that semantic test. +1. Foundation PR #1 — pre-refresh exact head `9a6aa93ed05dd9cc56825258e072b222d80f85de`, open/non-Draft/mergeable. A public-contract mismatch was reproduced locally before the repair: the Draft 2020-12 `semantic-candidate` schema accepted `publication_state=draft` together with `truth_status=authoritative`, although the Rust domain maps Draft/Validated/Reviewed to Inferred, Proposed to Proposed, Published to Authoritative, Superseded to Superseded and Rejected to Rejected. Test-first `18249652...` added an invalid fixture and Product AJV assertion; minimum fix `9a6aa93...` makes the language-neutral schema enforce the same state/truth mapping as Rust. Local exhaustive verification covered all 7 publication states × 6 truth statuses with zero mapping mismatches. This is local exact-contract GREEN, not hosted exact-head GREEN. Product `33871067722` / job `101016914449`, SAST `33871067759`, and Security `33871067702` remain non-terminal; Product has no executed steps yet. +2. Client Consumption PR #5 — pre-refresh exact head `4a771af962306febb4318aed4de48254d96f32f9`, Draft/open. It non-force adopted the Foundation truth-state contract. The first restack accidentally replaced the child-specific Product JSON-contract checks with the narrower Foundation workflow; that repair finding was fixed immediately by `4a771af...`. Old-child `67c104...` → current comparison now changes only the new candidate mismatch fixture, the candidate schema and five added Product-workflow lines, while all pre-existing semantic-release and supersession checks remain intact. Product `33871312459` is non-terminal. The next Client RED remains its missing language-neutral supersession/publication schema and fixtures after inherited Foundation quality is terminal. +3. Source Observation PR #6 — pre-refresh exact head `e72345ac9f407e5732b3e3cc5a2d78b55b10cad2`, Draft/open. It non-force adopted the same Foundation contract; old-child `a0197ba...` → current comparison changes only the candidate mismatch fixture, candidate schema and five Product-workflow lines. Source Observation semantic delta is preserved. Product `33871231617` is queued. The registry-identity test still requires immutable snapshot/source-receipt provenance to obey the Source Observation port's opaque ≤128-byte lowercase multiword `snake_case` key boundary; production `PostgresSchemaSnapshot::new` still validates this field only as nonblank, so that semantic lane remains open. -Foundation's returned inline review threads are resolved; its only returned APPROVE remains Noema on predecessor `bba351b...` and is not current-head independent approval. Fresh review of #5/#6 remains required after every material head movement; no predecessor review is promoted to current acceptance. +Predecessor reviews/checks never transfer to successor heads. No force-push, destructive rebase, self-approval, fail-open scanner substitution or routine administrator bypass is acceptance evidence. ## Foundation capability status | Area | Status | Evidence / next verification | | --- | --- | --- | -| Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define ConceptWeave ownership of `observe -> discover -> propose -> align -> validate -> review -> publish`, governed immutable semantic releases, and the stable Client contract. Foreign product truth remains behind released/versioned ports and ACLs. | -| Truth/publication lifecycle | ACTIVE_PR | Rust domain lifecycle enforces explicit governance authorization at steward/publication boundaries, immutable publication and supersession semantics, and evidence-bound authoritative state. Current returned inline review threads are resolved. | -| Source Observation | ACTIVE_CHILD | Immutable PostgreSQL table/column/PK/unique/FK/CHECK evidence, exact identifiers, canonical snapshot digest syntax, UTC provenance, exact receipts, bounded request budgets/cancellation and opaque source registry keys exist. Registry-key consistency at the immutable snapshot boundary is the current TDD lane. No live PostgreSQL adapter is claimed; ADR 0004 remains Proposed. | -| Client Consumption | ACTIVE_CHILD | Offline Published+Authoritative admission, explicit compatibility, exact resolution/diff, canonical digest verification, detached artifact verification and explicit supersession validation exist. Language-neutral public supersession/publication schema/fixtures remain intentionally absent pending the real current-head RED. | +| Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define ConceptWeave ownership of `observe -> discover -> propose -> align -> validate -> review -> publish`, governed immutable semantic releases and stable Client contracts. Foreign product truth remains behind released/versioned ports and ACLs. | +| Truth/publication lifecycle | REPAIRED_PENDING_CI | Rust already derives truth status from publication state. The public JSON Schema now enforces the same mapping for every state, preventing pre-publication `authoritative` claims by non-Rust consumers. New invalid fixture proves the previously admitted Draft+Authoritative combination. Hosted exact-head Product evidence is still non-terminal. | +| Source Observation | ACTIVE_CHILD | Immutable PostgreSQL table/column/PK/unique/FK/CHECK evidence, exact identifiers, canonical snapshot digest syntax, UTC provenance, receipts, bounded request budgets/cancellation and opaque source registry keys exist. Registry-key consistency at the immutable snapshot boundary is the current semantic TDD lane. No live PostgreSQL adapter is claimed; ADR 0004 remains Proposed. | +| Client Consumption | ACTIVE_CHILD | Offline Published+Authoritative admission, compatibility, exact resolution/diff, canonical digest verification, detached artifact verification and explicit supersession validation exist. Public supersession/publication schema/fixtures remain the next Client contract lane after Foundation terminal quality. | | Quality gate | ACTIVE_PR | Rust 1.98.0, unsafe forbidden, public docs required, exact checkout, fmt, Clippy, tests, rustdoc, owned 100% coverage, Draft-2020-12 schema fixtures, lock freshness and clean-tree checks. Every head movement requires fresh exact-head evidence. | -| Security / dependency review | BLOCKED_OWNER | Foundation predecessor Security reached a real runner and failed closed because authoritative GitHub Dependency Review availability was not satisfied. `.github#810` owns the central repair; scanner substitution and 403-as-success are forbidden. | -| Review / runner admission | BLOCKED_OWNER | Central queue pressure has fallen materially, but current ConceptWeave Product canaries remain non-terminal. `.github#712/#1531/#1796` own the central admission/review-amplification paths. Queue depth alone is not consumer GREEN. | -| Standards / research | REPAIRED_PENDING_CI | Primary-source doctoring now binds He et al. to CEUR Workshop Proceedings Vol. 3632 / ISWC 2023 and Amini et al. to the Springer LNCS 15459 version of record, pp. 17–31, published in 2025 while retaining KGSWC 2024 study/conference lineage in traceability. Exact-head hosted Product/security/SAST remain non-terminal, so no merge-ready GREEN is claimed. | +| Security / dependency review | BLOCKED_OWNER | Prior Security evidence showed authoritative GitHub Dependency Review availability was not satisfied. `.github#810` owns central repair; scanner substitution and 403-as-success are forbidden. | +| Review / runner admission | BLOCKED_OWNER | Central queue pressure is materially lower than its peak but current ConceptWeave canaries remain non-terminal. `.github#712/#1531/#1796` own central admission/review-amplification. Queue depth alone is not consumer GREEN. | +| Standards / research | REPAIRED_PENDING_CI | Doctoring binds He et al. to CEUR/ISWC 2023 and Amini et al. to the Springer LNCS 15459 version of record published in 2025 while retaining KGSWC 2024 study/conference lineage in traceability. Hosted exact-head evidence remains non-terminal. | | Release | NOT_STARTED | No immutable ConceptWeave release exists. Version/CHANGELOG/tag/package/semantic_release/SBOM/provenance/reproducibility/rollback are required on the exact protected release head. | ## Central control-plane evidence -Protected central source is `.github/main@9c7d28a1768fb5003a0ed46e4d7a957b9e64a7f2` at this snapshot. This is evidence only, not a ConceptWeave dependency. +Protected central source is `.github/main@c31d2e5471fc5daf9d72ff67cde6a8874b736deb` at this snapshot, after merged #1852 aligned current-main workflow contracts. This is evidence only, not a ConceptWeave dependency. - `.github#1821` queue-ownership repair remains integrated: organization sweep no longer owns repository-wide queued/in-progress Actions inventory or broad cancellation; native per-PR concurrency and repository-local exact-head coalescing own supersession. -- Subsequent central consolidation removed merge-scheduler required-check fanout, consolidated CodeQL ownership and further reduced duplicate required-workflow work. -- `.github#1847` is now protected-main state and consolidates empty-PR cleanup into the scheduler/control-plane path rather than preserving another independent automation lane. -- Fresh `.github` queued inventory is `254`, materially below earlier `289`, `991`, `914`, and ~`1,900` observations. This is diagnostic progress, not acceptance: current ConceptWeave jobs still lack terminal exact-head evidence. +- Later consolidation removed merge-scheduler required-check fanout, centralized CodeQL PR ownership and consolidated empty-PR/quality lanes. +- Fresh `.github` queued inventory is `245`. This is far below the ~1,900 peak but above some recent lower observations, so it is neither terminal recovery nor consumer acceptance. ## P0 product gaps after the current TDD lanes @@ -63,4 +62,4 @@ Protected central source is `.github/main@9c7d28a1768fb5003a0ed46e4d7a957b9e64a7 - Client Consumption depends only on governed release contracts, never generator-private classes, prompts, persistence tables or orchestration state. - `semantic-data-portal` remains catalog/governance/consumption rather than ConceptWeave persistence; `context-graph-contracts` owns interop contracts; `enterprise-architecture-core` owns EA; `contextual-orchestrator` owns provider routing. - Consuming products retain tenant/purpose authorization and physical query execution. -- Published semantic truth is immutable; corrections create a new release plus supersession evidence rather than in-place overwrite. \ No newline at end of file +- Published semantic truth is immutable; corrections create a new release plus supersession evidence rather than in-place overwrite. From 8e8783286eac7567803568d9a91010daaf028074 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 23:51:33 +0900 Subject: [PATCH 57/59] ci(actions): isolate superseded pull request runs (#14) Signed-off-by: Seongho Bae --- .github/workflows/product.yml | 4 ++-- scripts/check_ci_contract.py | 6 +++++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index b14f5465..8863e1ee 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -6,8 +6,8 @@ on: branches: [main] concurrency: - group: conceptweave-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: true + group: ${{ github.workflow }}-${{ github.repository }}-${{ github.event.pull_request.number || github.run_id }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} permissions: contents: read diff --git a/scripts/check_ci_contract.py b/scripts/check_ci_contract.py index e0e7130e..1342082b 100644 --- a/scripts/check_ci_contract.py +++ b/scripts/check_ci_contract.py @@ -14,7 +14,8 @@ def main() -> int: required_fragments = ( "runs-on: ubuntu-24.04", - "cancel-in-progress: true", + "group: ${{ github.workflow }}-${{ github.repository }}-${{ github.event.pull_request.number || github.run_id }}", + "cancel-in-progress: ${{ github.event_name == 'pull_request' }}", "actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0", "COVERAGE_TOOLCHAIN: nightly-2026-08-20", 'rustup toolchain install "$COVERAGE_TOOLCHAIN" --profile minimal --component llvm-tools-preview', @@ -31,6 +32,9 @@ def main() -> int: "evidence demonstrates selective floating-image starvation" ) + if "github.event.pull_request.number || github.ref" in workflow: + raise SystemExit("Product CI must isolate non-PR runs by run_id") + return 0 From 5cdd319b9425989e632149b243a3308dd630c0ae Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sat, 5 Sep 2026 07:34:10 +0900 Subject: [PATCH 58/59] ci(actions): skip draft pull request jobs Signed-off-by: Seongho Bae --- .github/workflows/product.yml | 4 +++- scripts/check_ci_contract.py | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/.github/workflows/product.yml b/.github/workflows/product.yml index 8863e1ee..8700247d 100644 --- a/.github/workflows/product.yml +++ b/.github/workflows/product.yml @@ -2,11 +2,12 @@ name: Product on: pull_request: + types: [opened, synchronize, reopened, ready_for_review, converted_to_draft, closed] push: branches: [main] concurrency: - group: ${{ github.workflow }}-${{ github.repository }}-${{ github.event.pull_request.number || github.run_id }} + group: ${{ github.workflow }}-${{ github.repository }}-${{ github.event_name == 'pull_request' && github.event.pull_request.number || github.run_id }} cancel-in-progress: ${{ github.event_name == 'pull_request' }} permissions: @@ -14,6 +15,7 @@ permissions: jobs: rust-quality: + if: ${{ github.event_name != 'pull_request' || (github.event.action != 'closed' && github.event.pull_request.draft == false) }} runs-on: ubuntu-24.04 env: COVERAGE_TOOLCHAIN: nightly-2026-08-20 diff --git a/scripts/check_ci_contract.py b/scripts/check_ci_contract.py index 1342082b..f23b6c3f 100644 --- a/scripts/check_ci_contract.py +++ b/scripts/check_ci_contract.py @@ -14,8 +14,10 @@ def main() -> int: required_fragments = ( "runs-on: ubuntu-24.04", - "group: ${{ github.workflow }}-${{ github.repository }}-${{ github.event.pull_request.number || github.run_id }}", + "types: [opened, synchronize, reopened, ready_for_review, converted_to_draft, closed]", + "group: ${{ github.workflow }}-${{ github.repository }}-${{ github.event_name == 'pull_request' && github.event.pull_request.number || github.run_id }}", "cancel-in-progress: ${{ github.event_name == 'pull_request' }}", + "if: ${{ github.event_name != 'pull_request' || (github.event.action != 'closed' && github.event.pull_request.draft == false) }}", "actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0", "COVERAGE_TOOLCHAIN: nightly-2026-08-20", 'rustup toolchain install "$COVERAGE_TOOLCHAIN" --profile minimal --component llvm-tools-preview', From b538470c963e6524ddc0c3f652a46a4fc8265150 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sat, 5 Sep 2026 10:47:30 +0900 Subject: [PATCH 59/59] docs(gap): refresh live ConceptWeave authority and dependency-review RCA --- docs/product-technical-gap-baseline.md | 79 +++++++++++++++----------- 1 file changed, 45 insertions(+), 34 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index bedacd5e..f7ba7dea 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1,18 +1,20 @@ # Product / Technical Gap Baseline -**Snapshot:** 2026-09-04 +**Snapshot:** 2026-09-05 -This file records code-current product and technical gaps. Exact PR/check/run coordinates below are evidence snapshots, not mutable-head dependencies. Live protected-branch, PR, issue and workflow state wins whenever it advances after this snapshot. Because this documentation update itself creates a successor Foundation head, the Foundation SHA below is explicitly the exact pre-refresh head. +This file records code-current product and technical gaps. Exact PR/check/run coordinates are evidence snapshots, not mutable-head dependencies. Live protected-branch, PR, issue and workflow state wins whenever it advances after this snapshot. Because this documentation update creates a Foundation successor, the Foundation SHA below is the exact pre-refresh head; PR metadata must be refreshed to the resulting successor SHA. ## Protected truth and active stack -Protected/default `main` remains `f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425`; only the bootstrap state is shipped there and no immutable ConceptWeave release exists yet. +Protected/default `main` remains `f4f440dd58c77d7cd90dff8a1eb2eeb9a9940425`; only the bootstrap state is shipped there and no immutable ConceptWeave release exists. -The active dependency stack observed immediately before this baseline refresh is: +The active roots observed immediately before this baseline refresh are: -1. Foundation PR #1 — pre-refresh exact head `9a6aa93ed05dd9cc56825258e072b222d80f85de`, open/non-Draft/mergeable. A public-contract mismatch was reproduced locally before the repair: the Draft 2020-12 `semantic-candidate` schema accepted `publication_state=draft` together with `truth_status=authoritative`, although the Rust domain maps Draft/Validated/Reviewed to Inferred, Proposed to Proposed, Published to Authoritative, Superseded to Superseded and Rejected to Rejected. Test-first `18249652...` added an invalid fixture and Product AJV assertion; minimum fix `9a6aa93...` makes the language-neutral schema enforce the same state/truth mapping as Rust. Local exhaustive verification covered all 7 publication states × 6 truth statuses with zero mapping mismatches. This is local exact-contract GREEN, not hosted exact-head GREEN. Product `33871067722` / job `101016914449`, SAST `33871067759`, and Security `33871067702` remain non-terminal; Product has no executed steps yet. -2. Client Consumption PR #5 — pre-refresh exact head `4a771af962306febb4318aed4de48254d96f32f9`, Draft/open. It non-force adopted the Foundation truth-state contract. The first restack accidentally replaced the child-specific Product JSON-contract checks with the narrower Foundation workflow; that repair finding was fixed immediately by `4a771af...`. Old-child `67c104...` → current comparison now changes only the new candidate mismatch fixture, the candidate schema and five added Product-workflow lines, while all pre-existing semantic-release and supersession checks remain intact. Product `33871312459` is non-terminal. The next Client RED remains its missing language-neutral supersession/publication schema and fixtures after inherited Foundation quality is terminal. -3. Source Observation PR #6 — pre-refresh exact head `e72345ac9f407e5732b3e3cc5a2d78b55b10cad2`, Draft/open. It non-force adopted the same Foundation contract; old-child `a0197ba...` → current comparison changes only the candidate mismatch fixture, candidate schema and five Product-workflow lines. Source Observation semantic delta is preserved. Product `33871231617` is queued. The registry-identity test still requires immutable snapshot/source-receipt provenance to obey the Source Observation port's opaque ≤128-byte lowercase multiword `snake_case` key boundary; production `PostgresSchemaSnapshot::new` still validates this field only as nonblank, so that semantic lane remains open. +1. Foundation PR #1 — pre-refresh exact head `5cdd319b9425989e632149b243a3308dd630c0ae`, Draft/open/mergeable. The current Foundation makes Product CI draft-aware while keeping Ready/non-Draft quality requirements intact. Product CI still cannot materialize from protected `main` because that branch does not yet contain `.github/workflows/product.yml`. +2. Product-CI bootstrap PR #35 — exact head `daa543ce2cc2b2eb6d35a7265abcf2a7466e7381`, open/non-Draft/mergeable. It adds only the pull-request form of Product CI so #1 can later be marked Ready without a no-op commit. Exact-head CodeQL PR, Security Scan and SAST Semgrep remain queued; `Security Scan / Detect changed scope` is pre-runner with no steps and no runner assignment, and no independent submitted review exists yet. +3. Client Consumption PR #5 — exact head `cbb9cda0c93d8b762195423834f1d6a27dbfa613`, Draft/open/mergeable. The current source retains language-neutral semantic-release admission, integrity, compatibility, diff/resolution and supersession validation. Previously valid review findings are source-repaired, but current protected evidence remains independently required. +4. Source Observation PR #6 — exact head `d255f5c08a621024809c7e076989eccf0662a330`, Draft/open/mergeable. PostgreSQL targeted `ON DELETE SET NULL (...)` / `SET DEFAULT (...)` column provenance and registry/ACL-resolved source identity are source-repaired. The next P0 slice is the concrete bounded read-only PostgreSQL adapter. +5. Zotero Research Classification root PR #9 — exact head `cda546672cd95b5f8bed7024f70e4e6b39a134c8`, Draft/open/mergeable. The dependent research/write-back stack remains proposal/review oriented and does not elevate local classifier output to semantic authority. Predecessor reviews/checks never transfer to successor heads. No force-push, destructive rebase, self-approval, fail-open scanner substitution or routine administrator bypass is acceptance evidence. @@ -21,38 +23,47 @@ Predecessor reviews/checks never transfer to successor heads. No force-push, des | Area | Status | Evidence / next verification | | --- | --- | --- | | Product boundary | ACTIVE_PR | PRD/TRD/ADR/context map define ConceptWeave ownership of `observe -> discover -> propose -> align -> validate -> review -> publish`, governed immutable semantic releases and stable Client contracts. Foreign product truth remains behind released/versioned ports and ACLs. | -| Truth/publication lifecycle | REPAIRED_PENDING_CI | Rust already derives truth status from publication state. The public JSON Schema now enforces the same mapping for every state, preventing pre-publication `authoritative` claims by non-Rust consumers. New invalid fixture proves the previously admitted Draft+Authoritative combination. Hosted exact-head Product evidence is still non-terminal. | -| Source Observation | ACTIVE_CHILD | Immutable PostgreSQL table/column/PK/unique/FK/CHECK evidence, exact identifiers, canonical snapshot digest syntax, UTC provenance, receipts, bounded request budgets/cancellation and opaque source registry keys exist. Registry-key consistency at the immutable snapshot boundary is the current semantic TDD lane. No live PostgreSQL adapter is claimed; ADR 0004 remains Proposed. | -| Client Consumption | ACTIVE_CHILD | Offline Published+Authoritative admission, compatibility, exact resolution/diff, canonical digest verification, detached artifact verification and explicit supersession validation exist. Public supersession/publication schema/fixtures remain the next Client contract lane after Foundation terminal quality. | +| Truth/publication lifecycle | REPAIRED_PENDING_CI | Rust and the public Draft 2020-12 semantic-candidate schema enforce compatible publication-state/truth-status semantics. Hosted exact-head Product evidence still requires the bootstrap workflow on protected `main`. | +| Source Observation | ACTIVE_CHILD | Immutable PostgreSQL table/column/PK/unique/FK/CHECK evidence, exact identifiers, targeted delete-column provenance, canonical snapshot digest syntax, UTC provenance, receipts, bounded request budgets/cancellation and registry-authorized opaque source identity exist. No live PostgreSQL adapter is claimed; ADR 0004 remains Proposed. | +| Client Consumption | ACTIVE_CHILD | Offline Published+Authoritative admission, compatibility, exact resolution/diff, canonical digest verification, detached artifact verification and explicit supersession validation exist. Current exact-head protected evidence and prerequisite integration remain outstanding. | | Quality gate | ACTIVE_PR | Rust 1.98.0, unsafe forbidden, public docs required, exact checkout, fmt, Clippy, tests, rustdoc, owned 100% coverage, Draft-2020-12 schema fixtures, lock freshness and clean-tree checks. Every head movement requires fresh exact-head evidence. | -| Security / dependency review | BLOCKED_OWNER | Prior Security evidence showed authoritative GitHub Dependency Review availability was not satisfied. `.github#810` owns central repair; scanner substitution and 403-as-success are forbidden. | -| Review / runner admission | BLOCKED_OWNER | Central queue pressure is materially lower than its peak but current ConceptWeave canaries remain non-terminal. `.github#712/#1531/#1796` own central admission/review-amplification. Queue depth alone is not consumer GREEN. | -| Standards / research | REPAIRED_PENDING_CI | Doctoring binds He et al. to CEUR/ISWC 2023 and Amini et al. to the Springer LNCS 15459 version of record published in 2025 while retaining KGSWC 2024 study/conference lineage in traceability. Hosted exact-head evidence remains non-terminal. | +| Security / dependency review | CONSUMER_REVALIDATION_PENDING | The earlier public non-fork exact-range HTTP 403 was traced to an uninitialized repository dependency graph, not to a retryable central workflow defect. `.github#1873` was closed unmerged after enabling Dependabot vulnerability alerts initialized affected graphs and the same exact comparison returned HTTP 200. The hard gate remains fail closed; a current ConceptWeave head must still execute the pinned Dependency Review action successfully before acceptance. | +| Review / runner admission | BLOCKED_OWNER | #35's exact-head central runs are still queued before useful execution; `Detect changed scope` has no runner assignment or steps. Queueing blocks this validation lane only and is not a reason to stop Source Observation or other repository-owned work. | +| Standards / research | REPAIRED_PENDING_CI | Doctoring remains bound to authoritative standards/primary research and exact implementation contracts; hosted exact-head evidence remains independently required after head changes. | | Release | NOT_STARTED | No immutable ConceptWeave release exists. Version/CHANGELOG/tag/package/semantic_release/SBOM/provenance/reproducibility/rollback are required on the exact protected release head. | +## Dependency Review incident correction + +The prior Foundation predecessor exposed a real hosted failure: the authenticated Dependency Review compare preflight returned HTTP 403 for a public, non-fork ConceptWeave exact range. The initially proposed central repair retried the same token-bound request while retaining fail-closed behavior. + +Fresh owner RCA invalidated that causal hypothesis. The same authenticated exact-range request returned HTTP 200 for a repository whose dependency graph was initialized and HTTP 403 for affected repositories whose graph was not initialized. Enabling Dependabot vulnerability alerts initialized the dependency graph in ConceptWeave and pingora-gateway, after which the exact compare endpoint returned HTTP 200. Therefore `.github#1873` was correctly closed without merge: retries would extend queue occupancy but would not establish repository capability. + +Acceptance remains stricter than the RCA. HTTP 200 availability alone is not GREEN. A fresh exact ConceptWeave consumer run must reach and complete the pinned Dependency Review action; 403, transport failure, skipped substitution or a sibling scanner cannot satisfy the hard gate. + ## Central control-plane evidence -Protected central source is `.github/main@c31d2e5471fc5daf9d72ff67cde6a8874b736deb` at this snapshot, after merged #1852 aligned current-main workflow contracts. This is evidence only, not a ConceptWeave dependency. - -- `.github#1821` queue-ownership repair remains integrated: organization sweep no longer owns repository-wide queued/in-progress Actions inventory or broad cancellation; native per-PR concurrency and repository-local exact-head coalescing own supersession. -- Later consolidation removed merge-scheduler required-check fanout, centralized CodeQL PR ownership and consolidated empty-PR/quality lanes. -- Fresh `.github` queued inventory is `245`. This is far below the ~1,900 peak but above some recent lower observations, so it is neither terminal recovery nor consumer acceptance. - -## P0 product gaps after the current TDD lanes - -1. **Concrete Source Observation adapter** — maintained Rust PostgreSQL driver behind `conceptweave-source-port`; adapter-local credential resolution; explicit read-only mode; statement timeout, cancellation, row/byte/concurrency budgets; complete immutable snapshot or fail closed; deterministic replay against a frozen anonymized GRC-shaped fixture. -2. **Ontology discovery** — deterministic term/concept/taxonomy/non-taxonomic-relation candidate generation with exact source receipts and abstention for unsupported semantics. -3. **Semantic-layer discovery** — dimensions, measures, grain, units, relationships and physical mappings with deterministic calculation contracts; do not infer business authority from relational structure alone. -4. **LLM Proposal** — every production model call through released `contextual-orchestrator`; outputs remain proposed/inferred and preserve source/model/prompt/provenance evidence. -5. **Alignment / matching** — retrieval/pruning/structural evidence first, bounded optional LLM assistance, OAEI-style evaluation, deterministic reproducibility and steward-visible decisions. -6. **Validation engine** — RDF/OWL/SKOS/SHACL and semantic-layer validation, consistency/conflict/duplicate detection, bounded reasoning, explicit unsupported-feature failure. -7. **Governance persistence** — PostgreSQL 3NF candidates/evidence/validation/review/release/supersession receipts, transactional outbox and temporal history only where domain semantics require it. -8. **Review workflow** — Keyverse identity context, tenant/role/purpose authorization, steward decisions, maker-checker where required, stale-decision protection and immutable publication receipt. -9. **Publication adapters** — versioned OWL/RDFS/SKOS/SHACL/JSON-LD plus explicitly version-bound Apache Ossie export; draft/incubating formats cannot be presented as final standards. -10. **Client completion** — language-neutral release/supersession contract, provenance/signature verification, relation/mapping/dimension/measure resolution, compatibility/deprecation, match/explain/query-plan contracts while downstream products retain physical authorization/execution. -11. **CWL integration** — only released/versioned semantic_release/contract/ACL seams to `semantic-data-portal`, `context-graph-contracts`, GRC, EA and other consumers; no source copying, cross-service SQL or mutable supplier heads. -12. **Evaluation / multilingual** — reviewed golden fixtures, ontology-learning/matching metrics, source-evidence binding, abstention, reproducibility, KO/EN/JA/ZH/VI/ES/DE/FR labels, CJK/font/text-expansion checks where UI or published labels are material. -13. **Observability / recovery / release** — structured telemetry, security evidence, backup/restore, package/SBOM/provenance/signing, reproducible build and rollback proof before immutable release. +Protected central source is `.github/main@b5efbc2762e472e4a380b0503b1f050f76fbb008` at this snapshot. This is evidence only, not a mutable ConceptWeave dependency. + +- The current central source includes queue/admission and changed-scope/review-runtime repairs already integrated through ordinary protected history. +- `.github#1873@41935494aa234eb458f1cc08f006daaa278b9760` is closed/unmerged because repository dependency-graph initialization, not its retry/sleep source delta, was the verified root cause of the observed public-repository 403. +- #35 remains an exact consumer canary for current runner admission and Dependency Review behavior. Its central workflows are queued, so no protected recovery or dependency-review success is inferred from repository settings alone. + +## P0 product gaps + +1. **Concrete Source Observation adapter** — maintained Rust PostgreSQL driver behind `conceptweave-source-port`; adapter-local registry/credential resolution; explicit read-only session/transaction; exact schema allowlist; total operation and statement deadlines; cancellation plus row/byte/concurrency budgets; complete immutable snapshot or fail closed; source-disappearance handling; deterministic replay against a frozen anonymized GRC-shaped fixture. +2. **Observed PostgreSQL surface completion** — domains/enums/indexes/comments, quoted identifiers and cross-schema collisions as generic observed evidence without importing source-system business truth. +3. **Ontology discovery** — deterministic term/concept/taxonomy/non-taxonomic-relation candidate generation with exact source receipts and abstention for unsupported semantics. +4. **Semantic-layer discovery** — dimensions, measures, grain, units, relationships and physical mappings with deterministic calculation contracts; do not infer business authority from relational structure alone. +5. **LLM Proposal** — every production model call through a released `contextual-orchestrator`; outputs remain proposed/inferred and preserve source/model/prompt/provenance evidence. +6. **Alignment / matching** — retrieval/pruning/structural evidence first, bounded optional LLM assistance, OAEI-style evaluation, deterministic reproducibility and steward-visible decisions. +7. **Validation engine** — RDF/OWL/SKOS/SHACL and semantic-layer validation, consistency/conflict/duplicate detection, bounded reasoning and explicit unsupported-feature failure. +8. **Governance persistence** — PostgreSQL 3NF candidates/evidence/validation/review/release/supersession receipts, transactional outbox and temporal history only where domain semantics require it. +9. **Review workflow** — Keyverse identity context, tenant/role/purpose authorization, steward decisions, maker-checker where required, stale-decision protection and immutable publication receipt. +10. **Publication adapters** — versioned OWL/RDFS/SKOS/SHACL/JSON-LD plus explicitly version-bound Apache Ossie export; draft/incubating formats cannot be presented as final standards. +11. **Client completion** — language-neutral release/supersession contract, provenance/signature verification, relation/mapping/dimension/measure resolution, compatibility/deprecation, match/explain/query-plan contracts while downstream products retain physical authorization/execution. +12. **CWL integration** — only released/versioned `semantic_release`/contract/ACL seams to `semantic-data-portal`, `context-graph-contracts`, GRC, EA and other consumers; no source copying, cross-service SQL or mutable supplier heads. +13. **Evaluation / multilingual** — reviewed golden fixtures, ontology-learning/matching metrics, source-evidence binding, abstention, reproducibility, KO/EN/JA/ZH/VI/ES/DE/FR labels, CJK/font/text-expansion checks where UI or published labels are material. +14. **Observability / recovery / release** — structured telemetry, security evidence, backup/restore, package/SBOM/provenance/signing, reproducible build and rollback proof before immutable release. ## DDD fitness constraints