Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
05d0fbc
test: require commercially compatible sync PostgreSQL boundary
seonghobae Sep 1, 2026
76153aa
feat: add pg8000 synchronous PostgreSQL adapter
seonghobae Sep 1, 2026
f359de1
build: replace psycopg2 with pg8000
seonghobae Sep 1, 2026
33fc33d
test: collect pg8000-backed optional PostgreSQL suites safely
seonghobae Sep 1, 2026
c1d466f
test: run PROV-O schema contract through pg8000 adapter
seonghobae Sep 1, 2026
3d4881e
test: migrate analysis authorization DB fixture to pg8000 adapter
seonghobae Sep 1, 2026
71ca95d
test: migrate reconstruction fixture to pg8000 adapter
seonghobae Sep 1, 2026
062cf60
test: preserve DSN options in synthetic cleanup fixture
seonghobae Sep 1, 2026
5e64e45
test: cover pg8000 optional-extra collection boundary
seonghobae Sep 1, 2026
a6bca54
test: migrate Voice history DB contract to pg8000 adapter
seonghobae Sep 1, 2026
c612f46
test: require SQLSTATE parity for migrated schema fixtures
seonghobae Sep 1, 2026
a23de60
fix: preserve SQLSTATE error semantics across pg8000 migration
seonghobae Sep 1, 2026
da64b7c
test(postgres): reject duplicate DSN options
seonghobae Sep 1, 2026
2b7dde4
fix(postgres): reject duplicate DSN options
seonghobae Sep 1, 2026
74433b6
test(postgres): reject raw SQL interpolation
seonghobae Sep 1, 2026
5c2dbf8
fix(postgres): restrict SQL interpolation
seonghobae Sep 1, 2026
0ebbd66
test(lock): require PostgreSQL driver lock convergence
seonghobae Sep 1, 2026
a30cedb
test(postgres): preserve connection failure contract
seonghobae Sep 1, 2026
1763fbe
fix(postgres): classify startup refusals as operational
seonghobae Sep 1, 2026
c4f8830
test(postgres): preserve libpq default user semantics
seonghobae Sep 1, 2026
6e9fffd
fix(postgres): retain default OS user for admin DSNs
seonghobae Sep 1, 2026
70554ce
test(collection): detect package-imported postgres adapter
seonghobae Sep 1, 2026
81e1b01
fix(collection): traverse imported local submodules
seonghobae Sep 1, 2026
ea3de30
test(postgres): reject silently discarded DSN fragments
seonghobae Sep 1, 2026
e70d7ad
fix(postgres): reject unsupported DSN fragments
seonghobae Sep 1, 2026
ba197bf
test(ci): preserve resolver candidate for stale uv lock
seonghobae Sep 1, 2026
86c1cb0
ci: preserve resolver output for stale dependency locks
seonghobae Sep 1, 2026
9c1bdee
test(postgres): reject non-finite connection deadlines
seonghobae Sep 1, 2026
fa7feeb
fix(postgres): bound connection deadlines to finite values
seonghobae Sep 1, 2026
debbcac
test(postgres): reject boolean connection deadlines
seonghobae Sep 1, 2026
99bdd0e
fix(postgres): require numeric connection deadlines
seonghobae Sep 1, 2026
1d23997
test(ci): reject invalid resolver-candidate publication
seonghobae Sep 1, 2026
7777035
ci: fail closed when lock resolution fails
seonghobae Sep 1, 2026
b6709c9
test(postgres): reject explicit zero DSN port
seonghobae Sep 1, 2026
c894ac2
fix(postgres): fail closed on invalid DSN ports
seonghobae Sep 1, 2026
f0d7c22
test(postgres): reject hostless DSN transport switch
seonghobae Sep 1, 2026
874c39b
fix(postgres): preserve hostless DSN transport semantics
seonghobae Sep 1, 2026
e1d1200
chore(stack): converge synchronous PostgreSQL migration with protecte…
seonghobae Sep 3, 2026
126887c
test(ci): preserve stale-lock fail-closed workflow contract
seonghobae Sep 3, 2026
3d0728c
fix(ci): preserve stale-lock evidence after main convergence
seonghobae Sep 3, 2026
b0218b1
test(ci): keep the existing stale-lock regression canonical
seonghobae Sep 3, 2026
ebef4f0
test(postgres): reject unrepresentable explicit connect timeout
seonghobae Sep 3, 2026
e6b95f0
fix(postgres): fail closed on unrepresentable connect timeout
seonghobae Sep 3, 2026
cb70ce3
chore(stack): converge PostgreSQL migration with protected main
seonghobae Sep 3, 2026
b389c48
docs(changelog): record synchronous PostgreSQL boundary hardening
seonghobae Sep 3, 2026
097b2d7
fix(postgres): complete synchronous driver migration
Sep 4, 2026
8d4d8cc
test(postgres): clarify rollback control flow
Sep 4, 2026
b84f805
test(postgres): pin review regression contracts
seonghobae Sep 4, 2026
c671fdb
fix(ci): preserve resolver lock evidence across reruns
seonghobae Sep 4, 2026
823a443
fix(postgres): translate transaction boundary errors
seonghobae Sep 4, 2026
ffde764
fix(postgres): document public sync adapter surface
seonghobae Sep 4, 2026
1167fea
test(postgres): pin libpq prefer transport ordering
seonghobae Sep 4, 2026
ff4b02c
fix(postgres): preserve libpq prefer SSL ordering
seonghobae Sep 4, 2026
8c0990a
test(postgres): pin libpq require TLS semantics
seonghobae Sep 4, 2026
d1650eb
fix(postgres): preserve libpq require TLS semantics
seonghobae Sep 4, 2026
1da5f83
chore(ci): adopt protected concurrency contract
seonghobae Sep 4, 2026
db6a0e9
chore(ci): adopt protected ontology concurrency
seonghobae Sep 4, 2026
59fffb9
chore(ci): adopt protected PROV-O concurrency
seonghobae Sep 4, 2026
cb430e7
test(ci): adopt protected workflow concurrency contract
seonghobae Sep 4, 2026
950e46f
chore(main): non-force converge protected #930
seonghobae Sep 4, 2026
007261b
chore(main): adapt #931 draft admission to resolver evidence
seonghobae Sep 4, 2026
7651ca9
chore(main): non-force converge protected #931
seonghobae Sep 4, 2026
307c29e
fix(postgres): verify server identity by default
Sep 4, 2026
76c0e67
docs(postgres): accept default TLS decision
Sep 4, 2026
1736565
test(adr): reserve non-colliding TLS decision number
seonghobae Sep 4, 2026
5712294
docs(adr): move PostgreSQL TLS proposal to 0366
seonghobae Sep 4, 2026
034dfc4
docs(adr): remove colliding TLS ADR 0363
seonghobae Sep 4, 2026
5d40eed
docs(postgres): remove ADR trailing whitespace
Sep 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,28 @@ jobs:
version: "0.11.28"
enable-cache: false

- name: Verify committed universal lock
id: lock
shell: bash
run: |
set -euo pipefail
if uv lock --check; then
echo "stale=false" >> "$GITHUB_OUTPUT"
exit 0
fi
uv lock
echo "stale=true" >> "$GITHUB_OUTPUT"
exit 1

- name: Preserve resolver-generated lock candidate
if: failure() && steps.lock.outputs.stale == 'true'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: uv-lock-candidate-${{ github.sha }}-${{ github.run_attempt }}
path: uv.lock
if-no-files-found: error
retention-days: 7

- name: Select pinned Rust toolchain
run: |
rustup toolchain install 1.97.1 --profile minimal
Expand Down
13 changes: 13 additions & 0 deletions CHANGELOG.d/2.28.0-postgres-sync-driver.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
## 2.28.0 — Commercial-safe synchronous PostgreSQL boundary

- Replace the synchronous seed/admin/schema `psycopg2-binary` dependency with
a pinned `pg8000` adapter while keeping runtime persistence on `asyncpg`.
- Preserve generated-identifier quoting, SQLSTATE-specific schema/security
assertions, libpq-style DSN semantics that are representable by the adapter,
and fail closed on unsupported or ambiguous connection options.
- Reject boolean, non-finite, non-positive, and float-unrepresentable explicit
connection timeouts through the adapter validation contract instead of
leaking arithmetic conversion errors.
- Keep dependency-lock verification fail closed: a stale committed `uv.lock`
fails before frozen install/tests while the pinned resolver output is retained
as short-lived evidence for an exact, non-hand-edited lock update.
1 change: 1 addition & 0 deletions backend/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@
)
from backend.app.occupational_construct_ingestion import (
load_occupational_construct_assertions,
load_occupational_construct_evidence_status,
)
from backend.app.occupational_construct_search import (
OccupationalConstructSearchError,
Expand Down
44 changes: 43 additions & 1 deletion backend/tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@

import asyncpg
import jwt
import psycopg2
import pytest
import redis

from lineageweave import postgres_sync as psycopg2
from lineageweave.http_client import HttpClientError, get_json, post_form
from lineageweave.knowledge_graph import knowledge_graph_edges_for_post
from lineageweave.post_summary import POST_SUMMARY_CONTRACT_VERSION
Expand Down Expand Up @@ -217,6 +217,40 @@
/ "migrations"
/ "0218_global_ask_public_verification.sql"
)
_TEPP_RECEIPT_MIGRATION = (
Path(__file__).resolve().parents[2]
/ "migrations"
/ "0217_analysis_run_tepp_receipt.sql"
)
_PROV_O_MIGRATION = (
Path(__file__).resolve().parents[2] / "migrations" / "0017_prov_o_standard_relations.sql"
)
_VOICE_TAXONOMY_MIGRATION = (
Path(__file__).resolve().parents[2] / "migrations" / "0235_voice_of_x_post_taxonomy.sql"
)
_ONTOLOGY_TRUTH_STATUS_MIGRATION = (
Path(__file__).resolve().parents[2] / "migrations" / "0175_ontology_truth_status.sql"
)
_CONVERSATION_EVIDENCE_MIGRATION = (
Path(__file__).resolve().parents[2]
/ "migrations"
/ "0233_source_conversation_turn_evidence.sql"
)
_SOURCE_POST_VOICE_MIGRATION = (
Path(__file__).resolve().parents[2]
/ "migrations"
/ "0237_source_post_voice_combination.sql"
)
_OCCUPATIONAL_CONSTRUCT_ASSERTION_MIGRATION = (
Path(__file__).resolve().parents[2]
/ "migrations"
/ "0238_occupational_construct_assertion.sql"
)
_OCCUPATIONAL_CONSTRUCT_EXTRACTION_MIGRATION = (
Path(__file__).resolve().parents[2]
/ "migrations"
/ "0240_occupational_construct_extraction_run.sql"
)
_GLOBAL_ASK_KNOWLEDGE_CUTOFF_MIGRATION = (
Path(__file__).resolve().parents[2]
/ "migrations"
Expand Down Expand Up @@ -332,6 +366,7 @@ def seeded_db(demo_analyst_token):
try:
with conn.cursor() as cur:
cur.execute(_MIGRATION_PATH.read_text())
cur.execute(_PROV_O_MIGRATION.read_text())
cur.execute(_REGISTRY_MIGRATION.read_text())
cur.execute(_RETENTION_MIGRATION.read_text())
cur.execute(_RECONSTRUCTION_MIGRATION.read_text())
Expand Down Expand Up @@ -415,16 +450,23 @@ def seeded_db(demo_analyst_token):
)
conn.autocommit = False
cur.execute(_GLOBAL_ASK_KNOWLEDGE_CUTOFF_MIGRATION.read_text())
cur.execute(_TEPP_RECEIPT_MIGRATION.read_text())
cur.execute(_GLOBAL_ASK_PUBLIC_VERIFICATION_MIGRATION.read_text())
cur.execute(_EVENT_OCCURRED_AT_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_AXIS_MIGRATION.read_text())
cur.execute(_CHANNEL_EVIDENCE_MIGRATION.read_text())
cur.execute(_ONTOLOGY_TRUTH_STATUS_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_UNEXPLAINED_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_CROSS_SHARE_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_RECONSTRUCTION_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_UNEXPLAINED_SHARE_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_EXPLAINED_SHARE_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_COORDINATES_MIGRATION.read_text())
cur.execute(_CONVERSATION_EVIDENCE_MIGRATION.read_text())
cur.execute(_VOICE_TAXONOMY_MIGRATION.read_text())
cur.execute(_SOURCE_POST_VOICE_MIGRATION.read_text())
cur.execute(_OCCUPATIONAL_CONSTRUCT_ASSERTION_MIGRATION.read_text())
cur.execute(_OCCUPATIONAL_CONSTRUCT_EXTRACTION_MIGRATION.read_text())
cur.execute(
"insert into common_lookup_value (lookup_category, lookup_code, lookup_label) values "
"('corporate_entity_level', 'group', 'Group'), "
Expand Down
48 changes: 48 additions & 0 deletions docs/adr/0366-synchronous-postgresql-default-tls.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# ADR 0366 — Verify synchronous PostgreSQL server identity by default

**Decision status:** Proposed
**Date:** 2026-09-05

## Context

LineageWeave's synchronous administrative and test adapter accepts PostgreSQL
URIs and translates their transport policy to pg8000. PostgreSQL documents
`sslmode=prefer` as a backwards-compatibility default that may reconnect in
plaintext and does not authenticate the server. That behavior is unsuitable as
an implicit product default for credentials in transit. Explicit DSN policy
must remain representable for controlled compatibility paths.

## Decision

A network DSN that omits `sslmode` uses `verify-full`: TLS is required, the
certificate chain is validated, and the requested hostname must match. It never
retries in plaintext. An explicit `sslmode=prefer` remains TLS-first with
plaintext fallback only after PostgreSQL's exact SSL-refusal response.
`disable`, `require`, `verify-ca`, and `verify-full` retain their documented
meanings. Unix-socket inference remains unavailable because the adapter cannot
recover libpq's socket selection from a hostless URI.

## Consequences

- Accidental network DSNs fail closed against untrusted or mismatched servers.
- Existing private-CA deployments must install their CA or state an explicit
reviewed compatibility mode; a silent downgrade is no longer possible.
- Explicit `prefer` remains weaker and must be chosen in the DSN rather than
inherited from an omission.

## Alternatives considered

### Keep implicit `prefer`

Rejected because an SSL-refusing endpoint can cause credential-bearing tooling
to reconnect without encryption.

### Default to `require`

Rejected because encryption without certificate and hostname verification does
not authenticate the server.

## References

- PostgreSQL 18, *SSL Support* and *Database Connection Control Functions*.
- pg8000 1.31.5, `ssl_context` connection contract.
19 changes: 16 additions & 3 deletions lineageweave/optional_extra_collection.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

OpenCode coverage-evidence runs in a networkless sandbox that supplies
pytest and coverage but not LineageWeave's optional backend extras
(``asyncpg``, ``psycopg2``, ``redis``, ``fast_mlsirm``, ``numpy``). Hosted CI
(``asyncpg``, ``pg8000``, ``redis``, ``fast_mlsirm``, ``numpy``). Hosted CI
installs those extras and collects every suite. This helper keeps
collection from failing with ``ModuleNotFoundError`` when extras are
absent, without skipping anything when they are present.
Expand All @@ -17,7 +17,7 @@

OPTIONAL_EXTRA_MODULES: tuple[str, ...] = (
"asyncpg",
"psycopg2",
"pg8000",
"redis",
"fast_mlsirm",
"numpy",
Expand All @@ -28,7 +28,14 @@


def _imported_module_names(source: str) -> frozenset[str]:
"""Return exact top-level module paths from syntactically valid imports."""
"""Return exact top-level module paths from syntactically valid imports.

For ``from package import submodule`` both the package and candidate
submodule path are retained. The local-source resolver later decides
whether that candidate is a real repository module, which lets collection
tracing follow imports such as ``from lineageweave import postgres_sync``
without mistaking ordinary imported attributes for modules.
"""
try:
tree = ast.parse(source)
except (SyntaxError, ValueError):
Expand All @@ -39,6 +46,12 @@ def _imported_module_names(source: str) -> frozenset[str]:
imported.update(alias.name for alias in node.names)
elif isinstance(node, ast.ImportFrom) and node.module:
imported.add(node.module)
if node.level == 0:
imported.update(
f"{node.module}.{alias.name}"
for alias in node.names
if alias.name != "*"
)
return frozenset(imported)


Expand Down
Loading
Loading