Skip to content

Commit 60d3496

Browse files
authored
test: 整合 PostgreSQL 14/16 與 17 真實版本回歸(07 證據) (#49)
* test: 納入 PostgreSQL 雙向跨版本正式來源回歸 把獨立驗收的 14/16 與 17 往返 fixture 改為直接引用 Core,避免凍結副本與正式程式脫節。每輪保留版本、映像 digest、來源 hash 與自有容器清理結果,文件明列 1007 列及空表的實測範圍,不擴大為全部版本或 schema 支援;發版仍暫停。 * ci: 檢查 PostgreSQL 版本回歸工程能持續建置 把新 harness 與腳本納入跨平台 workflow 路徑,四平台編譯直接引用 Core 的 fixture。真實跨版本執行另由隔離腳本記錄,不把編譯成功當成版本測試通過。
1 parent 4368883 commit 60d3496

5 files changed

Lines changed: 223 additions & 2 deletions

File tree

‎.github/workflows/cross-platform.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ on:
99
- "mySQLPunk.Desktop/**"
1010
- "mySQLPunk.CrossPlatform.SmokeTests/**"
1111
- "tests/Run-CrossPlatformLiveTests.sh"
12+
- "tests/Run-PostgreSqlVersionMigration.py"
13+
- "tests/PostgreSqlVersionMigrationHarness/**"
1214
- "tests/Run-DesktopProfileOperationTests.sh"
1315
- "tests/DesktopProfileOperationHarness/**"
1416
- "tests/Test-LinuxCrossPlatformPackage.sh"
@@ -24,6 +26,8 @@ on:
2426
- "mySQLPunk.Desktop/**"
2527
- "mySQLPunk.CrossPlatform.SmokeTests/**"
2628
- "tests/Run-CrossPlatformLiveTests.sh"
29+
- "tests/Run-PostgreSqlVersionMigration.py"
30+
- "tests/PostgreSqlVersionMigrationHarness/**"
2731
- "tests/Run-DesktopProfileOperationTests.sh"
2832
- "tests/DesktopProfileOperationHarness/**"
2933
- "tests/Test-LinuxCrossPlatformPackage.sh"
@@ -82,6 +86,9 @@ jobs:
8286
- name: Run cross-platform smoke tests
8387
run: dotnet run --project mySQLPunk.CrossPlatform.SmokeTests/mySQLPunk.CrossPlatform.SmokeTests.csproj -c Release --no-build
8488

89+
- name: Build formal PostgreSQL version fixture
90+
run: dotnet build tests/PostgreSqlVersionMigrationHarness/Review.csproj -c Release
91+
8592
- name: Run isolated Avalonia profile operation handlers
8693
if: matrix.name == 'Linux x64'
8794
run: ./tests/Run-DesktopProfileOperationTests.sh

‎docs/LOGICAL_MIGRATION_MATRIX.md‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,26 @@
99
| SQL Server → SQL Server | 2016/2017/2019/2022/2025 任意雙向及同版本,共 25 組 | Windows 既有專用工具;dbo heap、既有型別白名單、同定序、無 advanced features | 25 組預檢及交易 mock;另有正式來源 2019 ↔ 2022 隔離回歸,1007 列/7 欄+空表,不是 25 組真引擎驗證 |
1010
| SQLite → SQLite | 3.37–3.53 | STRICT、INTEGER/BLOB/TEXT、NULL/NOT NULL、非自增 PK、普通索引;詳細子集見下文 | 本機實際 SQLite 3.53.3;歷史 minor 版本未逐一跑 |
1111
| SQLite → PostgreSQL | SQLite 3.37–3.53 → PostgreSQL 14–17 | INTEGER → bigint、BLOB → bytea、TEXT BINARY → text C;非自增 PK、普通索引 | 隔離 CI PostgreSQL 16 fixture 加入文字/鍵/索引雙向 round-trip;只以這次新 head 結果為證據 |
12-
| PostgreSQL → SQLite | PostgreSQL 14–17 → SQLite 3.37–3.53 | 上述型別反向映射;PK → WITHOUT ROWID、普通索引 | 同上;14/15/17 未實機跑 |
13-
| PostgreSQL → PostgreSQL | 14–17 雙向及同版本 | 同一 bigint/bytea/text C、PK/索引子集與全部預檢 | 邏輯共用路徑;沒有完整跨版本真機矩陣 |
12+
| PostgreSQL → SQLite | PostgreSQL 14–17 → SQLite 3.37–3.53 | 上述型別反向映射;PK → WITHOUT ROWID、普通索引 | PG17 實際 roundtrip 與既有 PG16 fixture;PG15 未實機跑 |
13+
| PostgreSQL → PostgreSQL | 14–17 雙向及同版本 | 同一 bigint/bytea/text C、PK/索引子集與全部預檢 | 真實 17.10 ↔ 14.24、17.10 ↔ 16.15;不是全部版本配對驗證 |
14+
15+
## 可重跑的 PostgreSQL 跨版本證據
16+
17+
`python3 tests/Run-PostgreSqlVersionMigration.py 14 [新證據目錄]` 與 `python3 tests/Run-PostgreSqlVersionMigration.py 16 [新證據目錄]` 直接 ProjectReference 正式 Core,不使用凍結副本。需先快取官方 `postgres:14-alpine`/`postgres:16-alpine` 與 `postgres:17-alpine`;腳本只 image inspect,不拉取映像或調整網路/權限。每次建立唯一 ownership label、loopback 隨機 port 與 tmpfs 容器,最後核對 label 才清理自己的 fixture;密碼隨機產生且不輸出。
18+
19+
每輪為 SQLite seed → PG17 → 舊版 PG → 空 PG17 → SQLite,records 1007 列+empty_table 空表,複合 PK、具名普通索引、UTF8/emoji、NULL/空 binary/空 text、Int64 極值。每段使用 production 預檢、交易、schema 與 SHA256 多重集合驗證,最後另測重複 PK 被拒絕及索引存在。這不是 procedures、FK、identity、任意型別、SSH 或桌面 UI 的跨版本證明;1005 列 transaction fault 仍只屬既有 SQLite 回歸,provider 全套 56/56 仍是另一組測試。
20+
21+
2026-10-14 獨立證據先以 fa734e7 跑 17 ↔ 14/16,再以 06 工作樹跑 17 ↔ 14。06 最終 95c0be2(合入 4368883)三個 production 檔案 SHA256 均與獨立工作樹一致:Models/LogicalMigration.cs `a912ddcd778bac06c4980f3f3403d9b0d4563b81414c170894b79619bf79912e`、Services/LogicalMigrationService.cs `721ec60b1be6c2ada40114e59c323513c44aa1124e52ffa9411cb6cd23095fd4`、Services/LogicalMigrationKeys.cs `69fc1e7ea82ab7f27c2ce22d4d14613294f981f73bdb0b4f99e04893ef793cef`。新 harness 再以相同正式來源跑兩組成功;並非重跑舊 SHA CI。
22+
23+
當次實際官方映像 RepoDigest(後續 tag 可能改變,重跑以 evidence.json 的實際版本及 digest 為準):
24+
25+
| 實際版本 | 官方 RepoDigest |
26+
| --- | --- |
27+
| 14.24 | postgres@sha256:4ea9e5ed06591da7ea23eb65465e8d3187fe79f4d5ec3ae976d29a33b013e77a |
28+
| 16.15 | postgres@sha256:721873c34ceb9f8d8fc265984940dc982404c105f19ad51be9fdc5970a6080ea |
29+
| 17.10 | postgres@sha256:742f40ea20b9ff2ff31db5458d127452988a2164df9e17441e191f3b72252193 |
30+
31+
`migration.log`、`evidence.json` 保留實際版本、映像 ID/digest、正式來源 hash 與清理结果;不存密碼或資料值。原獨立日誌仍在 audit-evidence/postgresql-14-17-real-versions.log、postgresql-16-17-real-versions.log、postgresql-14-17-primary-name-fix.log,沒有覆寫。
1432

1533
## 第二階段:實用文字與鍵索引子集
1634

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
using Microsoft.Data.Sqlite;
2+
using MySqlPunk.Core.Models;
3+
using MySqlPunk.Core.Services;
4+
using Npgsql;
5+
6+
var oldMajor = int.Parse(Environment.GetEnvironmentVariable("MYSQLPUNK_OLD_PG_MAJOR") ?? "14");
7+
if (oldMajor is not (14 or 16)) throw new ArgumentException("Only the 14/17 and 16/17 fixtures are defined.");
8+
var ports = new[] { int.Parse(args[0]), int.Parse(args[1]) };
9+
var password = Environment.GetEnvironmentVariable("MYSQLPUNK_PG_TEST_PASSWORD")
10+
?? throw new InvalidOperationException("Missing isolated fixture password.");
11+
string Connection(int port, string database) => new NpgsqlConnectionStringBuilder
12+
{
13+
Host = "127.0.0.1", Port = port, Username = "postgres", Password = password,
14+
Database = database, SslMode = SslMode.Disable, Pooling = false, Timeout = 3
15+
}.ConnectionString;
16+
foreach (var port in ports)
17+
{
18+
await using var connection = new NpgsqlConnection(Connection(port, "postgres"));
19+
await connection.OpenAsync();
20+
Console.WriteLine("Actual PostgreSQL version=" + connection.PostgreSqlVersion);
21+
var expectedMajor = port == ports[0] ? oldMajor : 17;
22+
if (connection.PostgreSqlVersion.Major != expectedMajor) throw new Exception("Unexpected engine major.");
23+
}
24+
if (args.Length > 2) return;
25+
var root = Path.Combine(Path.GetTempPath(), "mysqlpunk-version-fixture-" + Guid.NewGuid().ToString("N"));
26+
Directory.CreateDirectory(root);
27+
try
28+
{
29+
foreach (var port in ports)
30+
{
31+
await using var connection = new NpgsqlConnection(Connection(port, "postgres"));
32+
await connection.OpenAsync();
33+
foreach (var database in new[] { "migration_source", "migration_back" })
34+
{
35+
await using var command = connection.CreateCommand();
36+
command.CommandText = "CREATE DATABASE " + database;
37+
await command.ExecuteNonQueryAsync();
38+
}
39+
}
40+
ConnectionProfile Pg(int index, string database) => new()
41+
{
42+
Name = "isolated PG", Provider = DatabaseProviderKind.PostgreSql, Host = "127.0.0.1",
43+
Port = ports[index], Username = "postgres", Password = password,
44+
Database = database, TlsMode = ConnectionTlsMode.Disabled
45+
};
46+
var source = new ConnectionProfile
47+
{
48+
Name = "isolated SQLite", Provider = DatabaseProviderKind.Sqlite,
49+
Database = Path.Combine(root, "source.db")
50+
};
51+
using (var connection = new SqliteConnection("Data Source=" + source.Database))
52+
{
53+
connection.Open();
54+
using var command = connection.CreateCommand();
55+
command.CommandText = """
56+
CREATE TABLE records (id INTEGER NOT NULL, label TEXT NOT NULL, note TEXT, payload BLOB,
57+
PRIMARY KEY(id,label)) STRICT, WITHOUT ROWID;
58+
CREATE INDEX records_label ON records(label);
59+
INSERT INTO records VALUES
60+
(1,'abc','繁體中文 😺',X'00FF'),(1,'ABC','',NULL),
61+
(2,'a','前後空白 ',X''),(2,'a ',NULL,X'010203'),(3,'é','é',NULL);
62+
CREATE TABLE empty_table (number INTEGER,payload BLOB,note TEXT) STRICT;
63+
WITH RECURSIVE numbers(number) AS (VALUES(1) UNION ALL SELECT number+1 FROM numbers WHERE number<1000)
64+
INSERT INTO records SELECT 100+number,'row_'||number,
65+
CASE WHEN number%3=0 THEN NULL ELSE '測試😀 ' END,
66+
CASE WHEN number%2=0 THEN NULL ELSE X'00FF' END FROM numbers;
67+
INSERT INTO records VALUES(-9223372036854775808,'min',NULL,X''),(9223372036854775807,'max','',NULL);
68+
""";
69+
command.ExecuteNonQuery();
70+
}
71+
async Task Move(ConnectionProfile from, ConnectionProfile to, int major, string label)
72+
{
73+
var preview = await LogicalMigrationService.PreviewAsync(from, to, major);
74+
Console.WriteLine(label + " preview=" + preview.CanStart + " " + string.Join(";", preview.Blockers));
75+
if (!preview.CanStart) throw new Exception(label + " preflight rejected");
76+
var result = await LogicalMigrationService.RunAsync(from, to, major, preview, to.Database);
77+
Console.WriteLine(label + " " + result.Outcome + " rows=" + preview.Tables.Sum(table => table.Rows));
78+
if (result.Outcome != LogicalMigrationOutcome.Verified) throw new Exception(label + " " + result.Report);
79+
}
80+
var old = Pg(0, "migration_source");
81+
var newer = Pg(1, "migration_source");
82+
var back = Pg(1, "migration_back");
83+
await Move(source, newer, 17, "SQLite -> PG17 seed");
84+
await Move(newer, old, oldMajor, "PG17 -> PG" + oldMajor + " downgrade");
85+
await Move(old, back, 17, "PG" + oldMajor + " -> PG17 upgrade");
86+
var final = source.Clone(); final.Database = Path.Combine(root, "back.db");
87+
using (var connection = new SqliteConnection("Data Source=" + final.Database)) connection.Open();
88+
await Move(back, final, 3, "PG17 -> SQLite roundtrip");
89+
using (var connection = new SqliteConnection("Data Source=" + final.Database))
90+
{
91+
connection.Open();
92+
using var command = connection.CreateCommand();
93+
command.CommandText = "SELECT count(*) FROM records";
94+
if (Convert.ToInt64(command.ExecuteScalar()) != 1007) throw new Exception("Roundtrip count.");
95+
command.CommandText = "SELECT count(*) FROM sqlite_schema WHERE type='index' AND name='records_label'";
96+
if (Convert.ToInt64(command.ExecuteScalar()) != 1) throw new Exception("Named index lost.");
97+
command.CommandText = "INSERT INTO records VALUES(1,'abc',NULL,NULL)";
98+
try { command.ExecuteNonQuery(); throw new Exception("Composite PK lost."); }
99+
catch (SqliteException exception) when (exception.SqliteErrorCode == 19) { }
100+
}
101+
Console.WriteLine("PASS actual PG17 -> PG" + oldMajor + " -> PG17 plus SQLite roundtrip; 1007 rows + empty table; composite PK, named index, UTF8/emoji, NULL/empty binary/text, signed Int64 extrema; production transaction and SHA256 verification");
102+
}
103+
finally
104+
{
105+
SqliteConnection.ClearAllPools();
106+
Directory.Delete(root, true);
107+
}
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
<PropertyGroup>
3+
<OutputType>Exe</OutputType>
4+
<TargetFramework>net8.0</TargetFramework>
5+
<ImplicitUsings>enable</ImplicitUsings>
6+
<Nullable>enable</Nullable>
7+
</PropertyGroup>
8+
<ItemGroup>
9+
<ProjectReference Include="../../mySQLPunk.Core/mySQLPunk.Core.csproj" />
10+
</ItemGroup>
11+
</Project>
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
import hashlib
2+
import json
3+
import os
4+
from pathlib import Path
5+
import secrets
6+
import subprocess
7+
import sys
8+
import time
9+
import uuid
10+
11+
root = Path(__file__).resolve().parents[1]
12+
old = int(sys.argv[1]) if len(sys.argv) > 1 else 14
13+
if old not in (14, 16):
14+
raise SystemExit('Only 14/17 and 16/17 fixture pairs are defined')
15+
output = Path(sys.argv[2]) if len(sys.argv) > 2 else Path('/tmp') / ('mysqlpunk-pg-version-' + uuid.uuid4().hex)
16+
output.mkdir(parents=True, exist_ok=False)
17+
project = root / 'tests/PostgreSqlVersionMigrationHarness/Review.csproj'
18+
subprocess.run(['dotnet', 'build', str(project), '-c', 'Release'], check=True)
19+
key = 'mysqlpunk-pg-version-' + uuid.uuid4().hex
20+
environment = os.environ.copy()
21+
environment['POSTGRES_PASSWORD'] = 'Aa!' + secrets.token_hex(18)
22+
environment['MYSQLPUNK_PG_TEST_PASSWORD'] = environment['POSTGRES_PASSWORD']
23+
environment['MYSQLPUNK_OLD_PG_MAJOR'] = str(old)
24+
containers = []
25+
metadata = {'sources': {}, 'images': [], 'cleanup': []}
26+
for path in [project, *project.parent.glob('*.cs'), Path(__file__).resolve(),
27+
*sorted((root / 'mySQLPunk.Core').rglob('*.cs'))]:
28+
if 'obj' not in path.parts and 'bin' not in path.parts:
29+
metadata['sources'][str(path.relative_to(root))] = hashlib.sha256(path.read_bytes()).hexdigest()
30+
31+
def docker(*arguments):
32+
result = subprocess.run(['docker', *arguments], env=environment, text=True, capture_output=True, timeout=60)
33+
if result.returncode:
34+
raise RuntimeError('Docker operation failed: ' + arguments[0])
35+
return result.stdout.strip()
36+
37+
try:
38+
ports = []
39+
for major in (old, 17):
40+
image = json.loads(docker('image', 'inspect', f'postgres:{major}-alpine', '--format', '{{json .}}'))
41+
container = docker('run', '-d', '--name', key + '-' + str(major),
42+
'--label', 'mysqlpunk.test.owner=' + key, '--cpus=1', '--memory=768m',
43+
'--pids-limit=512', '--mount', 'type=tmpfs,destination=/var/lib/postgresql/data,tmpfs-size=536870912',
44+
'-e', 'POSTGRES_PASSWORD', '-p', '127.0.0.1::5432', image['Id'])
45+
containers.append(container)
46+
ports.append(docker('port', container, '5432/tcp').split(':')[-1])
47+
metadata['images'].append({'major': major, 'id': image['Id'], 'digests': image.get('RepoDigests', [])})
48+
command = ['dotnet', str(project.parent / 'bin/Release/net8.0/Review.dll'), *ports]
49+
for attempt in range(30):
50+
probe = subprocess.run(command + ['probe'], env=environment, text=True, capture_output=True, timeout=15)
51+
if probe.returncode == 0:
52+
metadata['versions'] = probe.stdout.strip().splitlines()
53+
print('Actual versions: ' + ' / '.join(metadata['versions']), flush=True)
54+
break
55+
time.sleep(2)
56+
else:
57+
raise RuntimeError('Isolated PostgreSQL engines did not become ready')
58+
result = subprocess.run(command, env=environment, text=True, capture_output=True, timeout=180)
59+
text = (result.stdout + '\n' + result.stderr).replace(environment['POSTGRES_PASSWORD'], '[redacted]')
60+
(output / 'migration.log').write_text(text)
61+
print(text, flush=True)
62+
if result.returncode:
63+
raise RuntimeError('Formal Core project migration regression failed')
64+
finally:
65+
failures = []
66+
for container in containers:
67+
try:
68+
if docker('inspect', container, '--format', '{{index .Config.Labels "mysqlpunk.test.owner"}}') != key:
69+
raise RuntimeError('Refusing cleanup: ownership mismatch')
70+
docker('rm', '-f', container)
71+
metadata['cleanup'].append(container)
72+
except Exception as exception:
73+
failures.append({'container': container, 'error': str(exception)})
74+
metadata['cleanup_failures'] = failures
75+
(output / 'evidence.json').write_text(json.dumps(metadata, indent=2) + '\n')
76+
print('Evidence: ' + str(output), flush=True)
77+
if failures:
78+
raise RuntimeError('Owned fixture cleanup incomplete; see evidence.json')

0 commit comments

Comments
 (0)