Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/cross-platform.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ on:
- "mySQLPunk.Desktop/**"
- "mySQLPunk.CrossPlatform.SmokeTests/**"
- "tests/Run-CrossPlatformLiveTests.sh"
- "tests/Run-PostgreSqlVersionMigration.py"
- "tests/PostgreSqlVersionMigrationHarness/**"
- "tests/Run-DesktopProfileOperationTests.sh"
- "tests/DesktopProfileOperationHarness/**"
- "tests/Test-LinuxCrossPlatformPackage.sh"
Expand All @@ -24,6 +26,8 @@ on:
- "mySQLPunk.Desktop/**"
- "mySQLPunk.CrossPlatform.SmokeTests/**"
- "tests/Run-CrossPlatformLiveTests.sh"
- "tests/Run-PostgreSqlVersionMigration.py"
- "tests/PostgreSqlVersionMigrationHarness/**"
- "tests/Run-DesktopProfileOperationTests.sh"
- "tests/DesktopProfileOperationHarness/**"
- "tests/Test-LinuxCrossPlatformPackage.sh"
Expand Down Expand Up @@ -82,6 +86,9 @@ jobs:
- name: Run cross-platform smoke tests
run: dotnet run --project mySQLPunk.CrossPlatform.SmokeTests/mySQLPunk.CrossPlatform.SmokeTests.csproj -c Release --no-build

- name: Build formal PostgreSQL version fixture
run: dotnet build tests/PostgreSqlVersionMigrationHarness/Review.csproj -c Release

- name: Run isolated Avalonia profile operation handlers
if: matrix.name == 'Linux x64'
run: ./tests/Run-DesktopProfileOperationTests.sh
Expand Down
22 changes: 20 additions & 2 deletions docs/LOGICAL_MIGRATION_MATRIX.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,26 @@
| SQL Server → SQL Server | 2016/2017/2019/2022/2025 任意雙向及同版本,共 25 組 | Windows 既有專用工具;dbo heap、既有型別白名單、同定序、無 advanced features | 25 組預檢及交易 mock;另有正式來源 2019 ↔ 2022 隔離回歸,1007 列/7 欄+空表,不是 25 組真引擎驗證 |
| SQLite → SQLite | 3.37–3.53 | STRICT、INTEGER/BLOB/TEXT、NULL/NOT NULL、非自增 PK、普通索引;詳細子集見下文 | 本機實際 SQLite 3.53.3;歷史 minor 版本未逐一跑 |
| SQLite → PostgreSQL | SQLite 3.37–3.53 → PostgreSQL 14–17 | INTEGER → bigint、BLOB → bytea、TEXT BINARY → text C;非自增 PK、普通索引 | 隔離 CI PostgreSQL 16 fixture 加入文字/鍵/索引雙向 round-trip;只以這次新 head 結果為證據 |
| PostgreSQL → SQLite | PostgreSQL 14–17 → SQLite 3.37–3.53 | 上述型別反向映射;PK → WITHOUT ROWID、普通索引 | 同上;14/15/17 未實機跑 |
| PostgreSQL → PostgreSQL | 14–17 雙向及同版本 | 同一 bigint/bytea/text C、PK/索引子集與全部預檢 | 邏輯共用路徑;沒有完整跨版本真機矩陣 |
| PostgreSQL → SQLite | PostgreSQL 14–17 → SQLite 3.37–3.53 | 上述型別反向映射;PK → WITHOUT ROWID、普通索引 | PG17 實際 roundtrip 與既有 PG16 fixture;PG15 未實機跑 |
| PostgreSQL → PostgreSQL | 14–17 雙向及同版本 | 同一 bigint/bytea/text C、PK/索引子集與全部預檢 | 真實 17.10 ↔ 14.24、17.10 ↔ 16.15;不是全部版本配對驗證 |

## 可重跑的 PostgreSQL 跨版本證據

`python3 tests/Run-PostgreSqlVersionMigration.py 14 [新證據目錄]` 與 `python3 tests/Run-PostgreSqlVersionMigration.py 16 [新證據目錄]` 直接 ProjectReference 正式 Core,不使用凍結副本。需先快取官方 `postgres:14-alpine`/`postgres:16-alpine` 與 `postgres:17-alpine`;腳本只 image inspect,不拉取映像或調整網路/權限。每次建立唯一 ownership label、loopback 隨機 port 與 tmpfs 容器,最後核對 label 才清理自己的 fixture;密碼隨機產生且不輸出。

每輪為 SQLite seed → PG17 → 舊版 PG → 空 PG17 → SQLite,records 1007 列+empty_table 空表,複合 PK、具名普通索引、UTF8/emoji、NULL/空 binary/空 text、Int64 極值。每段使用 production 預檢、交易、schema 與 SHA256 多重集合驗證,最後另測重複 PK 被拒絕及索引存在。這不是 procedures、FK、identity、任意型別、SSH 或桌面 UI 的跨版本證明;1005 列 transaction fault 仍只屬既有 SQLite 回歸,provider 全套 56/56 仍是另一組測試。

2026-10-14 獨立證據先以 fa734e7 跑 17 ↔ 14/16,再以 06 工作樹跑 17 ↔ 14。06 最終 95c0be2(合入 4368883)三個 production 檔案 SHA256 均與獨立工作樹一致:Models/LogicalMigration.cs `a912ddcd778bac06c4980f3f3403d9b0d4563b81414c170894b79619bf79912e`、Services/LogicalMigrationService.cs `721ec60b1be6c2ada40114e59c323513c44aa1124e52ffa9411cb6cd23095fd4`、Services/LogicalMigrationKeys.cs `69fc1e7ea82ab7f27c2ce22d4d14613294f981f73bdb0b4f99e04893ef793cef`。新 harness 再以相同正式來源跑兩組成功;並非重跑舊 SHA CI。

當次實際官方映像 RepoDigest(後續 tag 可能改變,重跑以 evidence.json 的實際版本及 digest 為準):

| 實際版本 | 官方 RepoDigest |
| --- | --- |
| 14.24 | postgres@sha256:4ea9e5ed06591da7ea23eb65465e8d3187fe79f4d5ec3ae976d29a33b013e77a |
| 16.15 | postgres@sha256:721873c34ceb9f8d8fc265984940dc982404c105f19ad51be9fdc5970a6080ea |
| 17.10 | postgres@sha256:742f40ea20b9ff2ff31db5458d127452988a2164df9e17441e191f3b72252193 |

`migration.log`、`evidence.json` 保留實際版本、映像 ID/digest、正式來源 hash 與清理结果;不存密碼或資料值。原獨立日誌仍在 audit-evidence/postgresql-14-17-real-versions.log、postgresql-16-17-real-versions.log、postgresql-14-17-primary-name-fix.log,沒有覆寫。

## 第二階段:實用文字與鍵索引子集

Expand Down
107 changes: 107 additions & 0 deletions tests/PostgreSqlVersionMigrationHarness/Program.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
using Microsoft.Data.Sqlite;
using MySqlPunk.Core.Models;
using MySqlPunk.Core.Services;
using Npgsql;

var oldMajor = int.Parse(Environment.GetEnvironmentVariable("MYSQLPUNK_OLD_PG_MAJOR") ?? "14");
if (oldMajor is not (14 or 16)) throw new ArgumentException("Only the 14/17 and 16/17 fixtures are defined.");
var ports = new[] { int.Parse(args[0]), int.Parse(args[1]) };
var password = Environment.GetEnvironmentVariable("MYSQLPUNK_PG_TEST_PASSWORD")
?? throw new InvalidOperationException("Missing isolated fixture password.");
string Connection(int port, string database) => new NpgsqlConnectionStringBuilder
{
Host = "127.0.0.1", Port = port, Username = "postgres", Password = password,
Database = database, SslMode = SslMode.Disable, Pooling = false, Timeout = 3
}.ConnectionString;
foreach (var port in ports)
{
await using var connection = new NpgsqlConnection(Connection(port, "postgres"));
await connection.OpenAsync();
Console.WriteLine("Actual PostgreSQL version=" + connection.PostgreSqlVersion);
var expectedMajor = port == ports[0] ? oldMajor : 17;
if (connection.PostgreSqlVersion.Major != expectedMajor) throw new Exception("Unexpected engine major.");
}
if (args.Length > 2) return;
var root = Path.Combine(Path.GetTempPath(), "mysqlpunk-version-fixture-" + Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(root);
try
{
foreach (var port in ports)
{
await using var connection = new NpgsqlConnection(Connection(port, "postgres"));
await connection.OpenAsync();
foreach (var database in new[] { "migration_source", "migration_back" })
{
await using var command = connection.CreateCommand();
command.CommandText = "CREATE DATABASE " + database;
await command.ExecuteNonQueryAsync();
}
}
ConnectionProfile Pg(int index, string database) => new()
{
Name = "isolated PG", Provider = DatabaseProviderKind.PostgreSql, Host = "127.0.0.1",
Port = ports[index], Username = "postgres", Password = password,
Database = database, TlsMode = ConnectionTlsMode.Disabled
};
var source = new ConnectionProfile
{
Name = "isolated SQLite", Provider = DatabaseProviderKind.Sqlite,
Database = Path.Combine(root, "source.db")
};
using (var connection = new SqliteConnection("Data Source=" + source.Database))
{
connection.Open();
using var command = connection.CreateCommand();
command.CommandText = """
CREATE TABLE records (id INTEGER NOT NULL, label TEXT NOT NULL, note TEXT, payload BLOB,
PRIMARY KEY(id,label)) STRICT, WITHOUT ROWID;
CREATE INDEX records_label ON records(label);
INSERT INTO records VALUES
(1,'abc','繁體中文 😺',X'00FF'),(1,'ABC','',NULL),
(2,'a','前後空白 ',X''),(2,'a ',NULL,X'010203'),(3,'é','é',NULL);
CREATE TABLE empty_table (number INTEGER,payload BLOB,note TEXT) STRICT;
WITH RECURSIVE numbers(number) AS (VALUES(1) UNION ALL SELECT number+1 FROM numbers WHERE number<1000)
INSERT INTO records SELECT 100+number,'row_'||number,
CASE WHEN number%3=0 THEN NULL ELSE '測試😀 ' END,
CASE WHEN number%2=0 THEN NULL ELSE X'00FF' END FROM numbers;
INSERT INTO records VALUES(-9223372036854775808,'min',NULL,X''),(9223372036854775807,'max','',NULL);
""";
command.ExecuteNonQuery();
}
async Task Move(ConnectionProfile from, ConnectionProfile to, int major, string label)
{
var preview = await LogicalMigrationService.PreviewAsync(from, to, major);
Console.WriteLine(label + " preview=" + preview.CanStart + " " + string.Join(";", preview.Blockers));
if (!preview.CanStart) throw new Exception(label + " preflight rejected");
var result = await LogicalMigrationService.RunAsync(from, to, major, preview, to.Database);
Console.WriteLine(label + " " + result.Outcome + " rows=" + preview.Tables.Sum(table => table.Rows));
if (result.Outcome != LogicalMigrationOutcome.Verified) throw new Exception(label + " " + result.Report);
}
var old = Pg(0, "migration_source");
var newer = Pg(1, "migration_source");
var back = Pg(1, "migration_back");
await Move(source, newer, 17, "SQLite -> PG17 seed");
await Move(newer, old, oldMajor, "PG17 -> PG" + oldMajor + " downgrade");
await Move(old, back, 17, "PG" + oldMajor + " -> PG17 upgrade");
var final = source.Clone(); final.Database = Path.Combine(root, "back.db");
using (var connection = new SqliteConnection("Data Source=" + final.Database)) connection.Open();
await Move(back, final, 3, "PG17 -> SQLite roundtrip");
using (var connection = new SqliteConnection("Data Source=" + final.Database))
{
connection.Open();
using var command = connection.CreateCommand();
command.CommandText = "SELECT count(*) FROM records";
if (Convert.ToInt64(command.ExecuteScalar()) != 1007) throw new Exception("Roundtrip count.");
command.CommandText = "SELECT count(*) FROM sqlite_schema WHERE type='index' AND name='records_label'";
if (Convert.ToInt64(command.ExecuteScalar()) != 1) throw new Exception("Named index lost.");
command.CommandText = "INSERT INTO records VALUES(1,'abc',NULL,NULL)";
try { command.ExecuteNonQuery(); throw new Exception("Composite PK lost."); }
catch (SqliteException exception) when (exception.SqliteErrorCode == 19) { }
}
Console.WriteLine("PASS actual PG17 -> PG" + oldMajor + " -> PG17 plus SQLite roundtrip; 1007 rows + empty table; composite PK, named index, UTF8/emoji, NULL/empty binary/text, signed Int64 extrema; production transaction and SHA256 verification");
}
finally
{
SqliteConnection.ClearAllPools();
Directory.Delete(root, true);
}
11 changes: 11 additions & 0 deletions tests/PostgreSqlVersionMigrationHarness/Review.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="../../mySQLPunk.Core/mySQLPunk.Core.csproj" />
</ItemGroup>
</Project>
78 changes: 78 additions & 0 deletions tests/Run-PostgreSqlVersionMigration.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
import hashlib
import json
import os
from pathlib import Path
import secrets
import subprocess
import sys
import time
import uuid

root = Path(__file__).resolve().parents[1]
old = int(sys.argv[1]) if len(sys.argv) > 1 else 14
if old not in (14, 16):
raise SystemExit('Only 14/17 and 16/17 fixture pairs are defined')
output = Path(sys.argv[2]) if len(sys.argv) > 2 else Path('/tmp') / ('mysqlpunk-pg-version-' + uuid.uuid4().hex)
output.mkdir(parents=True, exist_ok=False)
project = root / 'tests/PostgreSqlVersionMigrationHarness/Review.csproj'
subprocess.run(['dotnet', 'build', str(project), '-c', 'Release'], check=True)
key = 'mysqlpunk-pg-version-' + uuid.uuid4().hex
environment = os.environ.copy()
environment['POSTGRES_PASSWORD'] = 'Aa!' + secrets.token_hex(18)
environment['MYSQLPUNK_PG_TEST_PASSWORD'] = environment['POSTGRES_PASSWORD']
environment['MYSQLPUNK_OLD_PG_MAJOR'] = str(old)
containers = []
metadata = {'sources': {}, 'images': [], 'cleanup': []}
for path in [project, *project.parent.glob('*.cs'), Path(__file__).resolve(),
*sorted((root / 'mySQLPunk.Core').rglob('*.cs'))]:
if 'obj' not in path.parts and 'bin' not in path.parts:
metadata['sources'][str(path.relative_to(root))] = hashlib.sha256(path.read_bytes()).hexdigest()

def docker(*arguments):
result = subprocess.run(['docker', *arguments], env=environment, text=True, capture_output=True, timeout=60)
if result.returncode:
raise RuntimeError('Docker operation failed: ' + arguments[0])
return result.stdout.strip()

try:
ports = []
for major in (old, 17):
image = json.loads(docker('image', 'inspect', f'postgres:{major}-alpine', '--format', '{{json .}}'))
container = docker('run', '-d', '--name', key + '-' + str(major),
'--label', 'mysqlpunk.test.owner=' + key, '--cpus=1', '--memory=768m',
'--pids-limit=512', '--mount', 'type=tmpfs,destination=/var/lib/postgresql/data,tmpfs-size=536870912',
'-e', 'POSTGRES_PASSWORD', '-p', '127.0.0.1::5432', image['Id'])
containers.append(container)
ports.append(docker('port', container, '5432/tcp').split(':')[-1])
metadata['images'].append({'major': major, 'id': image['Id'], 'digests': image.get('RepoDigests', [])})
command = ['dotnet', str(project.parent / 'bin/Release/net8.0/Review.dll'), *ports]
for attempt in range(30):
probe = subprocess.run(command + ['probe'], env=environment, text=True, capture_output=True, timeout=15)
if probe.returncode == 0:
metadata['versions'] = probe.stdout.strip().splitlines()
print('Actual versions: ' + ' / '.join(metadata['versions']), flush=True)
break
time.sleep(2)
else:
raise RuntimeError('Isolated PostgreSQL engines did not become ready')
result = subprocess.run(command, env=environment, text=True, capture_output=True, timeout=180)
text = (result.stdout + '\n' + result.stderr).replace(environment['POSTGRES_PASSWORD'], '[redacted]')
(output / 'migration.log').write_text(text)
print(text, flush=True)
if result.returncode:
raise RuntimeError('Formal Core project migration regression failed')
finally:
failures = []
for container in containers:
try:
if docker('inspect', container, '--format', '{{index .Config.Labels "mysqlpunk.test.owner"}}') != key:
raise RuntimeError('Refusing cleanup: ownership mismatch')
docker('rm', '-f', container)
metadata['cleanup'].append(container)
except Exception as exception:
failures.append({'container': container, 'error': str(exception)})
metadata['cleanup_failures'] = failures
(output / 'evidence.json').write_text(json.dumps(metadata, indent=2) + '\n')
print('Evidence: ' + str(output), flush=True)
if failures:
raise RuntimeError('Owned fixture cleanup incomplete; see evidence.json')
Loading