diff --git a/README.md b/README.md index 484b536..87bcccc 100644 --- a/README.md +++ b/README.md @@ -248,7 +248,7 @@ SQLite / PostgreSQL / SQL Server database rename 實機矩陣(需先啟動 Doc | Database inline rename | 可用 | Database、Table、View 可用 F2 或右鍵改名;Esc 只取消編輯。MySQL 以完整 SQL 串流複製並保留舊 DB,PostgreSQL/SQL Server 使用原生 rename,SQLite 會移動檔案並更新連線路徑。 | | PostgreSQL | 可用 | 支援 metadata、資料瀏覽、資料編輯、DDL、Dump、Table Designer;`public` 以外的 schema 會以 `schema.table` 顯示並可用於主要資料表操作,部分進階索引仍有限制。 | | SQLite | 可用 | 支援一般 SQLite 與 SpatiaLite 載入;欄位註解以 mySQLPunk sidecar metadata table 保存。 | -| SQL Server | 可用 | 支援 metadata、資料瀏覽、資料編輯、DDL、Dump、Table Designer;`dbo` 以外的 schema 會以 `schema.table` 顯示並可用於主要資料表操作。 | +| SQL Server | 可用 | 支援 metadata、資料瀏覽、資料編輯、DDL、Dump、Table Designer;`dbo` 以外的 schema 會以 `schema.table` 顯示並可用於主要資料表操作。Windows「執行 SQL 檔」提供串流 GO 匯入精靈、進度/取消與錯誤紀錄,詳見 [MSSQL SQL 檔匯入](docs/MSSQL_SQL_IMPORT.md)。 | | Oracle | 部分可用 | 支援 schema/table/view metadata、資料瀏覽、資料編輯、DDL、Dump、Table Designer;部分 DDL 仍受權限、語法與物件型態限制。 | | MongoDB | 第三期可用 | 支援一般與 SRV 連線、URI 匯入、database/collection/view metadata、抽樣 schema 推斷、索引與統計資訊、分頁文件瀏覽與 JSON find 查詢;文件檢視器提供可展開文件樹、單一文件安全編輯(`_id` 鎖定+樂觀並行比對)、文件新增與安全刪除。standalone 4.4/7.0/8.0 已通過實機矩陣;Atlas/SRV 驗證環境與 Aggregation Pipeline 仍在後續階段。 | | Redis / Microsoft Garnet | 第三期可用 | 支援 `redis://`/`rediss://`、ACL/密碼驗證、logical db、TLS 直連、key 型別/TTL/摘要瀏覽與 pattern/type/單一 key 受限查詢;key 編輯器依型別切換,string 值、hash 欄位、list 元素(含依索引安全刪除與尾端新增)、set 成員與 zset 分數都以 WATCH/MULTI/EXEC 並行保護寫入,另有 TTL 設定/移除與刪除 key。資料庫節點可開啟 INFO 即時監控及 Pub/Sub 訊息工作區,支援 channel/pattern 訂閱與明確發布。Redis 6.2、Redis 7 與 Garnet standalone 既有各 39 項實機矩陣通過,新增案例待重跑;Cluster/Sentinel 仍待補。 | diff --git a/docs/MSSQL_SQL_IMPORT.md b/docs/MSSQL_SQL_IMPORT.md new file mode 100644 index 0000000..0e44800 --- /dev/null +++ b/docs/MSSQL_SQL_IMPORT.md @@ -0,0 +1,123 @@ +# MSSQL SQL 檔匯入 + +從資料庫樹選定 MSSQL 資料庫,再使用既有「執行 SQL 檔」入口,即可開啟原生四步匯入精靈。 +本流程適用 mySQLPunk 匯出檔與本版支援的一般 T-SQL;請先確認腳本來源可信。 + +## 四步操作 + +1. **檔案與目標**:選擇 SQL 檔,核對檔案大小及已選資料庫。 + 目標欄位唯讀;若要換目標,關閉精靈,在資料庫樹另選資料庫,再重新開啟。 + 實際解析的 server/instance/database 會於預檢後顯示,請以該識別為準。 +2. **匯入檢查**:完整串流掃描檔案,檢查編碼、GO、批次大小、native 摘要與目標條件。 + 預檢只讀來源檔與資料庫 metadata,不執行檔案內的 SQL;失敗時不能進入執行。 + 完成前請等待掃描進度;更換檔案或錯誤政策後須重新檢查。 +3. **確認執行**:核對實際目標、來源、錯誤政策與完整 log 路徑。 + 勾選「已確認目標資料庫,並了解異常時可能保留部分資料。」後按「開始匯入」。 + 開始前仍會重查目標狀態;預檢通過不保證任意 SQL 都有權限或能成功。 +4. **執行與結果**:查看已處理範圍、批次、行號、時間、錯誤及最後結果。 + 結束後可開啟 log、開啟資料夾、複製路徑,或按「重新檢查匯入」。 + 重新檢查會回到預檢,不會直接重播失敗批次。 + +來源檔以唯讀方式保持開啟,預檢後以同一 handle 重新讀取;來源內容不改寫。 +開始前請自行確認備份與目標現況,尤其是已有資料的一般腳本或資料型匯出。 + +## 進度、Punky 與聲音 + +Punky 自動循環動畫;每次實際執行只在首輪播放一次聲音。 +使用「靜音/開啟聲音」切換;取消靜音不會重播,沒有動畫播放/暫停按鈕。 +預檢進度與執行進度分開;執行 bytes 只在批次回應並處理完成後確認。 +讀檔預取的位置不是已匯入位置;成功與失敗/略過的範圍分別統計。 +最後只有註解的 bytes,也要等 EOF 安全檢查通過後才確認。 +略過錯誤時,「已處理檔案範圍」到 100% 仍可能有失敗批次。 +ETA 由最近確認的 bytes 速率估算,需先取得足夠樣本。 +長指令、索引、約束、結果核對或久無新進度時會顯示計算中/目前指令執行中。 +請勿把讀完整份檔案或 ETA 歸零當成匯入與核對已完成。 + +## 錯誤政策與部分寫入 + +預設遇到 SQL 錯誤停止。可勾選「遇到 SQL 錯誤時,略過失敗 GO 批次並繼續」。 +這是略過整個失敗 GO 批次,不是略過單一 row;同批較早已生效的語句可能保留。 +只有一般 SQL 錯誤、原連線與目標正常、前後無交易、native LOB 狀態安全且 log 已保存時才可略過。 +timeout、取消、斷線、嚴重錯誤、結果不明、交易/LOB 異常及 log 寫入失敗仍停止。 +精靈不自動重試、不支援斷點續傳,也不替整份腳本加上外層 transaction。 +腳本自行開啟的交易照原順序處理;跨 GO 交易遇錯不能自動略過。 +成功批次的回應不等於其中交易已提交;已提交變更不會因後續停止而整份撤銷。 + +## 支援的 SQL 檔格式 + +- 嚴格 UTF-8(有/無 BOM),以及有 BOM 的 UTF-16 LE/BE。 +- 無效解碼序列拒絕執行;SQL 文字中的邏輯 U+0000 不支援,會回報 EmbeddedNul。 + 它可能是沒有 BOM 的 UTF-16,也可能是 UTF-8 中的 literal NUL,不能一概當成編碼錯誤。 +- GO 不分大小寫,須是字串、identifier 與註解外的獨立一行;可有前後空白。 +- 支援 `GO -- 註解`;字串/註解裡的 GO 保持原文,不送分隔用的 GO 給 SQL Server。 +- 不支援 `GO n`、同一 GO 分隔行的 block comment、SQLCMD 指令或變數展開。 + `:r`、`:setvar`、`:connect`、`!!` 等有效位置的指令會被預檢拒絕。 +- EOF 的最後一個非空批次會執行;未閉合字串、identifier 或 block comment 會被預檢擋下。 +- 不用分號拆開任意 T-SQL;單一 GO 批次解碼後上限為 32 MiB(16,777,216 UTF-16 code units)。 + 超過上限會在來源 SQL 寫入前停止;請調整來源產生方式,勿任意插 GO 破壞 SQL 語意。 + +32 MiB 是批次界限,不是 LOB 欄位大小限制,也不是整個程序的記憶體上限。 +mySQLPunk 既有文字/binary/Geometry 分段協定保持原樣,使用同一 session 組裝與驗證。 +缺失、重複、亂序、未消耗或 Failed 的 native chunk 狀態不能靠略過錯誤當成成功。 + +## Session 與目標邊界 + +全部 GO 批次依序使用同一專用 session,保留 SET、#temp、IDENTITY_INSERT 與交易連續性。 +批次間必要的目標/交易探測可能改變 `@@ROWCOUNT` 與 `@@ERROR`。 +若腳本需要這兩值,請在同一批次先存入變數/temp table;本版不保證跨 GO 保留。 +直接 `USE` 的解碼目標必須與選定資料庫完全相符(Ordinal,含大小寫);其他直接 USE 會被預檢拒絕。 +三段名稱、動態 SQL 或程序仍可能存取其他資料庫。 +請確認腳本可信;本功能不是 SQL sandbox,也不提供完整 SSMS/sqlcmd 相容環境。 +精靈不自動 GRANT、建立/清空/覆寫資料庫、變更資料庫設定、放寬約束為 NOCHECK 或 KILL session。 +執行中的腳本仍依登入權限變更資料;上述邊界不是任意 SQL 的隔離保證。 + +## 取消與清理 + +按「取消匯入」或執行中關閉視窗會停止排程並嘗試取消目前 command。 +「正在停止…」表示仍在等待實際結束,不能當成已取消,也不能立即開始另一輪。 +取消約 10 秒未得到回應時,會嘗試關閉本次擁有的連線;仍須等待實際 unwind/清理證據。 +清理包含嘗試 rollback 本 session 未提交交易及釋放 command、session、來源 handle 等資源。 +若釋放仍 pending 或執行/rollback 結果無法確認,結果會明示 CleanupUncertain 與診斷。 +Cancelled 也不表示整份還原到執行前;已提交或部分生效的資料可能保留。 +失敗/取消後先依 log 核對目標現況;精靈不自動清除部分物件或重跑。 + +## 完整錯誤紀錄 + +每次建立獨立的新檔,不覆寫舊 log: +`%LOCALAPPDATA%\mySQLPunk\ImportLogs\import--.log`。 +來源 SQL 開始寫入前須已成功建立 log;追加或 flush 失敗就停止,不能宣稱完整保存。 +完整 log 保留所有 SQL error detail、批次/來源座標、採取動作、狀態探測與最後清理結果。 +來源執行錯誤與 Probe/Cleanup 錯誤分開標示;後兩者不能冒稱來源腳本行號。 +畫面錯誤區唯讀,只保留最近 100 筆或 32,768 UTF-16 code units(先到者為準)。 +單筆過長也可能截斷;截段提示與總錯誤數不代表完整 log 遺失,請查看完整路徑。 +「開啟 log」「開啟資料夾」「複製路徑」使用實際已建立的檔案路徑。 +log 不附整段 SQL/LOB 或密碼,但 SQL Server 原始錯誤訊息可能含資料值,請妥善保存。 + +## 如何理解完成結果 + +NativeSchema(含完整結構的 native 匯出)要求已知空庫、完整 metadata 可見性及可寫目標。 +需要建表時也須具 CREATE TABLE 能力;原有 schema 本身不代表庫內已有使用者物件。 +完整 metadata 採保守證明:dbo,或資料庫 VIEW DEFINITION;SQL major ≥16 的非 dbo 還需 VIEW SECURITY DEFINITION。 +非 dbo 的有效身分若有任何明確 DENY,也可能擋下 NativeSchema,即使該 DENY 與可見物件無關。 +Generic/DataOnly 可保留可見性警告,再由實際 SQL 權限判定;沒有一律要求 CREATE VIEW。 +無錯誤的 NativeSchema 會核對實際 user table 數與逐表 COUNT_BIG,對照 footer 摘要。 +Generic/DataOnly 無同等內容 manifest;成功只聲明批次完成與必要安全檢查,未提供資料內容核對。 +所有模式仍要通過適用的 EOF、交易、目標、digest 與 native LOB 檢查。 +「執行完成,但有錯誤」不等於完整還原通過;請核對失敗/略過及未確認批次。 +表數/筆數與 footer 相符也不是逐欄值或來源資料庫 hash 相等的證明。 +來源 READ COMMITTED 的跨表不一致可能造成後續外鍵失敗;精靈不自動刪資料或放寬約束。 + +## 已有證據與待驗收 + +[設計與相容邊界](design/mssql-import/design.md);[詳細驗證紀錄](design/mssql-import/verification.md)。 +專用 LocalDB 上已匹配 65 個實際 PASS case receipts,包含大 LOB/Geometry、錯誤與取消邊界。 +已測 64 MiB binary、18 MiB Unicode、>16 MiB Geometry;測試資料另有 bytes/hash/metadata 比對。 +約 256 MiB/2.5 GiB 相同批次形狀檔案已量測記憶體;這是該 fixture 的觀測值,不是通用上限。 +SQL Server 2019/Transfer 已實際試跑一次,於第 1,481 批因日期字串轉換錯誤 241 停止;未完整還原。 +已確認前 1,480 批成功;停止後唯讀獨立核對為 126 張表/46 筆,不符合來源摘要 3,247,785 筆。 +既有匯出器將 legacy datetime/smalldatetime 寫成七位小數;現已修正為三位,並通過獨立 Core 日期 bytes 比對。 +原始大檔與 Transfer 部分狀態、一次性 marker/log 均保留,沒有修改來源、清庫或重跑;修正不代表原始大檔已通過。 +實際音訊聽感、實體高 DPI/鍵盤操作與 production 驗收仍未確認;本機證據不延伸為 IIS/DB 生產驗收。 + +註:Task7 一次試跑 harness 的 64 GiB 可用空間入口、<20 GiB 時取消與固定 marker 是該試驗的防護條件。 +該 harness 的私有來源路徑與限制不是一般精靈的 UI 限制;普通匯入依上述檔案與目標檢查進行。 diff --git a/docs/design/mssql-import/design.md b/docs/design/mssql-import/design.md new file mode 100644 index 0000000..27577f6 --- /dev/null +++ b/docs/design/mssql-import/design.md @@ -0,0 +1,184 @@ +# MSSQL SQL 檔匯入精靈設計 + +日期:2026-10-13 +基底:master,d45c903(已合併 MSSQL 匯出修正) +狀態:使用者於 2026-10-13 核准本規格;[實作計畫](implementation-plan.md) 的功能與隔離驗證已完成。SQL2019 實檔於日期格式錯誤停止,完整還原未通過;已修正舊匯出器並保留部分目標與紀錄。詳見 [驗證結果](verification.md)。 + +## 1. 需求與本次範圍 + +把 MSSQL 的「執行 SQL 檔/匯入」升級成專用精靈,支援 mySQLPunk 匯出的數 GiB SQL 檔及一般 T-SQL/GO。沿用匯出精靈的 Punky、自動循環動畫、一次聲音與靜音、進度、預估剩餘時間及預計完成時間。 + +使用者可勾選「遇到 SQL 錯誤時,略過失敗批次並繼續」,預設不勾。所有錯誤都寫入本機 error log;畫面提供唯讀 textarea,數量太多時顯示最近一段並清楚標示完整 log 路徑。 + +這次只改 MSSQL 的檔案執行入口,保留 MySQL/MariaDB/SQLite 及一般查詢編輯器的既有行為。支援順序執行,不做平行匯入、斷點續傳、自動重試、SQLCMD 指令、原生 .bak 還原或自動清空資料庫。 + +### 現有問題與方案選擇 + +舊入口會把檔案整份讀成 byte[]/string,再依分號拆解並使用共用 provider;不支援 GO、獨立取消或可信進度,傳入的 databaseName 也沒有在執行前明確切換。新流程使用獨立擁有的連線與串流 GO reader,不把大檔交給舊 splitter。 + +| 方案 | 取捨 | +| --- | --- | +| 程式內串流 GO reader+專用 SqlClient session(採用) | 能取得每個批次的實際回應、錯誤、取消與進度;須完整驗證 lexer 及 bounded memory | +| 呼叫外部 sqlcmd | 可作獨立驗證工具,但需處理版本、外部程序及每批回應;不作精靈主要執行器 | +| 擴大舊整檔讀取流程 | 記憶體隨檔案增加,也無法保持正確 GO/LOB session 行為,不採用 | + +## 2. 已核准畫面與保存位置 + +互動原型:[index.html](index.html)。原型的 SQL 執行、error log 與數值均為模擬,不連接 DB,也不寫入實際匯入紀錄。 + +截圖存入本設計的 `snapshot/`,保留原始瀏覽器截圖,沒有重新縮圖: + +| 畫面 | 圖片 | +| --- | --- | +| 淺色,匯入進度、ETA 與 Punky | [progress-light.jpg](snapshot/progress-light.jpg) | +| 深色,帶錯誤完成與大量 error log | [error-log-dark.jpg](snapshot/error-log-dark.jpg) | + +四步流程如下: + +1. **檔案與目標**:選 SQL 檔,顯示大小、編碼、來源摘要及實際解析的 server/instance/database。使用既有連線設定;本次本機試跑使用 Windows 整合登入。 +2. **匯入檢查**:完整串流預掃描,檢查 GO/編碼/批次上限、檔案完整標記、目標狀態、基本權限與 log 可寫性。顯示掃描進度,失敗時指出批次/行號並停用下一步。 +3. **確認執行**:再次顯示檔案、實際目標、錯誤政策、log 路徑及部分寫入可能保留的說明。略過錯誤預設關;使用者確認目標後才可開始。 +4. **執行與結果**:依序送出批次,顯示已處理檔案範圍、成功/失敗批次、目前行號、已用時間、ETA 與最近錯誤。結束後保留結果與 log 開啟/資料夾/複製路徑功能。 + +Punky 預設自動循環播放,不設播放或暫停按鈕;聲音僅首輪一次,可靜音。執行、停止中、完成、帶錯誤及失敗使用既有素材與狀態文案。過程中取消按鈕保持可到達;視窗縮放、高 DPI、深淺色與鍵盤操作均須驗證。 + +## 3. 檔案預掃描與 GO 規則 + +檔案以唯讀 handle 開啟,採 `FileShare.Read`,在預掃描、確認與執行期間持續持有;離開精靈或更換檔案才釋放。無法取得穩定讀取 handle 時不開始。預掃描只讀檔與 DB metadata,不執行檔內 SQL。 + +採兩次順序讀取:先完整掃描取得總 bytes/批次數/最大批次/SHA-256/格式資訊,再 rewind 同一個 handle 執行。兩次都用固定 buffer,不保存所有 batch 或全部行號索引。更換檔案、目標或關鍵設定會使預檢失效;開始前再次讀取目標狀態。 + +- v1 支援嚴格 UTF-8(有/無 BOM)及有 BOM 的 UTF-16 LE/BE;拒絕無法判定或有無效序列的編碼,不能以替代字元默默執行。來源檔案不改寫。 解碼後的 U+0000 在 v1 SQL 文字中不支援,回報 EmbeddedNul;它可能來自沒有 BOM 的 UTF-16,也可能是合法 UTF-8 的 literal NUL,不誤稱必然是錯誤編碼。這是腳本文字相容邊界,既有 exporter 的資料值採 binary 表達,不新增欄位限制。 +- GO 不分大小寫,須位於字串、quoted identifier、bracket identifier 與註解之外的獨立一行。支援前後空白及 `GO -- 註解`;GO 不送給 SQL Server。v1 不支援 GO 重複次數與同一分隔行的 block comment,預掃描發現這類 GO 語法時明確拒絕。 +- lexer 跨 buffer 維持狀態,處理單引號/N 字串與 `''`、雙引號與 `""`、方括號與 `]]`、單行註解、巢狀 block comment、CRLF/LF 及多位元字元。字串/註解中的 GO 保留原文。行本身也不得以無上限 ReadLine 累積。 +- EOF 的最後一個非空批次照常執行;未閉合的字串、identifier 或 block comment 在預掃描擋下。空批次不送出,不用分號拆解任意 T-SQL。 +- `:r`、`:setvar`、`:connect`、`!!` 等 SQLCMD 指令在有效指令位置出現時拒絕;不展開 `$(name)`,字串中的相同文字維持原值。 + +**單一 GO 批次上限為解碼後 32 MiB,即 16,777,216 個 UTF-16 code unit。** 超過時於預掃描、任何 DB 寫入前停止,列出批次及行號。不能插入 GO 或用分號硬切來避開限制。這是執行批次的記憶體界限,不是欄位大小限制:數十 MiB 甚至更大的文字/binary/Geometry 仍可透過既有分段腳本跨批次匯入。 + +既有 exporter 的資料批次最多約 4 MiB UTF-8,LOB chunk 更小;schema/view 定義沒有同樣的通用上限,仍須實測預掃描。32 MiB 也不是整個程序的總記憶體承諾,StringBuilder、最終 string 及 SqlClient 可能同時持有多份;驗收要量測峰值及長時間記憶體是否隨檔案成長。 + +辨識目前 mySQLPunk 完整匯出檔時讀取 footer 的 job/tables/rows 及來源讀取模式;缺少或格式不完整的 native 完成標記不視為可完整還原的匯出檔。一般 T-SQL 不要求該 footer,但顯示「無匯出筆數摘要」。識別規則以既有 exporter 的結構及受控 marker 為準;註解與 footer 都不是簽章或安全認證。執行時再計算串流 digest 與預掃描比對,差異會令結果失敗,不能藉此回復已執行的 SQL。 + +## 4. 專用連線、目標與交易 + +每個作業經既有 `ConnectionOpenService`/owned session 開啟專用連線,以選定的 Initial Catalog 明確指定目標。複製必要的連線設定,不借用資料庫樹或查詢頁面的共用 provider;SSH owner 也隨本次 session 釋放。所有 GO 批次都使用這個 session,不自動重連或換線。 + +開始時記錄實際 server/instance、版本、DB_NAME、database_id、可寫狀態及目前登入可見的 metadata。確認頁與結果頁顯示解析後的實際目標;讀取失敗、目標消失/唯讀、身份不符或無法確認必要條件時不開始。沒有 sysadmin/ALTER DATABASE 並非一律拒絕的理由;依腳本範圍檢查基本建立/寫入能力,實際 SQL 權限錯誤仍需完整記錄。 + +辨識為包含建立結構的 mySQLPunk 匯出檔時,目標須無使用者資料表/檢視/程序等物件,且需有足夠 metadata 可見性來確認;資料庫既有 schema 本身不代表非空。資料型匯出與一般 T-SQL 不一律要求空庫。權限或語法預檢不是任意 SQL 能成功的證明,也不能靠試跑 SQL 檢查權限。 + +精靈不會自動建立、清空、覆寫資料庫,不切 recovery model、不開關 SNAPSHOT、不發 GRANT 或 KILL,也不會放寬外鍵/檢查約束。使用者選擇執行的 SQL 本身仍可變更資料;一般腳本依 SQL Server 原有權限執行,本功能不是 SQL sandbox。 + +v1 不接受獨立 T-SQL `USE` 切換至其他資料庫;lexer 能辨識這類直接語法並在預掃描擋下。三段名稱、動態 SQL 或 stored procedure 仍可能存取其他 DB,不能宣稱 target pinning 會攔住所有跨庫存取;執行前文案須要求使用者確認腳本來源與目標。每批完成再確認目前 database_id,發現改變就停止。 + +保留批次原文、順序及 session 的 SET、IDENTITY_INSERT、temp table、LOB staging 狀態。不中途插入全檔外層 transaction,也不改寫腳本自己的 transaction;允許一般腳本的交易跨 GO,但這種情況遇錯不能套用自動略過。 + +批次間為確認交易與目標而插入的狀態查詢可能改變 @@ROWCOUNT/@@ERROR;v1 不保證這兩個 statement intrinsics 跨 GO 保持原值,一般腳本預檢與文件需明示。需要保留時應在同一 batch 先自行存入變數/temp table;既有 exporter 腳本不依賴這兩值跨 GO。此相容邊界於實作計畫與覆核裁決中固定,並以測試驗證。 + +成功回應只代表這個 batch 已完成執行,不代表其中的交易已提交;失敗/取消批次也可能已部分寫入。先前已提交的變更不會因取消整份還原。自然 EOF 尚有未結交易時不能顯示成功;停止後嘗試回復本次專用 session 的未提交交易並釋放連線,回復失敗或結果不明須記錄。 + +## 5. 順序執行、大欄位與錯誤政策 + +worker 一次最多執行一個 SqlCommand,下一批在前一批結果已處理後才送出。CommandTimeout 預設 0(允許長指令),取消機制獨立可用;若未來加入有限 timeout,timeout 一律停止,不自動重試。產生結果集的一般 T-SQL 也要逐 result/逐 row 讀完並丟棄,不建 DataTable 或將整個結果載入畫面。 + +mySQLPunk LOB/Geometry 使用同一 job 的 `#mp__*` temp tables,維持原本順序/長度/Consumed/Failed 檢查,Geometry 依原腳本 Deserialize。不能清除失敗旗標、跳過 chunk 或重播已消耗 row 來假裝完成。 + +| 情況 | 行為 | +| --- | --- | +| 預設未勾略過,收到 SQL 錯誤 | 記完整 log,停止排程,結果為失敗 | +| 已勾略過,一般 SQL 錯誤符合下述條件 | 記完整 log,略過失敗的 GO batch,再送下一批;最後為「執行完成,但有錯誤」 | +| timeout、取消、斷線、嚴重錯誤、未知執行結果 | 停止;不假裝是可略過的普通錯誤 | +| parser/編碼/檔案 IO/log IO 錯誤 | 停止;不得在沒有可保存錯誤紀錄的狀態繼續 | +| 未結/不可提交/無法查明的交易,或 LOB 協定失敗 | 即使勾略過仍停止,保留可追查狀態 | + +一般 SQL 錯誤只有在以下條件全部成立時才可略過: + +1. 使用者已勾選;原連線仍開啟,未收到取消;錯誤屬一般 SQL 錯誤(class 11–16),不是 timeout/attention/傳輸錯誤或 native 協定錯誤。 +2. 本批執行前與錯誤後均能確認 `@@TRANCOUNT=0`、`XACT_STATE()=0`,且目前仍為選定 database_id;狀態查詢使用獨立 5 秒 timeout,失敗就停止。 +3. 該 batch 不涉及 native LOB 初始化/組裝/提交,且已辨識的 job 狀態沒有 Failed、未消耗 row 或不符預期的狀態;無法確認狀態就停止。 +4. 本批所有 SQL error detail 均已成功寫入並 flush 至 log。 + +以 provider 錯誤碼、class、交易狀態及受控協定識別判定,不以英文/中文 error message 字串決定恢復。native 批次識別與 temp identifier 必須經 lexer 及 GUID 格式驗證;不能把任意 footer 內容接成 SQL。初始化尚未發生與預期 state table 卻遺失須分開。 + +特別注意:LOB CATCH 會先 rollback 短交易,再把 temp state 的 Failed 設為 1。因此交易已回到 0 仍不表示可繼續,包含最後 INSERT 的轉型/約束失敗也要停止。無法安全辨識的 native 狀態採停止,不能默默降為一般略過。 + +「略過」是失敗 batch,不是單一 row;同批已生效的 SQL 不會自動撤回。成功數、失敗/略過數及當前結果不確定的批次分開,不能由 ExecuteNonQuery 的 affected rows 推算實際匯入筆數,因為 LOB staging 更新也會計入。 + +## 6. Error log 與 UI 記憶體界限 + +每次作業建立獨立 UTF-8 log,預設 `%LOCALAPPDATA%\mySQLPunk\ImportLogs\import--.log`。開始 DB 寫入前必須已成功建立並寫入作業表頭;檔名唯一且不覆寫既有 log。檔案路徑在確認、執行與結果頁可見。 + +表頭包含 job、來源路徑/大小/digest/編碼、實際 server/database、錯誤政策與開始時間。每個 SQL 錯誤保存時間、batch Int64 序號、來源行範圍與原始 byte 範圍、provider error number/state/class/相對與換算行號/message、採取的動作及狀態檢查結果。多個 SqlError 都要記錄,不只第一個。完整 log 指所有錯誤都有紀錄,不額外附整段 SQL、LOB payload、連線字串或密碼;SQL Server 原始錯誤訊息本身可能含資料值,紀錄維持本機使用者目錄。 + +error log 串流追加,每次要略過前 flush;錯誤不能累積到最後才存。job 結尾寫入結果、成功/失敗/未確認批次統計、最後確認位置、取消及 cleanup 結果。log 建立/追加/flush 失敗就停止,畫面保留 bounded 最新錯誤及存檔失敗原因,不宣稱完整 log 已保存。 + +textarea 唯讀,以 ring buffer 保留最近 100 筆或 64 KiB UTF-16 文字(先到者為準),單筆過長只截 UI。總錯誤數另以 Int64 累計;不要在記憶體保存所有錯誤/失敗批次或每批 log string。截段文案顯示「僅顯示最近 … 筆;共 … 筆,完整紀錄:」。開啟完整 log/資料夾/複製路徑直接作用於實際已建立的檔案;成功、帶錯誤、失敗與取消都保留紀錄。 + +## 7. 進度、ETA 與結果核對 + +預掃描與執行分開計算進度。執行進度以原始 bytes 為基礎,只在 command 已取得結果並完成處理後推進;不能用 read-ahead 的 FileStream.Position 當已匯入進度。成功覆蓋 bytes 與失敗/略過覆蓋 bytes 分開,勾略過時主標籤為「已處理檔案範圍」,不可把 100% 說成全部成功。 + +UI 約每 200 ms 合併更新,worker 不對每個 row/chunk 送 UI invoke。目前 batch 序號、來源行範圍與已確認邊界使用 Int64。只有能由 native 格式可靠辨識的表名/階段才顯示,否則顯示目前批次;不猜測任意 T-SQL 的物件清單。 + +ETA 採最近約 10 秒已確認處理 bytes 的速率,至少 3 次樣本、2 秒觀察後才顯示「約 …/預計 …」。5 秒無新確認、長指令、索引/約束階段或波動太大時改顯示「計算中/目前指令執行中」,不倒數至負數、不把讀完檔案等同工作結束。建立索引、外鍵與結果核對完成前仍為執行中;預掃描、執行及結果核對各有狀態。 + +所有跑到 EOF 的作業(包含已略過錯誤者)都必須檢查 session/交易與檔案 digest。所有已辨識的 native 模式(含 DataOnly)若曾初始化 LOB,結尾須確認預期 state table 存在且只有一筆、Failed=0、Consumed=1、Sequence=ConsumedThrough;未初始化時不要求 state table。任何這類檢查失敗都進入 Failed/CleanupUncertain,不能以 CompletedWithErrors 蓋過未結交易、未消耗 LOB 或 digest 不符。 + +摘要筆數核對才限定無錯誤的 native 完整結構匯出:在原本空的目標列出實際 user table 數並逐表 COUNT_BIG,對照 footer 表數/總筆數;這個核對步驟可取消,且要單獨顯示進度。資料型匯出或一般 T-SQL 沒有足以進行同樣比較的 manifest,無錯誤結果只聲明「所有批次執行完成,未提供資料內容核對」;已略過者維持帶錯誤結果與明確未通過完整核對的說明。 + +footer 是預期摘要,COUNT_BIG 比對也不是逐欄值/來源 DB 雜湊證明;不連正式來源庫做比較。來源 READ COMMITTED 的跨表不一致警告仍保留;因此最後外鍵建立可能失敗,不能自動改成 NOCHECK 或刪資料。帶錯誤時不給「完整還原通過」標示。 + +| 結果 | 顯示與證據 | +| --- | --- | +| Completed | 所有批次完成、無未結交易、必要檢查通過;明示實際完成了哪些核對 | +| CompletedWithErrors | 已遍歷整份檔案,但有失敗/略過;琥珀色,完整 log 路徑與統計 | +| Failed | 停止原因、最後確認批次、當前失敗/不確定批次與 cleanup 結果 | +| Cancelled | 已停止排程、command 已結束、session 已釋放;明示先前寫入可能保留 | +| CleanupUncertain | 無法確認當前結果/釋放狀態時明示仍有不確定性,不冒稱安全取消或還原完成 | + +## 8. 取消與生命週期 + +按取消後停止排程並顯示「正在停止匯入…」,向本次 SqlCommand 發出 Cancel;取消不是保證立即中斷。等待真正回應期間 UI 保持可回應,不能先把結果改成已取消,也不能開始另一輪匯入。 + +command 回應後,用本次 session 進行必要的未提交交易 cleanup(獨立 10 秒 timeout),釋放 command/reader/session/SSH owner/file handle 並完成 log。若 Cancel 10 秒內未得到回應,嘗試關閉本次擁有的連線;仍無法確認時進入 CleanupUncertain,保留工作識別與最後位置。不能 KILL 其他 session、在仍使用中的 worker 上二次執行,或以自動重連掩蓋狀態。 + +視窗關閉時若仍在跑,走同一取消流程並避免 callback 更新已 Dispose UI。沿用 export wizard 的 job generation/owned-resource 模式。取消可能等待伺服器 rollback;已提交部分不會自動撤銷。結束後「重新檢查匯入」回到預檢,不直接重跑失敗 batch,不自動刪除部分匯入物件。 + +## 9. 程式邊界 + +沿用 .NET Framework 4.7.2、WinForms、System.Data.SqlClient 及專案 MSBuild 工作流,不新增 WebView 或外部 sqlcmd 執行依賴。 + +- `Form1.MssqlImport.cs`:集中 MSSQL 檔案入口、明確目標與精靈啟動,既有其他引擎路由不變。 +- `MssqlImportWizardForm`:四步狀態、Punky、確認/取消/結果/bounded log UI。 +- `lib/MssqlImport`:分離 request/result/progress 模型、streaming lexer/preflight、owned-session sequential runner、error log 及 ETA estimator;核心流程可獨立測試。 +- 共用已有的連線擁有權與 Punky 控制,不擴大 `IDatabase` 公開契約,也不把新 reader 塞進舊 MySQL splitter。 + +以上是責任邊界,具體工作順序與測試安排於規格核准後列入實作計畫。 + +## 10. 驗收條件與目前證據 + +### 實作後必須取得的證據 + +1. Lexer:每種字串/identifier/註解內 GO、巢狀 comment、跨 buffer/UTF-8/UTF-16/CRLF、超大單行、EOF、GO n/SQLCMD 拒絕;預掃描錯誤時 DB 寫入數為零。 +2. 有界資源:相同批次形狀的 1 倍/10 倍大檔,無整檔 bytes/string/AST/batch list;含 32 MiB 邊界與大批次拒絕,峰值記憶體受批次界限控制且不隨檔案長度線性增加。 +3. 同 session:temp table、SET、IDENTITY_INSERT、跨 GO transaction;64 MiB binary、18 MiB Unicode、Geometry 分段匯入後以來源測試資料的 bytes/hash/SRID/NULL 及 metadata 比對。 +4. 錯誤:預設停止、勾選後一般錯誤繼續;交易內錯誤、XACT_ABORT、斷線、timeout、log IO、LOB chunk 遺失/重複/亂序、Geometry/XML 轉型及 final INSERT 約束失敗仍停止。當前部分寫入及未知結果不得被標為成功。 +5. Log/進度:所有 SqlError 落檔、UI 截段/完整路徑、故障 flush、byte 與批次 Int64、read-ahead 不提前算成功、ETA 暖機/長指令/索引/核對階段、取消與關閉生命週期。 +6. WinForms 與既有 smoke/Debug/Release build;測試執行檔依專案既有 smoke 入口,不重建先前被防毒隔離的 ReportExportTests.exe。 +7. 最後才對使用者授權的本機 SQL Server 2019/Transfer 執行實際 4.17 GiB 檔案試跑,保留耗時、峰值記憶體、log、物件/筆數及約束核對結果;失敗部分保留,清除/重跑前先呈現具體狀態,不自動破壞使用者 DB。 + +### 本次已取得的證據 + +- 提供檔案為 `C:\Users\johnho\Desktop\Transfer_dump_20261013_165730.sql`,4,478,414,151 bytes,UTF-8 無 BOM/CRLF;僅 bounded 首尾檢查,footer 預期 126 tables/3,247,785 rows,來源 READ COMMITTED。尚未完整掃描/執行此檔。 +- 本次研究時以 Windows 整合登入唯讀確認 `localhost` 解析為 `JOHN\MSSQL2019`,版本 15.0.2000.5,Transfer compatibility 150、ONLINE/READ_WRITE/FULL,當時 user tables/views/procedures 均為 0;具有 VIEW DEFINITION/CREATE TABLE/ALTER 基本權限。執行前須重新檢查,這不是未來狀態保證。 +- 目標 data/log 當時各 8 MiB、64 MiB 固定成長且位於 C 槽;目前 C 槽約 152.7 GiB 可用,D 槽約 2 GiB。來源位於 C,log 預設也位於 C;SQL 文字檔大小不能直接當作所需 DB/transaction log 空間,實際匯入須觀察增長。 +- 瀏覽器原型已驗證四步導覽、超大 batch 阻擋、預設不略過、勾選後模擬 SQL 547 繼續到帶錯誤完成、未勾時停止、1,283 筆只顯示最近 100、唯讀 textarea、log 示意操作、取消先停止中後取消、深淺色、640 px 無水平溢位與靜音切換。console warning/error 為 0。 +- 尚未實作 native importer、實際 log 落檔/DB 寫入、完整大檔還原、native UI/實際音訊聽感驗收;上述畫面證據不能取代這些驗收。 + +## 11. 主要技術依據 + +- [SQL Server GO](https://learn.microsoft.com/en-us/sql/t-sql/language-elements/sql-server-utilities-statements-go?view=sql-server-ver15):GO 是 client batch delimiter,不是可直接送給 SqlCommand 的 T-SQL;本版支援範圍另按本規格明訂。 +- [SqlCommand.Cancel](https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlcommand.cancel?view=netframework-4.8.1):取消是嘗試取消執行,不可把呼叫成功當作伺服器已停止。 +- [TRY…CATCH](https://learn.microsoft.com/en-us/sql/t-sql/language-elements/try-catch-transact-sql?view=sql-server-ver15):client attention/broken connection 不保證由 T-SQL CATCH 處理,cleanup 不可只靠腳本 CATCH。 +- 專案來源:`Form1.cs` 的 ImportSqlWithDialog/ImportSqlScript/SplitSqlScript;`MssqlLobScriptWriter` 的 Guarded/Consumed/Failed 協定;`MssqlSchemaScriptWriter`、`MssqlExportArtifact`、`ConnectionOpenService`、`MssqlOwnedSession` 與 export wizard/smoke tests。 diff --git a/docs/design/mssql-import/implementation-plan.md b/docs/design/mssql-import/implementation-plan.md new file mode 100644 index 0000000..293a002 --- /dev/null +++ b/docs/design/mssql-import/implementation-plan.md @@ -0,0 +1,184 @@ +# MSSQL SQL Import Wizard Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** 以專用精靈安全且有界地執行 mySQLPunk 大型 MSSQL SQL 檔及一般 T-SQL/GO,提供 Punky、錯誤紀錄、取消、可信進度與 ETA。 + +**Architecture:** 同一唯讀檔案 handle 先完整掃描,再由單一 owned SqlClient session 依序執行 GO batches。reader、native 格式辨識、session adapter、worker、log/ETA 與 WinForms 各有獨立責任,保留既有 exporter 的 LOB/Geometry 協定。 + +**Tech Stack:** .NET Framework 4.7.2、WinForms、System.Data.SqlClient、既有 MSBuild/smoke runner;不新增套件、WebView 或外部 sqlcmd 執行依賴。 + +**Spec:** [design.md](design.md),使用者於 2026-10-13 回覆「好,進實作」核准書面規格。 + +**Status:** Task1–7 的實作、逐項與整體覆核、建置及授權試跑已執行。隔離 DB 驗證通過;SQL2019 使用者實檔於第1481批 SQL241停止,獨立核對僅126表/46筆,完整還原驗收未通過。舊匯出器日期格式已修正並通過小型 Core 還原;原檔、部分 Transfer、marker與log保留,未清庫或重跑。進度與裁決存於 `.superpowers/sdd/implementation-plan/progress.md`。 + +**Execution:** 保留先前選定的「分工實作並逐項覆核」。一次一位 implementer,各 task 經獨立覆核後才接下一項;最後整體覆核。build、native UI 與 DB tests 由單一 owner 串行執行,避免共享輸出/fixture 衝突。 + +## Global Constraints + +- 保留既有編碼/BOM/換行;不碰 unrelated `mySQLPunk.Desktop/Properties/`,不讀取帳號 token/瀏覽器 session/已存連線或登入設定。 +- 只替換 MSSQL 的檔案執行入口;其他 DB 引擎及查詢編輯器不改,`IDatabase` 公開契約不擴張。 +- 單一 GO 批次上限為解碼後 32 MiB,即 16,777,216 個 UTF-16 code unit;不是欄位大小上限。不能插 GO/依分號硬切。禁止整檔 bytes/string/AST/batch list。 +- 嚴格 UTF-8 有/無 BOM、UTF-16 LE/BE 有 BOM;解碼後 U+0000 回報 EmbeddedNul,明示 v1 literal-NUL 相容邊界,不能誤稱 UTF-8 NUL 一定是無效編碼;保留原文及同一 session。支援獨立 GO/GO -- 註解,拒絕 GO n、同分隔行 block comment、SQLCMD 指令與跨 DB 的直接 USE。 +- 預設不略過錯誤;略過對象為失敗 GO batch,不是單筆 row。timeout/斷線/log IO/未知交易/native LOB 失敗仍停止,不重連、重試、重播或續傳。 +- 專用連線明確指定 Initial Catalog;ConnectRetryCount=0,單一 command,CommandTimeout=0。狀態查詢 5 秒、cleanup 10 秒,取消 10 秒無回應後只嘗試關閉本次 owned connection。 +- 不新增全檔外層 transaction、不自動清庫/DROP 使用者物件、不改 SNAPSHOT/recovery model/constraint trust、不 GRANT/KILL。 隔離驗收例外僅限新建 GUID-owned LocalDB 測試庫:WITHOUT LOGIN fixture users/custom roles 可作 GRANT/DENY 與 readonly/recreate target 測試;不擴張至 server login、master/msdb、Transfer 或既有真人帳號,清除仍須雙重 ownership/路徑驗證。 +- 所有走到 EOF 的作業都做 session/交易/digest/已初始化 LOB 終態檢查,包括已略過錯誤者。native LOB 結尾要求唯一 state row、Failed=0、Consumed=1、Sequence=ConsumedThrough。 +- error log 預設 `%LOCALAPPDATA%\mySQLPunk\ImportLogs\import--.log`;略過前落檔並 flush。UI 最近 100 筆或 64 KiB UTF-16 文字,先到者為準,完整路徑可見。 +- 進度與 batch/line/byte/error 統計為 Int64;command/result drain/狀態處理完成才更新確認邊界。affected rows 不能標為實際匯入筆數。 +- UI 約 200 ms 合併更新;ETA 約 10 秒窗口、至少 3 次樣本與 2 秒觀察,5 秒未確認或長指令/索引/核對中改為未知。樣本最多64筆、200ms合併,正速率CV>1也回未知;latest counts與lastAdvance不得被節流丟失。 +- Punky 自動 loop,聲音首輪一次、可靜音,沒有播放/暫停按鈕。取消、關閉與 late callback 沿用 job generation/owned resource 模式。 +- 測試用既有 `mySQLPunk.SmokeTests.exe`,不執行/重建 ReportExportTests.exe,不調整防毒,不 push/merge/release。 +- 大型 SQL、DB files、logs、manifest 與原始驗收證據只存 C 槽 local-only,不進 Git。使用者實檔/Transfer 不受 fixture 自動清除權限涵蓋。 + +## Review Focus + +1. 批次間安全查詢會改變 `@@ROWCOUNT`/`@@ERROR`:Task3/4 明訂相容邊界與跨 GO 測試,不能宣稱保存所有 session intrinsics。 +2. 一般 SELECT 回傳數百萬列或 64 MiB LOB:Task3/6 逐結果串流 drain,可取消,未 drain 完不能確認 batch。 +3. 同批先寫成功再出錯:Task4/6 同時驗證 stop/skip,保留部分寫入說明與所有 SqlError,不虛構 row 成功數。 +4. 取消與成功、狀態查詢、COUNT_BIG、log flush 或 form Dispose 競爭:Task4/5/6 不提前宣告取消、不釋放別人的 session、不啟動重疊工作。 +5. native DataOnly/偽造或缺失 header/footer/job/失敗但交易為 0:Task1/3/4/6 保守分類、驗證 GUID/終態,不默默改為 generic skip 或完整還原通過。 + +## Files and shared contracts + +所有新型別 internal,位於 `mySQLPunk.lib.MssqlImport`;下列介面是 tasks 交接契約。具體小型 DTO constructor 可採唯讀 properties,但名稱/單位固定。 + +| 檔案 | 責任/主要型別 | +| --- | --- | +| `mySQLPunk/lib/MssqlImport/MssqlImportModels.cs` | Request/batch/scan/progress/result/error/target/session-state immutable DTO 與 enums | +| `MssqlImportInput.cs`, `MssqlGoBatchReader.cs` | 唯讀 handle、strict decoding/SHA、bounded lexical GO reader | +| `MssqlImportScriptInspector.cs` | native 格式/job/batch role、unsupported 指令與直接 USE 的 lexical 檢查 | +| `MssqlImportLog.cs`, `MssqlImportProgressEstimator.cs` | 完整串流 log、bounded tail、速率/ETA | +| `MssqlImportSession.cs` | owned SqlClient adapter、metadata/transaction/LOB 狀態、result drain、取消/cleanup | +| `MssqlImportService.cs` | 預檢、順序 worker、skip 政策、EOF 驗證與結果組裝 | +| `mySQLPunk/MssqlImportWizardForm.cs`, `mySQLPunk/Form1.MssqlImport.cs` | 四步 UI、MSSQL 入口/目標 settings snapshot | +| `tests/SmokeTests.MssqlImport.{Parser,Worker,Integration,Wizard}.cs` | 沿用現有 partial smoke 類別的測試 | +| `tests/Run-MssqlImportIntegration.ps1`, `tests/Run-MssqlImportTransferTrial.ps1` | 隔離 fixture/實際 Transfer 試跑,兩者的授權與 cleanup 完全分開 | + +表格中未寫完整路徑的 backend 檔案皆位於 `mySQLPunk/lib/MssqlImport/`。 + +- `MssqlImportRequest(Guid jobId, string sourcePath, string databaseName, bool skipSqlErrors = false)`:不含帳密;`JobId/SourcePath/DatabaseName/SkipSqlErrors`。 +- `MssqlImportBatch`:`long Number/StartByte/EndByte/StartLine/EndLine`、`string Text`;byte 區間為原始檔案的半開區間,不能以解碼字元數代替。 +- `MssqlImportInputException : Exception`:固定 `string Code` 與 `long BatchNumber/Line/ByteOffset`,讓預掃描失敗有穩定診斷。 +- `MssqlImportScan`:`long FileBytes/BatchCount/MaxBatchCodeUnits`、`string EncodingName/Sha256`、`MssqlImportScriptInfo Script`;不保存所有批次。 +- `MssqlImportScriptInfo`:`Kind { Generic, NativeSchema, NativeData }`、`Guid? NativeJobId`、`long? ExpectedTables/ExpectedRows`、`string SourceReadMode`、`bool HasLobProtocol`;native marker 候選毀損是預檢失敗,不降級 generic。 +- `MssqlImportInput.Open(string path, string databaseName = null): MssqlImportInput`;`Scan(CancellationToken, Action): MssqlImportScan`;`ReadBatches(CancellationToken): IEnumerable`;執行第二輪可取得 `ExecutionSha256`,物件為 IDisposable 並持有同一 handle。 +- `MssqlImportSqlError`:`int Number/State/Class/RelativeLine`、`string Message`;`MssqlImportBatchException` 保留本批全部 `IReadOnlyList Errors` 及 `bool OutcomeUncertain`,原始 exception 不送 UI。 +- `MssqlImportSessionState`:`bool IsOpen`、`int DatabaseId/TransactionCount/XactState`;target identity 另含實際 ServerName/DatabaseName/DatabaseId/DatabaseCreateDate/Version/Writable/metadata 可見性/user object count。 +- `MssqlImportLobState`:`bool Exists`、`long RowCount/Sequence/ConsumedThrough`、`bool Consumed/Failed`;native identifier 僅能由驗證過的 GUID 產生。 +- `IMssqlImportSession : IDisposable`:`ReadTarget(CancellationToken): MssqlImportTarget`、`ReadState(CancellationToken): MssqlImportSessionState`、`ReadLobState(Guid, CancellationToken): MssqlImportLobState`、`ExecuteBatch(MssqlImportBatch, CancellationToken): void`、`CountTargetRows(Action, CancellationToken): MssqlImportCounts`、`CancelActiveCommand(): void`、`Cleanup(): MssqlImportCleanup`。同步方法只由 background worker 呼叫;adapter 管理實際 async reader/Cancel race。 +- `MssqlImportPreflight`:持有 Input/Scan/Target/LogPath/Passed/固定 Code 的 Issues;IDisposable,重選與關閉時釋放 input 及未用的 job log;Run 原子領取一次 execution ownership,重用同一預檢拒絕,重複 Dispose 安全。 +- `MssqlImportService(Func openSession, Func openLog)`;`Preflight(MssqlImportRequest, Action, CancellationToken): MssqlImportPreflight`;`Run(MssqlImportRequest, MssqlImportPreflight, Action, CancellationToken): MssqlImportResult`。Run 再核對 target;factory 每作業開一次專用 execution session。 +- `IMssqlImportLog : IDisposable`:`string Path`、`WriteStart(request,scan,target)`、`WriteError(batch,error,action)`、`Flush()`、`WriteEnd(result)`;failure 直接向 worker 冒泡,不能吞掉後繼續。`MssqlImportLogTail.Add(string)`/`Snapshot(): string`、`long TotalErrors`。 +- `MssqlImportProgress`:`Guid JobId`、`Stage { Scanning, Executing, Verifying, Stopping, Finished }`、`long FileBytes/ScannedBytes/ProcessedBytes/SuccessfulBytes/SkippedBytes/SuccessfulBatches/FailedBatches`、`long BatchNumber/StartLine/EndLine/StartByte/EndByte`(無目前批次為0,不能帶Batch.Text進UI)/固定 Phase/可選 TableName、TimeSpan Elapsed/TimeSpan? Remaining/DateTime? EstimatedFinish;bounded string RecentErrorText/LogPath、long TotalErrors、bool LogTruncated。`MssqlImportProgressEstimator.Observe(progress, DateTime utcNow)`/`Estimate(DateTime utcNow): MssqlImportEstimate`、`ShouldReport(MssqlImportProgress, DateTime utcNow): bool`。Estimate含TimeSpan? Remaining/DateTime? EstimatedFinish(UTC)/bool IsReliable;phase固定None/Schema/Data/Indexes/Constraints,StageVerifying/Stopping/Finished不估。 +- `MssqlImportResult`:`Status { Completed, CompletedWithErrors, Failed, Cancelled, CleanupUncertain }`、統計/LastAcknowledgedBatch/UncertainBatch/LogPath/Cleanup/ValidationSummary/TimeSpan Elapsed;`MssqlImportCounts(long tables,long rows)`。`MssqlImportCleanup` 記錄是否確認 command 結束、session 釋放、未提交交易 cleanup 與固定 issue code,不把 Dispose 呼叫等同伺服器 rollback 已完成。 + +## Planning decisions that must remain explicit + +- 現有 native header 沒有 mode/job manifest:使用 exporter 固定 Source read mode/空庫提示、實際 lexer token 及 footer 共同分類;LOB `#mp__s/_v/_c` 必須與 footer job 一致。不新增 exporter 格式要求讓使用者現有 4.17 GiB 檔案失效。 +- 現有 `OpenMssqlSession(settings)` 已開線,不能事後補 ConnectRetryCount=0。Task3 新增 importer 專用 pre-open option;其他 callers 維持原連線行為。 +- 一般腳本可跨 GO 保留 temp tables/SET/IDENTITY_INSERT/transaction;為了偵測交易與目標變化,插入的狀態查詢可能改變 `@@ROWCOUNT`/`@@ERROR`。v1 不保證這兩個 statement intrinsics 跨 GO 保持原值,文件與 generic-script 預檢說明需明示;這不影響 exporter 已產生的腳本。需要在同一 batch 保存這些值的腳本應自行先存入變數/temp table。語義依據:Microsoft [@@ROWCOUNT](https://learn.microsoft.com/en-us/sql/t-sql/functions/rowcount-transact-sql?view=sql-server-ver15) 與 [@@ERROR](https://learn.microsoft.com/en-us/sql/t-sql/functions/error-transact-sql?view=sql-server-ver15)。 +- 現有 export integration runner 硬編碼的 ignored fixture manifest 不在目前 master。新 runner 採明確 `-AuthorizationPath`,驗證既有已授權專用 LocalDB 名稱/C 槽 fixture root/owner token;不把缺檔當已授權成功,也不去讀 Codex 帳號設定找憑證。 + +## Task 1: Bounded GO reader、檔案身份與 native 格式預掃描 + +**Files:** Create `MssqlImportModels.cs`, `MssqlImportInput.cs`, `MssqlGoBatchReader.cs`, `MssqlImportScriptInspector.cs`, `tests/SmokeTests.MssqlImport.Parser.cs`。Modify `mySQLPunk/mySQLPunk.csproj`, `tests/SmokeTests.cs`, `tests/Run-SmokeTests.ps1` 的 explicit source registration。 + +**Interfaces:** 產出上述 Batch/Scan/ScriptInfo/Input;inspector `Observe(MssqlImportBatch)`、`Complete(): MssqlImportScriptInfo`,保留有限 metadata,不保存完整 batches。其餘 models 在使用它的 task 補齊,不提前加入未使用的空架構。 + +- [x] **Step 1: 加入並跑失敗的 parser tests。** `MssqlImportParserPreservesContexts` 驗證字串/雙引號/方括號/巢狀 comment 中 GO 不拆;1/2/3/7-byte buffer 切在 UTF-8、BOM、CRLF、`''`、`]]` 中間仍原文相同。`MssqlImportParserLimits` 的斷言:16,777,216 code unit 的 Scan 成功且 MaxBatchCodeUnits 相等;16,777,217 時拋 MssqlImportInputException、Code="BatchTooLarge"。 +- [x] **Step 2: 實作固定 byte/decoder buffer、bounded current batch 與 lexical line candidate。** 不用無界 ReadLine、ReadAllText/ReadAllBytes/ReadToEnd;strict decoder 的 EOF flush 要抓無效序列。明訂 EndByte 包含目前 batch 的 GO delimiter,但 SQL Text 不含 GO。純註解/空白尾段不送空 command,其 byte 覆蓋只在 EOF 驗證通過後才計入最終 ProcessedBytes=FileBytes。 +- [x] **Step 3: 補 unsupported/native tests 並實作 inspector。** GO n、GO block comment、SQLCMD 真指令、跨庫直接 USE、未閉合 EOF、無效 encoding 拒絕;字串中的相同文字不拒絕。同 DB USE 在傳入 databaseName 且 decoded identifier 完全 Ordinal 相等時可接受;unknown target 或不同大小寫也保守拒絕,避免 case-sensitive server 的跨庫同批寫入。native Schema/DataOnly/無 LOB、正確/缺失/重複/溢位 footer、錯 job、偽造 GUID、header 不一致都要測;native candidate 不可因缺 footer 轉 generic。合法 Generic 不要求 footer。 +- [x] **Step 4: 以同一 handle 掃描→rewind→執行第二輪,測 SHA 一致與 FileShare.Read 阻止一般寫入/刪除;檔案重選 Dispose 後可寫。** 原始 byte/line offset、無 GO 的最後批次、僅註解檔不產生可執行空 command;大檔統計使用 Int64。 +- [x] **Step 5: 驗證並提交精確檔案。** `MYSQLPUNK_SMOKE_FILTER='MSSQL import parser'` 跑 `tests/Run-SmokeTests.ps1 -Configuration Debug`,必須 matched tests 全過;`git diff --check` 通過。Commit `feat: add bounded MSSQL GO input preflight`。 + +## Task 2: Durable error log、bounded tail 與進度/ETA + +**Files:** Create `MssqlImportLog.cs`, `MssqlImportProgressEstimator.cs`, `tests/SmokeTests.MssqlImport.Worker.cs`;補齊 Models 的 error/progress/result 與 log 實際消費的 Request/Target/Cleanup contracts,註冊 sources。 + +**Interfaces:** 產出 IMssqlImportLog/Tail/Estimator 及 terminal DTO;log 可注入 Stream factory 與 clock,測 IO 失敗不需修改檔案權限。Estimator 僅消費已確認 progress,不推測 affected rows。 + +- [x] **Step 1: 加入失敗 tests。** `MssqlImportLogRetainsAllErrors` 寫入 1,283 個多 SqlError detail 的事件,磁碟都有、UI ≤100筆且 ≤32,768 chars;單筆 >64 KiB 不截完整 log。唯一檔名 CreateNew、UTF-8、包含 batch/line/byte/codes/path/job,沒有整個 SQL 或連線字串。 +- [x] **Step 2: 實作串流 log 與 tail。** Start 必須在 DB 寫入前 flush;每次準備 skip 前 flush;create/append/flush/end/Dispose fault 均可被 worker 看見。TotalErrors 不隨 UI 淘汰減少;不累積所有失敗批次,log 存開啟/資料夾/複製可用的真實路徑。 +- [x] **Step 3: 加入並實作 ETA tests。** 暖機少於 3樣本或2秒不估;約10秒窗口;5秒停滯/Indexes/Verifying 回未知;有 skip 時 Processed=Successful+Skipped、成功比例獨立;read-ahead 和執行中 batch 不算確認。測 Int64 >2 GiB/高 batch 序號、clock injected、200 ms throttle 不漏最終狀態。 +- [x] **Step 4: 驗證 fault assertions。** Task2驗證Flush失敗實際向caller拋出;`FlushFailsBeforeSkip` 的 `NextBatchCalls==0` 排程斷言移交Task4真實worker,不能用test-only假worker驗證自己;密集錯誤10倍數量下 tail 記憶體固定;UI 文案顯示完整路徑且不冒稱所有資料成功。 +- [x] **Step 5: 跑 worker filter、提交。** `MYSQLPUNK_SMOKE_FILTER='MSSQL import worker'`、Debug smoke 全 matched tests 通過。Commit `feat: add MSSQL import logging and progress estimates`。 + +## Task 3: Import-only owned session、target preflight 與 bounded result drain + +**Files:** Create `MssqlImportSession.cs`,補齊 Models 的 target/session/LOB/counts/cleanup;Modify `mySQLPunk/lib/ConnectionConfigurationService.cs`, `mySQLPunk/lib/ConnectionOpenService.cs` 的 importer-only overload;tests Worker/Parser 與新 `tests/SmokeTests.MssqlImport.Integration.cs`。 + +**Interfaces:** `ConnectionOpenService.OpenMssqlImportSession(Dictionary): MssqlOwnedSession`,在開線前套 `ConnectRetryCount=0`、`MultipleActiveResultSets=false`;原 `OpenMssqlSession` 行為維持。`MssqlImportSqlSession(MssqlOwnedSession)` implements IMssqlImportSession,owner 只釋放一次。 + +- [x] **Step 1: 加入失敗 tests。** importer builder 明確 Initial Catalog/retry0/MARS false;舊 caller 既有值不變。使用 fake owner 驗證取消、初始化失敗、Dispose race 只有本次 session/SSH lease 被釋放,樹 provider 不受影響。 +- [x] **Step 2: 實作唯讀 target/state/LOB probes。** server/DB id/create_date/版本/writable/metadata 可見性/object count;必要能力不以 sysadmin 或 ALTER DATABASE 一票否決。NativeSchema 非空/無法確認空庫擋下;Generic/NativeData 不套空庫要求。所有 identifiers quote,LOB temp name 只由 Guid.ToString("N") 組成;state query 5秒,lookup 不能讀任意帳號設定。 +- [x] **Step 3: 實作原文 ExecuteReader 與所有 result drain。** `CommandBehavior.SequentialAccess`,涵蓋 ExecuteReader/Read/NextResult/reader Dispose 的全部 SqlError;不建 DataTable/回傳資料集合。CommandTimeout0,token+Cancel 包住啟動與後續讀取,保留目前 command 的可靠 ownership。 +- [x] **Step 4: 實作及測 cancellation/cleanup seam。** 不在 active reader 尚在跑時發第二條 status query;cleanup 獨立10秒,不使用已取消 token;關閉本 owner 不宣稱 rollback 已完成。一般 GO 跨 session 測 temp/SET/identity/transaction;另測 @@ROWCOUNT/@@ERROR 邊界並更新 generic-script 說明,避免不實相容宣稱。 +- [x] **Step 5: 跑 worker/parser filters,提交。** native live cases 暫列 Task6 執行,不能以 fake 通過冒稱 DB 通過。Commit `feat: add isolated MSSQL import session adapter`。 + +## Task 4: Sequential worker、recoverable skip、EOF checks 與取消結果 + +**Files:** Create `MssqlImportService.cs`;補 Worker tests;Models/Input/Log 僅做必要整合,不改 exporter 協定。 + +**Interfaces:** 完成 Service.Preflight/Run;Preflight 保留 input/scan/target identity/已建立 log;Run 在新專用 execution session 重查相同 server/DB id/create_date 與能力,所有 batches 同線。 + +- [x] **Step 1: 先寫失敗的 fake-session tests。** `DefaultStopsAtFirstError`、`SkipContinuesOnlySafeBatch`:未勾後續0calls;勾選一般class11–16、前後交易0/XactState0、same DB、無LOB pending、log flush成功才下一批。任何 SqlError 嚴重/timeout/傳輸/不明 outcome 不可 skip,不比對 message 語言。`FlushFailsBeforeSkip` 注入真實log flush fault,斷言 `NextBatchCalls==0`。 +- [x] **Step 2: 實作預檢→確認→執行流程。** 所有 input encoding/batch/native/目標/log 可寫性問題於第一個 DB write 前阻擋,parser失敗時execution factory calls=0;不同檔案/target/skip設定令預檢失效,ReusedPreflightIsRejected 測試第二次 Run 不送 SQL。一次一批,結果與 state 查明後才確認 bytes;目前batch可能部分生效,分開成功、失敗與未知批次。 +- [x] **Step 3: 實作 native batch role/state gate 及 EOF。** native LOB initialization/assemble/commit 任一 error fatal,即使 XACT_STATE0;普通batch遇錯也查已初始化 native state。每個走到 EOF 的作業都查交易、identity、digest、LOB唯一state/Failed0/Consumed1/Sequence=ConsumedThrough,包含 skip/DataOnly;pending/missing/wrongjob 不得 amber完成。無錯 NativeSchema 才逐表 COUNT_BIG 對 footer tables/rows,核對可取消。 +- [x] **Step 4: 實作取消/cleanup terminal tests。** 先 Stopping、不排下一批;Cancel10秒無回應只關本 session;cleanup10秒限時。自然 EOF 未結交易、取消在成功回應邊界、狀態查詢/COUNT_BIG/log flush 中取消、cleanup未確認都測。已失敗的原始結果不能被 cleanup exception 覆蓋;若最後 log失敗要顯示存檔未完整。 +- [x] **Step 5: 跑 worker filter、提交。** `PartialBatchEffectsAreNotReportedAsRows`、`SkippedRunStillChecksEofState`、`NativeLobPendingAtEofFails`、`NoReconnectOrReplay` 均通過。Commit `feat: execute MSSQL import batches with explicit outcomes`。 + +## Task 5: Four-step native wizard、Punky 與 MSSQL routing + +**Files:** Create `mySQLPunk/MssqlImportWizardForm.cs`, `mySQLPunk/Form1.MssqlImport.cs`, `tests/SmokeTests.MssqlImport.Wizard.cs`;Modify Form1.cs 的 ImportSqlWithDialog、csproj/smoke source registrations;只必要時局部擴充既有 theme helper。 + +**Interfaces:** `MssqlImportWizardForm(MssqlImportRequest initialRequest, Func serviceFactory)`;Form1 `OpenMssqlImportWizard(TreeDatabaseTarget)`。沿用 PunkyReportLoadingControl.BeginExport/EndExport/SetRunning/Muted;routing 測試可注入 wizard runner,不需真連線。 + +- [x] **Step 1: 先寫失敗 wizard tests。** 四步導覽、未預檢/有阻擋不能開始、確認target必勾、skip預設false;重選檔案/目標會失效舊預檢;關閉釋放 input。所有檔案入口在 MSSQL 先路由新 wizard,其他引擎與查詢編輯器維持既有呼叫。 +- [x] **Step 2: 實作符合核准原型的 WinForms layout。** 顯示實際 server/db、檔案size/編碼/native摘要、READ COMMITTED警告、一般腳本intrinsics邊界與部分寫入說明;結果明示實際核對範圍。log textarea ReadOnly,最近100/64KiB與總數/fullpath,open/folder/copy操作真實檔案。 +- [x] **Step 3: 接 background worker 與單一 cancellation owner。** jobId+generation 擋 late callback;timer約200ms拉取bounded snapshot;忙碌時不重入、FormClosing轉取消,收尾後才關窗;Disposed UI 不被 callback更新。Punky預設loop、first-cycle音效/靜音、無播放/暫停按鈕。 +- [x] **Step 4: 執行 native UI tests。** STA+WindowsFormsSynchronizationContext、form.Show/PerformClick/bounded DoEvents;深淺色、高 DPI、縮放後取消可到達、error截段、停止中與終態、close/complete race。若使用 preview,只新增 `MYSQLPUNK_MSSQL_IMPORT_WIZARD_PREVIEW/THEME` 且不開真DB,不能混用 export preview knob。 +- [x] **Step 5: 跑 wizard filter及路由回歸、提交。** 把 native screenshots 存 `docs/design/mssql-import/snapshot/`,與原型JPEG分開命名;記錄動畫視覺/聲音哪些是實際人工確認。Commit `feat: add MSSQL SQL import wizard`。 + +## Task 6: Authorized isolated DB roundtrip 與 fault acceptance + +**Files:** Complete `tests/SmokeTests.MssqlImport.Integration.cs`,Create `tests/Run-MssqlImportIntegration.ps1` 及必要小型 `tests/MssqlImportFixtures/` SQL;記錄 `docs/design/mssql-import/verification.md`。大型生成檔與 evidence 不進 Git。 + +**Interfaces:** runner 參數 `-Run -AuthorizationPath -Profile Core|Lob|Geometry|Faults|Memory|All -OutputDirectory `。授權必須解析成 `(localdb)\mySQLPunkExportTest_20261012`、明確C fixture root及owner token;沒有有效manifest不得寫DB。runner 設定並還原 `MYSQLPUNK_MSSQL_IMPORT_INTEGRATION/SERVER/OUTPUT/AUTH/PROFILE`,filter固定 `MSSQL import integration`,要求實際 profile receipt,不能把未啟用測試當 live pass。runner 不自動啟動不明instance;必要時由controller啟動先前已核准的專用名稱。 + +- [x] **Step 1: 補 runner guard 失敗 tests,再實作授權/cleanup保護。** 無-Run/錯server/錯root/reparse/非空output/不足20GiB皆不寫;GUID資料庫與實體檔、owner.token+DB owner extended property雙重驗證才可清。Existing export runner缺manifest不可默默跳過;明確AuthorizationPath,必要時從已核准fixture重新建立可審查manifest,不擴張到Transfer。 +- [x] **Step 2: Core profile。** 新建owned source/target,以現有 exporter產生SQL,再由新 importer本體還原;比對表/欄位/型別/identity/PK/indexes/FK trust/view/table與column extended properties。General腳本含GO、temp/SET/transaction、SELECT多result;無外部sqlcmd作SUT,獨立驗證可另行使用。 +- [x] **Step 3: Lob/Geometry profiles。** 64MiB binary、18MiB Unicode、NULL/empty/XML/Geometry含SRID/Z/M/curves與invalid-but-storable;以串流rawbytes/hash及metadata驗證。native DataOnly也測;注入missing/duplicate/reordered/replayed chunk、Failed1但交易0、missingstate、pendingEOF及final INSERT constraint/Deserialize失敗,即使skip也停止且下一row不寫。 +- [x] **Step 4: Faults/Memory profiles。** 一般batch先INSERT再失敗的stop/skip;所有SqlError落檔;XACT_ABORT/未結交易/斷線/有限timeout seam/logcreate-append-flush-end失敗;取消在resultdrain/stateprobe/COUNT_BIG/close race。結果集數百萬列與64MiB回傳LOB應bounded drain。相同batchshape生成256MiB與2.5GiB串流檔,記錄x64/peak private bytes/working set;暖機後10倍檔案不能線性10倍成長,差異超過256MiB須調查並修復,不能只增加記憶體上限。 +- [x] **Step 5: 各profile通過後提交tests及verification摘要。** 記錄命令、實際matched tests、SQL版本、耗時/artifactsize/hash/cleanup/memory/取消結果;失敗不寫pass,無法驗證明列邊界。Commit `test: verify native MSSQL import roundtrips and failures`。 + +## Task 7: SQL2019 Transfer 實檔試跑、整體驗證與交付 + +**Files:** Create `tests/Run-MssqlImportTransferTrial.ps1`, `docs/MSSQL_SQL_IMPORT.md`;Update verification.md、既有README適當入口/history(ignored)及plan完成狀態。產品修正僅限實際驗收發現的問題並覆核。 + +**Interfaces:** trial runner 必須明確 `-Run -Server localhost -Database Transfer -InputFile C:\Users\johnho\Desktop\Transfer_dump_20261013_165730.sql -OutputDirectory `;Windows整合登入,實際server必須`JOHN\MSSQL2019`、version15.x、compat150、ONLINE/READ_WRITE且NativeSchema空庫。無fixture drop/cleanup/recovery設定入口。runner 設定並還原 `MYSQLPUNK_MSSQL_IMPORT_TRIAL/SERVER/DATABASE/INPUT/OUTPUT`,由 Integration partial 中註冊的 `MssqlImportTransferTrial` 執行,filter `MSSQL import Transfer trial`;沒有 trial opt-in 時明確 skipped,-Run 時必須驗證真正執行 receipt,不能以 skipped 當驗收。 + +**驗收結果:FAILED。** 一次實檔試跑已停止並完成唯讀核對;勾選代表下列執行與記錄工作已完成,不代表3247785筆已完整還原。成功重匯仍需修正後的新來源,以及使用者對清理/重跑的額外授權。詳見 [verification.md](verification.md)。 + +- [x] **Step 1: 先測trial guard再唯讀重查。** 目標已非空、身份/DB create_date變化、檔案不符、空間不足或權限不足時只呈現實際狀態,不能自動清庫、換帳號或重試。記錄source完整預掃描/SHA、batch數/最大batch;不要複製或公開服務這份4.17GiB檔案。 +- [x] **Step 2: 依使用者已有授權執行一次本機實檔。** default-stop,log與監測在C;記錄耗時、CPU/peakmemory、DBdata/log成長、最後位置、實際完成數/錯誤;READCOMMITTED來源的FK失敗不得改NOCHECK。失敗部分保留,先回報具體狀態,額外清除/重跑需使用者針對該狀態授權。 +- [x] **Step 3: 核對結果。** 126 tables/3,247,785 rows為檔案footer預期,COUNT_BIG與實際objects/indexes/constraints/views/comments狀態獨立記錄;不把job完成當資料落地、不把count相同當逐欄hash相同。使用者未人工確認的聲音與操作體驗列pending。 +- [x] **Step 4: 跑最終建置與既有回歸。** 清除本task filter後 `powershell.exe -NoProfile -File tests/Run-SmokeTests.ps1 -Configuration Release`;`./build.bat debug`、`./build.bat release`;AutoReleasePolicy通過。僅在新修正/失敗才重跑相關profiles;不反覆無理由全量還原。全分支一次fresh reviewer,再對實質fix做一次scoped覆核。 +- [x] **Step 5: 文件、歷程與分支checkpoint。** docs說明支援GO/encoding/32MiB batch與LOB無關、generic intrinsics限制、skip/partial effects/logs/取消/ETA/需信任來源;列實際本機證據與未驗證production邊界。保留隔離worktree,精確stage,未獲本次授權前不push/merge/release。Commit `docs: document MSSQL SQL import and acceptance evidence`。 + +## Commands and execution setup + +實作開始時使用 `using-git-worktrees` 檢查已有隔離環境,再優先使用 Codex managed worktree,ref須包含本計畫checkpoint;不可從較舊remote branch遺漏已核准設計。不覆寫primary checkout的untracked檔。D槽空間偏低,managed worktree/生成fixture/build暫存優先C槽,先核對實際容量。 + +本專案新source與test partial都要登錄explicit清單。測試需匹配至少一項,filter沒匹配退出2不得當成功;第一次失敗退出1。測試命令範例(執行owner須保存並還原既有環境值): + +```powershell +$env:MYSQLPUNK_SMOKE_FILTER = 'MSSQL import parser' # 或 worker/wizard +powershell.exe -NoProfile -File tests/Run-SmokeTests.ps1 -Configuration Debug +Remove-Item Env:MYSQLPUNK_SMOKE_FILTER +powershell.exe -NoProfile -File tests/Run-SmokeTests.ps1 -Configuration Release +.\build.bat debug +.\build.bat release +``` + +每task依序記錄BASE/實作commit/測試證據/diff package/覆核結論。只修正實質spec或correctness問題,不做無意義review循環;user/production資料不拿來做破壞式fixture。依使用者明確進實作的授權,沿用分工逐項覆核方式,不再重問執行授權或方法。 diff --git a/docs/design/mssql-import/index.html b/docs/design/mssql-import/index.html new file mode 100644 index 0000000..52711b8 --- /dev/null +++ b/docs/design/mssql-import/index.html @@ -0,0 +1,149 @@ + + + + + +mySQLPunk · MSSQL 匯入精靈畫面原型 + + + + +
+
mySQLPunk/畫面原型 · 全程模擬
+ + + + +
+
+
匯入資料庫 · SQL Server
mySQLPunk
+
DATABASE IMPORT

資料庫匯入精靈

Transfer MSSQL
localhost → JOHN\MSSQL2019
+
1檔案與目標
2匯入檢查
3確認執行
4執行與結果
+
+ + + +
+
讀取與執行遇錯停止

正在匯入資料

dbo.TransferDetail · 明細資料(示意)

+
43%
已完成執行的檔案範圍1.79 / 4.17 GiB
已完成 928 個批次檔案預期 3,247,785 筆
正在執行批次 929 · 行 2,842,105–2,844,816
+
已經過
04:32
預估剩餘
約 6 分 10 秒
預估完成時間
—

依近期批次速度估算;索引、外鍵與結果核對的時間仍可能變動。

+
正在努力把資料放回家!批次依序執行,請稍等一下。
+
讀取與執行›建立索引與外鍵›結果檢查

目前處理佇列

表名與批次為示意

目前沒有錯誤 · 示意紀錄
+ +
+
+
+
+
互動原型 · 檔案摘要與目標識別作為畫面範例;執行進度、批次、錯誤與結果均為模擬。沿用匯出精靈與 Punky 素材
+ + + + + diff --git a/docs/design/mssql-import/snapshot/error-log-dark.jpg b/docs/design/mssql-import/snapshot/error-log-dark.jpg new file mode 100644 index 0000000..75a2b62 Binary files /dev/null and b/docs/design/mssql-import/snapshot/error-log-dark.jpg differ diff --git a/docs/design/mssql-import/snapshot/native-confirm-dark-1.5.png b/docs/design/mssql-import/snapshot/native-confirm-dark-1.5.png new file mode 100644 index 0000000..a7e8bd0 Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-confirm-dark-1.5.png differ diff --git a/docs/design/mssql-import/snapshot/native-confirm-dark-1.png b/docs/design/mssql-import/snapshot/native-confirm-dark-1.png new file mode 100644 index 0000000..cbc5c45 Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-confirm-dark-1.png differ diff --git a/docs/design/mssql-import/snapshot/native-confirm-dark-2.png b/docs/design/mssql-import/snapshot/native-confirm-dark-2.png new file mode 100644 index 0000000..9ad26e7 Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-confirm-dark-2.png differ diff --git a/docs/design/mssql-import/snapshot/native-confirm-light-1.5.png b/docs/design/mssql-import/snapshot/native-confirm-light-1.5.png new file mode 100644 index 0000000..1a6b25f Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-confirm-light-1.5.png differ diff --git a/docs/design/mssql-import/snapshot/native-confirm-light-1.png b/docs/design/mssql-import/snapshot/native-confirm-light-1.png new file mode 100644 index 0000000..f7669d8 Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-confirm-light-1.png differ diff --git a/docs/design/mssql-import/snapshot/native-confirm-light-2.png b/docs/design/mssql-import/snapshot/native-confirm-light-2.png new file mode 100644 index 0000000..e2d6e09 Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-confirm-light-2.png differ diff --git a/docs/design/mssql-import/snapshot/native-error-log-dark.png b/docs/design/mssql-import/snapshot/native-error-log-dark.png new file mode 100644 index 0000000..486f6cd Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-error-log-dark.png differ diff --git a/docs/design/mssql-import/snapshot/native-error-log-light.png b/docs/design/mssql-import/snapshot/native-error-log-light.png new file mode 100644 index 0000000..70738ed Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-error-log-light.png differ diff --git a/docs/design/mssql-import/snapshot/native-progress-dark.png b/docs/design/mssql-import/snapshot/native-progress-dark.png new file mode 100644 index 0000000..486f6cd Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-progress-dark.png differ diff --git a/docs/design/mssql-import/snapshot/native-progress-light.png b/docs/design/mssql-import/snapshot/native-progress-light.png new file mode 100644 index 0000000..1a0f432 Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-progress-light.png differ diff --git a/docs/design/mssql-import/snapshot/native-stopping-light.png b/docs/design/mssql-import/snapshot/native-stopping-light.png new file mode 100644 index 0000000..91d1590 Binary files /dev/null and b/docs/design/mssql-import/snapshot/native-stopping-light.png differ diff --git a/docs/design/mssql-import/snapshot/progress-light.jpg b/docs/design/mssql-import/snapshot/progress-light.jpg new file mode 100644 index 0000000..1e7b4d3 Binary files /dev/null and b/docs/design/mssql-import/snapshot/progress-light.jpg differ diff --git a/docs/design/mssql-import/verification.md b/docs/design/mssql-import/verification.md new file mode 100644 index 0000000..b5624dc --- /dev/null +++ b/docs/design/mssql-import/verification.md @@ -0,0 +1,110 @@ +# Native MSSQL import acceptance + +Task 6 was run on the dedicated `(localdb)\mySQLPunkExportTest_20261012`, SQL Server 17.0.4025.3, using the actual importer and native exporter. All 65 expected case IDs in `tests/MssqlImportFixtures/expected-cases.json` matched executed PASS receipts. Every profile and both memory children confirmed fixture cleanup. Raw scripts, logs, hashes and receipts remain local on C, outside Git. + +## Reproduce + +The controller must first inspect the approved manifest and start its exact dedicated instance. The runner checks that it is already running before opening SQL connections and never starts it. + +```powershell +powershell -NoProfile -File tests/Test-MssqlImportIntegrationGuards.ps1 +powershell -NoProfile -File tests/Run-MssqlImportIntegration.ps1 -Run -AuthorizationPath -Profile Core -OutputDirectory +# Repeat with Lob, Geometry, Faults and Memory, each in a new output directory. +$env:MYSQLPUNK_SMOKE_FILTER='MSSQL import' +powershell -NoProfile -File tests/Run-SmokeTests.ps1 -Configuration Debug +powershell -NoProfile -File tests/Run-SmokeTests.ps1 -Configuration Release +``` + +`-Profile All` also runs all five profiles. The runner restores environment flags/filter in `finally`. Missing, skipped, incomplete, wrong-run or mismatched case receipts fail acceptance; executable and case-receipt SHA256 values are checked. Ordinary offline runs skip the opt-in live entry and are not live acceptance. + +Fixture creation uses new GUID database names and exact C descendant MDF/LDF paths. Cleanup requires the generated/created allowlist, private owner.token, matching database extended property, and the exact two physical files with reparse checks. No existing user database is a cleanup target. Permission cases create users WITHOUT LOGIN and narrow roles/GRANT/DENY only inside these fixtures; impersonation uses nonpooled owned connections with REVERT/finally close. No server login, snapshot, recovery or KILL changes are required. + +## Evidence from 2026-10-13/14 + +The evidence base is `C:\Users\johnho\AppData\Local\Temp\mysqlpunk-export-20261012-7cadb673332543c3b17c61924490a702`. Each relative output below contains `/profile-receipt.json`, the exact case receipts, result JSON and complete error logs. + +| Profile | Matched PASS cases | Relative output | Main checks | +| --- | ---: | --- | --- | +| Core | 12 | import-core-final-b7e2fa6305464e4bb5f18aa9a3a534de | Full/SchemaOnly/DataOnly, 12 tables/15 rows, identities, indexes, FK trust including composite FK, views/properties, same-session temp/SET/identity/transaction/multiple results, parser no-write and target recreation | +| Lob | 14 | import-lob-2d8d118a9e714bb09a711d2cbaef67bb | 64 MiB binary, 18 MiB Unicode, NULL/empty/XML bytes/hashes, DataOnly; ten corruption/failure cases under both skip policies | +| Geometry | 11 | import-geometry-a72bcca2f4ee4945b4c98f83762e4621 | Raw Serialize hashes, >16 MiB geometry, SRID/Z/M/curves/NULL/empty/invalid values, DataOnly; seven corruption/failure cases under both skip policies | +| Faults | 22 | import-faults-readonly-lease-c5aff2d055d94acd9b0d019ced823a9d | Partial writes stop/skip, all three provider errors and flush ordering, real transaction states 1/-1, EOF transaction, timeout -2, owned transport loss, log IO, cancellation at reader/probe/count stages, close/success races, permission-filtered metadata, read-only before/after preflight and recreated targets | +| Memory | 6 | import-memory-9d9b76eacca74e55b494272552b45c91 | 2.5 million rows and 64 MiB returned LOB drained with zero value getters; warmed separate x64 streaming children and comparison | + +Selected input artifacts (full SHA256 values are also in result receipts): + +| Artifact | Bytes | SHA256 | +| --- | ---: | --- | +| Final Core full | 65,270 | 2f6d2102b149605e2a8f4ca48b35b9734e90141cb39d91b0c9f6f5e631e40750 | +| Lob full | 174,573,590 | 26142c71de9a3acceee96037ae884b0f6147fa080f6ad419e31313473c0ac085 | +| Geometry full | 36,193,436 | bc39f43927d2284d8a79b47bf6ef99651c07e560e6ceb92c635c2ab69277a093 | +| Memory 1x | 268,443,904 | 62c2cd573a5f692c1dd1ad4e3461e77773f2e31060224875125aec829ce98a34 | +| Memory 10x | 2,684,439,040 | e6f4c9c45c2749346e028732d35c4274d19463f2ead6d6c51ff66b6d6a494052 | + +Memory children use identical approximately 1 MiB batch shapes: 256/2,560 batches. Private-byte peaks were 90,361,856 (~86.2 MiB) and 81,547,264 (~77.8 MiB); working-set peaks were 102,776,832 and 91,635,712. Baseline-adjusted peak difference was -9,043,968 bytes, below the 268,435,456-byte investigation threshold. This is the measured result for these fixtures, not a universal memory ceiling. `Result.Elapsed` was 27.256 s / 261.797 s for Run execution, verification and cleanup, excluding preflight. Parent case durations, including child startup and scanning, were 55.949 s / 518.074 s. Stage aggregates and 25 ms peak samples are retained in memory-comparison.json/memory-measurement.json. + +The original Task 6 offline Debug and Release commands each passed 38 matched entries (including the inert live entry), plus AutoReleasePolicy. Nineteen worker groups and five parser groups include regressions for the corrections below. Pure PowerShell authorization/cleanup/receipt guards also passed without SQL. Local ignored logs are `.superpowers/sdd/implementation-plan/task-6-import-final-{debug,release}-green.log`, `task-6-runner-guard-final-green.log`, and `task-6-core-final-runner.log`. + +## Corrections established by failures + +- On this SQL17 instance, `SELECT DB_ID(),@@TRANCOUNT,XACT_STATE()` on a fresh session returned transaction count 0 with XACT_STATE 1. A state-only SELECT returned 0/0. Capturing DB_ID in a completed DECLARE statement before selecting transaction state restored the observed caller state without normalizing it. Initial native full import failed EofTransactionOpen; the corrected actual full import passed. Real open/committable/doomed transaction fault cases still stop. This observation is not claimed for every SQL version. +- Native SchemaOnly FK actions were mistaken for DELETE/UPDATE data. A bounded FK/REFERENCES/action context now recognizes NO ACTION/CASCADE/SET NULL/SET DEFAULT, including composite column lists. Exact parser RED/GREEN and final Core cover this; genuine DML after SET ... ON and after FK clauses remains rejected. +- Cleanup rollback and owner-release SqlError details previously disappeared. Immutable current-cleanup details now reach the complete log as Origin=Cleanup with zero source coordinates and DuringBatch before terminal flush. Typed fake-command/fake-owner adapter regressions and the actual worker/logger prove detail retention, merged ordering, cached-receipt deduplication and unchanged cleanup truth; the bounded asynchronous publication contract is described below. The pending cleanup assertion now waits for adapter publication, with a deterministic owner pre-return race test. + +Failed receipts were retained: the initial state-probe failure, a test observer sharing failure (fixed only FileShare.ReadWrite), and an invalid fixture CREATE SCHEMA batch (fixed fixture setup). The runner also rejected a stopped instance before SQL or output creation; the controller restarted the exact authorized instance, and only small Core was repeated. The stop cause was not established. + +## Review repair I1/I2 + +The initial Faults receipt `import-faults-7578048db9354ebfa6b6cef5d8ac5215` covered recreation only. Review found the missing live read-only coverage and an inverse cancellation race in release diagnostics. Both were repaired together after the separate Task 5 terminal-log fix, based on `152e4192d9e775b9529ea1653d357be72ad5ad1f`. + +For I1, the provider can unwind before a watchdog's owner.Dispose completes. Cleanup now waits under the existing gate for release publication before its immutable final receipt/log, using a monotonic deadline of 10 seconds in production and Monitor.Wait/Pulse. Details delivered within that window appear exactly once before End. Expiry produces explicit SessionReleasePending and uncertain cleanup; release details are not yet available, and no late logger, repeated Dispose or unbounded watcher is created. A deterministic actual-service/adapter test with typed fake-owner errors first failed because the worker finished before publication, then passed with complete two-error logging. Its second order holds the owner past a 150 ms test window, verifies bounded pending completion, observes late publication under the gate, and proves no redispose or post-terminal log mutation. Actual RED/GREEN logs: `task-6-fix-round1-inverse-{red,green}.log` under the ignored plan directory. + +For I2, the existing `faults.target-unavailable-boundary` case now includes read-only before preflight, read/write-to-read-only after preflight, and the original recreation. Every fixed READ_ONLY/READ_WRITE WITH NO_WAIT transition freshly checks the private token, DB ownership property, generated/created name and exact C physical files. Both read-only paths restore READ_WRITE in finally. The final Faults output above proves: + +- `target-readonly-preflight.json`: Passed=false, Writable=false, TargetReadOnly, independently checked zero source writes. +- `target-{probe-open,before-run,factory-open-3}.json`: fresh MASTER is_read_only=true; `target-sut-3.json`: actual adapter Writable=false with the exact target identity. +- `result-target-readonly-after-preflight.json`: Failed/TargetReadOnly, ProcessedBytes=0, SuccessfulBatches=0, FailedBatches=0 and all cleanup flags true. `target-readonly-effects.json`: source table absent; equivalent direct CREATE through a fresh independent SqlConnection failed with SQL 3906. +- The separate fixture probe holds no active command or transaction across SUT execution, then closes before finally restoring READ_WRITE. `target-restored-3.json` confirms is_read_only=false. Recreation still rejects TargetChanged with zero processed bytes; profile cleanup confirms no remaining owned DBs. + +Two failed reproduction leaves are retained: `import-faults-fix1-a6a5c4afb0174212a3ef818e9677bdcc` and `import-faults-readonly-observe-91859e49728e4a5488f8dd1a17449a42`. Without the probe lease, the fixture had already reverted to READ_WRITE before SUT Run; the importer read the current writable state and CREATE succeeded. This was not a demonstrated importer gate failure. AUTO_CLOSE=true was observed, but the reversion's cause was not established and AUTO_CLOSE was not changed. The probe lease maintains the deliberately read-only fixture state without weakening the expected rejection. + +Repair Phase A Debug and Release each passed 39 importer entries plus AutoReleasePolicy (`task-6-fix-round1-offline-{debug,release}-green.log`); the count includes the accepted Task 5 regression. After the final live fixture implementation, its Release build/guard entry passed, final workers19 and parsers5 passed, and pure PowerShell guards passed again (`task-6-fix-round1-{lease-build,worker-final-green,parser-final-green,pure-guards-final-green}.log`). Final Faults22 exact IDs/case SHA/executable SHA/Confirmed cleanup were checked; executable SHA256 was `632157dbfc2c513d4d5130bc1868726823991fc0134f0496ec4aed9846b8c710`. Other live profiles and giant memory files were not rerun for this repair. + +## Boundaries + +Live cleanup and cancellation prove successful cleanup and truthful uncertainty. A naturally occurring provider SqlError during rollback/owner disposal was not induced; its complete-error contract has adapter/worker/log regression evidence only. Owner-release diagnostics and composite-FK changes were followed by offline checks and small Core. The review repair was followed by Faults22 and focused offline checks; large profiles remain the historical measured evidence above. At the Task6 checkpoint, full unrelated regression, native UI/audio/DPI, SQL Server2019/Transfer and the user's 4.17 GiB dump, browser/IIS/production remained separate acceptance work. No user dump was opened or executed by Task6; the actual Task7 outcome is recorded below. + +Task 6 independent scoped re-review approved I1/I2 at `5c848ae`. The terminal truncation fix at `152e419` passed its actual-service/native UI regression plus13wizard/19worker groups. Whole-branch review and SQL2019 Transfer were deferred to Task7; its actual stopped-failure boundary is below. + +## SQL2019 Transfer trial: guards and actual stopped failure + +Task7 is a separate one-shot user-file acceptance harness, not a LocalDB fixture profile. `tests/Run-MssqlImportTransferTrial.ps1` requires explicit `-Run -Server localhost -Database Transfer -InputFile C:\Users\johnho\Desktop\Transfer_dump_20261013_165730.sql -OutputDirectory `. It cannot start an instance, clear/drop Transfer, change recovery/constraints/permissions or retry. The fixed C-only atomic attempt marker is independent of output directory and remains after every claimed outcome; any existing/corrupt/interrupted marker blocks reuse. + +Phase A local evidence: Debug and targeted Release builds plus the registered C# Transfer guard group passed, including strict bounded JSON, exact measured product/smoke executable pin predicates, SID/name mismatch, owned-token monitor cancellation on <20GiB/IO failure, tiny owned-marker CreateNew/race/unchanged-existing cases and primary-failure retention. SQL factories were explicitly blocked in this offline group; no SQL Open occurred. The actual runner NoRun with foreign opt-ins set returned SKIPPED before source/target work; wrong-server `-Run` rejected before pin/child/source/SQL. Windows PowerShell 5.1.26100.9549 passed 103 pure decision assertions plus seven strict JSON negatives. Existing opt-ins/filter were saved/restored; AutoReleasePolicy passed. Exact final source/build/pin evidence is recorded in the local Task7 handoff, outside Git. + +Phase A ended before opening/scanning the user file or connecting SQL2019. The controller subsequently approved the narrow guard/monitor/metadata repair reviews and freshly verified 269 source files, six registration files, both manifests and both binaries before accepting the 224445b runtime pin. The actual one-shot outcome below supersedes the former runtime-pending boundary. Count equality is not source field-hash or source-definition equality. Human sound/interaction, physical DPI and production/browser/IIS remain separate unverified boundaries. + +Final existing regression was completed before the actual data trial: unfiltered Release smoke240 PASS plus AutoReleasePolicy, build.bat Debug and Release each PASS. SQL opt-ins/filter were saved/cleared/restored; no live profile was repeated. The controller reports the whole-branch review through 56f4ddf and the scoped metadata repair review approved with no findings. No push/merge/release is implied by the local checkpoint. + +### SQL2019 pre-write harness rejection and repair + +The first released harness invocation retained `transfer-trial-20261014-0124/final.json`: `Executed=false`, `AttemptMarkerDurable=false`, `Status=NotStarted`, `FailureCode=TrialFailure:InvalidCastException`. It stopped in the read-only identity SELECT before monitor creation, source preflight/scan, fixed marker claim or service.Run. No source SQL was executed; this flow does not independently prove current target emptiness. The original output is retained and never relabeled PASS. + +A narrowly authorized read-only provider-shape probe confirmed DB_ID ordinal2 is SQL smallint/System.Int16 (compatibility ordinal5 is tinyint/Byte). The harness had used GetInt32 directly; existing product metadata already used Convert.ToInt32. The actual DataTableReader Int16 shape reproduced the failure offline, then the shared numeric conversion fixed both identity and used-log DB_ID checks. Wrapper reporting now preserves the fixed child NotStarted failure instead of trying to open an absent marker. Focused Release C#51 assertions and AutoReleasePolicy PASS; WinPS106 pure decisions plus seven strict JSON negatives PASS. Candidate pins were revoked before rebuild, then freshly inspected and accepted by the controller at 224445b. No DB cleanup/settings change occurred. The unconsumed import authorization was used exactly once in the actual trial below. +### Actual one-shot SQL2019 result: Failed, partial state preserved + +Local evidence is `transfer-trial-20261014-0133` under the ignored plan directory. The first `0124` pre-write output remains unchanged. At runtime source commit `224445b`, fresh integrated identity was `JOHN\MSSQL2019` / Transfer, SQL Server 15.0.2190.7, compatibility150, ONLINE/READ_WRITE/FULL. Native preflight proved visible zero user objects and table/view creation rights. The held 4,478,414,151-byte UTF-8 source scan completed with SHA256 `c84db0d32b4a9b3c2fd940eb809c724f67cd31d26a0f292fd31f11a65e364eff`, 206,409 batches and max4,194,300 UTF-16 code units. Its native READ COMMITTED warning remains applicable. + +The durable fixed attempt marker was claimed and service.Run executed once. Stop-on-first-error rejected batch1481, source lines5131–5328, byte interval[1957461,2018937), SQL241/state1/class16: date/time string conversion failed. The service acknowledged1480 successful batches /1,957,461 successful bytes, one failed batch, zero skipped bytes and no uncertain batch. The final receipt is Failed / TrialSutNotCompleted; command ended, session released, transaction cleanup confirmed and cleanup errors0. Cleanup confirms owned session/transaction handling, not removal of already committed effects. Wrapper exit1 reports secondary `ReceiptType` because a Failed final has no success-only independent-count fields; this is not the underlying SQL failure or acceptance. Its final receipt and complete SUT log retain the actual failure and log path. + +Total receipt wall time was199.847864s, including full preflight; SUT-reported elapsed was10.039839s. The197 one-second samples recorded last CPU189,422ms (sampled, not exact process-exit CPU), peak private172,118,016B and peak working set186,707,968B. First/last sampled data and log allocations were8,388,608B each, observed allocation growth0; terminal allocation was not separately sampled after exit. Actual used-log snapshots were909,312B before Run and3,751,936B at terminal (+2,842,624B). Minimum sampled C free space was159,196,868,608B. Scanning ETA remained null; later execution ETA was an estimate invalidated by the failure, not a completion promise. + +An explicitly authorized fresh read-only independent probe matched the same server/database/id/create_date and current/original Windows SID/name, with dbo complete metadata visibility. Per-table COUNT_BIG measured126 tables /46 rows, not3,247,785. Catalog inventory contained126 USER_TABLE and203 DEFAULT_CONSTRAINT records; no indexes, views, FK, CHECK or extended-property names were present at this partial point. No FK/check trust state existed to compare. Private `partial-independent-{identity,summary}.json` and counts/catalog JSONL receipts retain the details without business row values. Object-definition/field-content equality is unverified. No source rewrite, target cleanup, recovery/constraint/permission change or Transfer retry occurred; the one actual import authorization is consumed and the failed partial state/marker remains preserved. + +### Existing exporter legacy-date correction and bounded verification + +A bounded read of only the61,476-byte failing source batch found seven-fraction ISO literal shape `####-##-##T##:##:##.#######` targeting three legacy datetime(scale3) columns. No business row values were printed. Read-only synthetic SQL2019 SELECTs confirmed seven-fraction TRY_CONVERT yields NULL for datetime/smalldatetime (direct CONVERT errors241/295), while three fractions convert; datetime2(7) accepts and retains seven. The existing exporter grouped these three types under seven-fraction formatting. Commit `ea733a6` splits datetime/smalldatetime to three fractions and leaves datetime2/datetimeoffset/time precision unchanged; importer execution semantics and the original user file remain unchanged. + +Meaningful codec RED/GREEN covers invariant legacy literal width and stored SqlDateTime min/.003/.007/max ticks; existing seven-digit modern-type assertions still pass. Focused Release scalar1 + parser5 + worker19 groups passed, as did AutoReleasePolicy and both build.bat Debug/Release. SQL opt-ins/filter were saved/cleared/restored. No full240 suite or heavy live profiles were repeated for this local change. + +Only the existing dedicated Core fixture was extended with two legacy date columns and min/max/NULL values, preserving12 expected case IDs and15 rows. Its first setup failed SQL207 because UPDATE was compiled with ALTER ADD in one batch; the failed leaf `import-core-legacy-date-3c6cae9db65342dcb9c97481acd4f951` records no cases and Confirmed cleanup. Commit `1ff20c9` separates the two owned-source Exec calls. Fresh compile/integration guard1/AutoReleasePolicy passed. The controller authorized one replacement Core, `import-core-legacy-date-790caeac0e5d438a8e5487cc265e6608`, on the exact owned SQL17 LocalDB; all12 IDs/statuses/case SHA256 values/executable SHA256 matched, complete=true, cleanup Confirmed. Actual full and DataOnly source/target raw bytes comparisons passed for the legacy date boundaries/NULL and existing modern dates. Full export was65,529B SHA256 `70e7e8a61ece756c30303a6cc119e6975a85d90ef0b58876cd5b88a0e529a296` /80 acknowledged batches; DataOnly58,384B /52 batches, both zero failures with confirmed session/transaction cleanup. This replaces the small Core evidence for the date fix; historical65-case evidence remains separate and no additional case IDs are claimed. It does not establish a successful reimport of the user's original dump. diff --git a/mySQLPunk/Form1.MssqlImport.cs b/mySQLPunk/Form1.MssqlImport.cs new file mode 100644 index 0000000..bf3849f --- /dev/null +++ b/mySQLPunk/Form1.MssqlImport.cs @@ -0,0 +1,54 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Windows.Forms; +using mySQLPunk.lib; +using mySQLPunk.lib.MssqlImport; +namespace mySQLPunk +{ + public partial class Form1 + { + internal Func, MssqlImportRequest, MssqlImportResult> MssqlImportWizardRunner; + internal Func SqlImportOpenDialogRunner; + internal Action MssqlImportStatusSink; + private static bool IsMssqlImportTarget(TreeDatabaseTarget target) => target != null && + (string.Equals(target.ProviderName, "mssql", StringComparison.OrdinalIgnoreCase) || string.Equals(target.ProviderName, "sqlserver", StringComparison.OrdinalIgnoreCase)); + private DialogResult ShowSqlImportFileDialog(OpenFileDialog dialog) => SqlImportOpenDialogRunner == null ? dialog.ShowDialog() : SqlImportOpenDialogRunner(dialog); + private void SetMssqlImportStatus(string text) { if (MssqlImportStatusSink == null) UpdateMainStatus(text); else MssqlImportStatusSink(text); } + private void OpenMssqlImportWizard(TreeDatabaseTarget target) + { + try + { + var settings = new Dictionary(StringComparer.OrdinalIgnoreCase); + foreach (var pair in target.ConnectionInfo) + { + if (string.Equals(pair.Key, "pdo", StringComparison.OrdinalIgnoreCase) || string.Equals(pair.Key, "isConnect", StringComparison.OrdinalIgnoreCase) || string.Equals(pair.Key, "connString", StringComparison.OrdinalIgnoreCase)) continue; + if (pair.Value == null || pair.Value is string || pair.Value.GetType().IsPrimitive || pair.Value is decimal) settings[pair.Key] = pair.Value; + } + settings["db_kind"] = "mssql"; settings["provider"] = "mssql"; settings["initial_database"] = target.DatabaseName; + var initial = new MssqlImportRequest(Guid.NewGuid(), "", target.DatabaseName); + MssqlImportResult result; + if (MssqlImportWizardRunner != null) result = MssqlImportWizardRunner(settings, initial); + else + { + using (var wizard = new MssqlImportWizardForm(initial, () => new MssqlImportService( + () => new MssqlImportSqlSession(ConnectionOpenService.OpenMssqlImportSession(settings)), job => new MssqlImportLog(job)))) + { wizard.ShowDialog(this); result = wizard.Result; } + } + if (result == null) return; + string text; + switch (result.Status) + { + case MssqlImportStatus.Completed: text = ReportText.Get("SQL Server 匯入執行完成", "SQL Server import execution completed"); break; + case MssqlImportStatus.CompletedWithErrors: text = ReportText.Get("SQL Server 執行完成,但有錯誤/部分寫入", "SQL Server completed with errors/partial writes"); break; + case MssqlImportStatus.Cancelled: text = ReportText.Get("SQL Server 匯入已取消;先前寫入可能保留", "SQL Server import cancelled; previous writes may remain"); break; + case MssqlImportStatus.CleanupUncertain: text = ReportText.Get("SQL Server 匯入/清理狀態未確認", "SQL Server import/cleanup unconfirmed"); break; + default: text = ReportText.Get("SQL Server 匯入停止:發生錯誤", "SQL Server import stopped: error"); break; + } + if (!string.IsNullOrWhiteSpace(result.LogPath)) text += " · " + result.LogPath; + SetMssqlImportStatus(text); + } + catch (Exception) { SetMssqlImportStatus(ReportText.Get("無法開啟 SQL Server 匯入精靈,請確認已選目標。", "Cannot open SQL Server import wizard; review the selected target.")); } + } + } +} diff --git a/mySQLPunk/Form1.cs b/mySQLPunk/Form1.cs index 4617393..9bae155 100644 --- a/mySQLPunk/Form1.cs +++ b/mySQLPunk/Form1.cs @@ -7334,6 +7334,12 @@ private async void ImportSqlWithDialog() return; } + if (IsMssqlImportTarget(target)) + { + OpenMssqlImportWizard(target); + return; + } + MySqlImportOptions mysqlOptions = null; if (IsMySqlTarget(target)) { @@ -7347,7 +7353,7 @@ private async void ImportSqlWithDialog() dialog.Filter = Localization.T("Common.SqlFilesFilter"); dialog.Multiselect = false; - if (dialog.ShowDialog() != DialogResult.OK) + if (ShowSqlImportFileDialog(dialog) != DialogResult.OK) { return; } diff --git a/mySQLPunk/MssqlImportWizardForm.cs b/mySQLPunk/MssqlImportWizardForm.cs new file mode 100644 index 0000000..0104fc1 --- /dev/null +++ b/mySQLPunk/MssqlImportWizardForm.cs @@ -0,0 +1,475 @@ +using System; +using System.Diagnostics; +using System.Drawing; +using System.Globalization; +using System.IO; +using System.Linq; +using System.Threading; +using System.Threading.Tasks; +using System.Windows.Forms; +using mySQLPunk.lib; +using mySQLPunk.lib.MssqlImport; +namespace mySQLPunk +{ + internal sealed class MssqlImportWizardForm : Form + { + private readonly Func serviceFactory; + private readonly string database; + private readonly Panel[] pages = new Panel[4]; + private readonly TextBox source = new TextBox(), target = new TextBox(), logPath = new TextBox(), errors = new TextBox(); + private readonly CheckBox skip = new CheckBox(), confirm = new CheckBox(); + private readonly Label steps = new Label(), status = new Label(), facts = new Label(), checkSummary = new Label(), confirmSummary = new Label(); + private readonly Label stage = new Label(), coverage = new Label(), position = new Label(), elapsed = new Label(), eta = new Label(), errorCount = new Label(), resultText = new Label(); + private readonly Button next = new Button(), back = new Button(), recheck = new Button(), cancel = new Button(), browse = new Button(), mute = new Button(); + private readonly Button openLog = new Button(), openFolder = new Button(), copyPath = new Button(); + private readonly ProgressBar bar = new ProgressBar(), scanBar = new ProgressBar(); + private readonly PunkyReportLoadingControl punky = new PunkyReportLoadingControl(), confirmPunky = new PunkyReportLoadingControl(); + private readonly System.Windows.Forms.Timer timer = new System.Windows.Forms.Timer { Interval = 200 }; + private readonly Stopwatch clock = new Stopwatch(); + private readonly object progressGate = new object(); + private MssqlImportProgress latest; + private MssqlImportPreflight prepared; + private MssqlImportRequest request; + private MssqlImportService service; + private CancellationTokenSource cancellation; + private Task cancellationDispatch; + private bool acceptingProgress, terminalRendered; + private long confirmedBytes, lastAdvance, fileBytes; + private int generation, step; + private Guid activeJob; + private bool busy, executing, closing, cancelRequested, disposing; + internal MssqlImportResult Result { get; private set; } + // Instance action boundaries prevent native tests from launching apps or changing the clipboard. + internal Action PathOpener; + internal Action ClipboardWriter; + private static string T(string zh, string en) => ReportText.Get(zh, en); + internal MssqlImportWizardForm(MssqlImportRequest initialRequest, Func serviceFactory) + { + if (initialRequest == null) throw new ArgumentNullException(nameof(initialRequest)); + this.serviceFactory = serviceFactory ?? throw new ArgumentNullException(nameof(serviceFactory)); database = initialRequest.DatabaseName; + Name = "mssqlImportWizard"; Text = T("匯入 SQL 檔案 · SQL Server", "Import SQL file · SQL Server"); + AutoScaleMode = AutoScaleMode.Dpi; Font = UiKit.Body; ClientSize = new Size(1100, 750); + MinimumSize = new Size(640, 480); StartPosition = FormStartPosition.CenterParent; + var root = new TableLayoutPanel { Dock = DockStyle.Fill, Padding = new Padding(24), ColumnCount = 1, RowCount = 4 }; + root.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 100)); + root.RowStyles.Add(new RowStyle(SizeType.AutoSize)); root.RowStyles.Add(new RowStyle(SizeType.Percent, 100)); + root.RowStyles.Add(new RowStyle(SizeType.AutoSize)); root.RowStyles.Add(new RowStyle(SizeType.AutoSize)); Controls.Add(root); + var header = Stack(); header.Controls.Add(L(T("SQL 檔案匯入精靈", "SQL file import wizard"), UiKit.Title)); + steps.AutoSize = true; steps.Margin = new Padding(0, 12, 0, 16); header.Controls.Add(steps); root.Controls.Add(header, 0, 0); + var body = new Panel { Dock = DockStyle.Fill }; root.Controls.Add(body, 0, 1); + for (int i = 0; i < 4; i++) { pages[i] = new Panel { Name = "importStep" + i, Dock = DockStyle.Fill, AutoScroll = true }; body.Controls.Add(pages[i]); } + BuildSource(initialRequest); BuildCheck(); BuildConfirm(); BuildExecution(); + status.Name = "importStatus"; status.AutoSize = true; status.Margin = new Padding(0, 10, 0, 8); root.Controls.Add(status, 0, 2); + var footer = new FlowLayoutPanel { Name = "importFooter", Dock = DockStyle.Fill, AutoSize = true, WrapContents = true, FlowDirection = FlowDirection.RightToLeft }; + Configure(cancel, "importCancel", T("取消", "Cancel"), (s, e) => { if (busy) RequestCancel(); else Close(); }); + Configure(next, "importNext", T("下一步 →", "Next →"), async (s, e) => { + if (busy || closing || disposing) return; + if (step == 0) await CheckAsync(); + else if (step == 1 && prepared?.Passed == true) { step = 2; ShowPage(); } + else if (step == 2 && prepared?.Passed == true && confirm.Checked) await RunAsync(); + else if (step == 3 && Result != null) await CheckAsync(); + }); + Configure(back, "importBack", T("← 上一步", "← Back"), (s, e) => { if (!busy && step > 0 && step < 3) { step--; ShowPage(); } }); + Configure(recheck, "importRecheck", T("重新檢查", "Recheck"), async (s, e) => { if (!busy && !closing) await CheckAsync(); }); + footer.Controls.AddRange(new Control[] { cancel, next, back, recheck }); root.Controls.Add(footer, 0, 3); + ThemeManager.MarkAsPrimary(next); AcceptButton = next; CancelButton = cancel; cancel.DialogResult = DialogResult.None; + timer.Tick += (s, e) => RenderHeartbeat(); + ClientSizeChanged += (s, e) => { + root.Padding = new Padding(ClientSize.Width < 800 || ClientSize.Height < 600 ? 12 : 24); + steps.MaximumSize = new Size(Math.Max(180, root.ClientSize.Width - root.Padding.Horizontal), 0); status.MaximumSize = steps.MaximumSize; + foreach (var button in new[] { next, back, cancel, recheck }) { button.Margin = new Padding(3); button.MinimumSize = new Size(85, 32); } + Reflow(); + }; + Shown += (s, e) => { + var work = Screen.FromControl(Owner ?? this).WorkingArea; + MinimumSize = new Size(Math.Min(MinimumSize.Width, work.Width), Math.Min(MinimumSize.Height, work.Height)); + Size = new Size(Math.Min(Width, work.Width), Math.Min(Height, work.Height)); + Location = new Point(Math.Max(work.Left, Math.Min(Left, work.Right - Width)), Math.Max(work.Top, Math.Min(Top, work.Bottom - Height))); Reflow(); + }; + FormClosing += (s, e) => { if (busy) { e.Cancel = true; closing = true; RequestCancel(); } else DiscardPrepared(); }; + ThemeManager.ApplyTo(this); ShowPage(); + } + private static FlowLayoutPanel Stack() => new FlowLayoutPanel { Dock = DockStyle.Top, AutoSize = true, FlowDirection = FlowDirection.TopDown, WrapContents = false, Margin = Padding.Empty }; + private static Label L(string text, Font font = null) => new Label { AutoSize = true, Text = text, Font = font ?? UiKit.Body, Margin = new Padding(0, 6, 0, 8) }; + private static void Configure(Button b, string name, string text, EventHandler handler) + { b.Name = name; b.Text = b.AccessibleName = text; b.AutoSize = true; b.MinimumSize = new Size(90, 34); b.Margin = new Padding(3); b.Click += handler; } + private void BuildSource(MssqlImportRequest initial) + { + var content = Stack(); pages[0].Controls.Add(content); + content.Controls.Add(L(T("把 SQL 檔案,帶進目標資料庫", "Bring a SQL file into the selected database"), UiKit.Subtitle)); + content.Controls.Add(L(T("先選檔案,再串流檢查;預檢不執行檔案中的 SQL。", "Choose a file, then check it by streaming. Preflight does not execute its SQL."))); + source.Name = "importSourcePath"; source.Text = initial.SourcePath; source.ReadOnly = true; source.Width = 800; + source.AccessibleName = T("SQL 檔案完整路徑", "SQL file path"); source.TextChanged += (s, e) => { InvalidatePrepared(); ReadFileSize(); }; + content.Controls.Add(L(T("SQL 檔案", "SQL file"))); content.Controls.Add(source); + Configure(browse, "importBrowse", T("選擇檔案…", "Choose file…"), (s, e) => { + if (busy) return; + using (var dialog = new OpenFileDialog { Filter = Localization.T("Common.SqlFilesFilter"), Multiselect = false, CheckFileExists = true }) + if (dialog.ShowDialog(this) == DialogResult.OK) source.Text = dialog.FileName; + }); content.Controls.Add(browse); + facts.Name = "importFileFacts"; facts.AutoSize = true; content.Controls.Add(facts); + target.Name = "importTarget"; target.ReadOnly = true; target.Width = 800; + target.Text = T("已選資料庫:", "Selected database: ") + database + T(";實際伺服器識別將於預檢顯示。", "; resolved server identity appears after preflight."); + target.AccessibleName = T("已選目標資料庫", "Selected target database"); content.Controls.Add(L(T("目標 · 使用目前資料庫樹的連線設定", "Target · existing selected connection"))); content.Controls.Add(target); + content.Controls.Add(L(T("更換目標請關閉此精靈,於資料庫樹選擇另一個資料庫後重新開啟。", "To change target, close this wizard and select another database in the tree."))); + content.Controls.Add(L(T("支援 mySQLPunk 匯出 SQL 與一般 T-SQL/GO;嚴格偵測 UTF-8,或有 BOM 的 UTF-16。單一 GO 批次上限 32 MiB(16,777,216 個 UTF-16 code unit),不是欄位大小限制。", "Supports mySQLPunk SQL and ordinary T-SQL/GO; strict UTF-8 or BOM-marked UTF-16. One GO batch is limited to 32 MiB decoded (16,777,216 UTF-16 code units), not a field-size limit."))); + content.Controls.Add(L(T("不支援 SQLCMD 指令或直接 USE 切換資料庫。三段名稱、動態 SQL 或程序仍可能存取其他資料庫,請確認腳本來源。精靈不會自動建立、清空或覆寫資料庫。", "No SQLCMD directives or direct USE changes. Three-part names, dynamic SQL and procedures can still access other databases; review the script source. No automatic database creation or clearing."))); ReadFileSize(); + } + private void BuildCheck() + { + var content = Stack(); pages[1].Controls.Add(content); + content.Controls.Add(L(T("先確認,這份檔案能不能匯入", "Check the file and target before importing"), UiKit.Subtitle)); + content.Controls.Add(L(T("完整串流預掃描:編碼、GO 批次、格式摘要、目標與 log 可寫性。這是分析進度,尚未執行 SQL。", "Streaming preflight checks encoding, GO batches, format summary, target and log writability. No source SQL has executed."))); + scanBar.Name = "importScanProgress"; scanBar.Width = 800; scanBar.Height = 10; content.Controls.Add(scanBar); + checkSummary.Name = "importCheckSummary"; checkSummary.AutoSize = true; checkSummary.Text = T("尚未開始檢查。", "Not checked yet."); content.Controls.Add(checkSummary); + } + private void BuildConfirm() + { + var content = Stack(); pages[2].Controls.Add(content); + content.Controls.Add(L(T("準備好,開始匯入這份資料", "Review before starting this import"), UiKit.Subtitle)); + confirmSummary.Name = "importConfirmSummary"; confirmSummary.AutoSize = true; content.Controls.Add(confirmSummary); + skip.Name = "importSkipErrors"; skip.AutoSize = true; skip.Text = T("遇到 SQL 錯誤時,略過失敗 GO 批次並繼續(變更後須重新檢查)", "Skip failed GO batches on eligible SQL errors (recheck after changing)"); + skip.AccessibleName = skip.Text; skip.Checked = false; skip.CheckedChanged += (s, e) => InvalidatePrepared(); content.Controls.Add(skip); + content.Controls.Add(L(T("略過不會撤回該批已生效的語句。timeout、斷線、交易/LOB 狀態不明及 log 寫入失敗仍停止。", "Skipping does not undo statements already applied. Timeout, disconnection, uncertain transaction/LOB state and log failures still stop."))); + confirm.Name = "importConfirmTarget"; confirm.AutoSize = true; confirm.Text = T("已確認目標資料庫,並了解異常時可能保留部分資料。", "I confirm the target and understand that partial writes may remain."); + confirm.AccessibleName = confirm.Text; confirm.CheckedChanged += (s, e) => UpdateButtons(); content.Controls.Add(confirm); + confirmPunky.Name = "importConfirmPunky"; confirmPunky.Size = new Size(200, 200); content.Controls.Add(confirmPunky); + } + private void BuildExecution() + { + var content = Stack(); pages[3].Controls.Add(content); + stage.Name = "importStage"; stage.Font = UiKit.Subtitle; coverage.Name = "importCoverage"; position.Name = "importPosition"; elapsed.Name = "importElapsed"; eta.Name = "importEta"; + resultText.Name = "importResult"; resultText.AutoSize = true; resultText.Visible = false; + var statsRow = new TableLayoutPanel { Name = "importStatsLayout", AutoSize = true, ColumnCount = 2, RowCount = 1, Width = 1000 }; + statsRow.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 72)); statsRow.ColumnStyles.Add(new ColumnStyle(SizeType.Percent, 28)); + var stats = Stack(); statsRow.Controls.Add(stats, 0, 0); content.Controls.Add(statsRow); + foreach (var label in new[] { stage, coverage, position, elapsed, eta }) { label.AutoSize = true; label.Margin = new Padding(0, 4, 0, 4); stats.Controls.Add(label); } + bar.Name = "importProgress"; bar.Width = 650; bar.Height = 10; stats.Controls.Add(bar); + var mascot = new FlowLayoutPanel { AutoSize = true, WrapContents = true, FlowDirection = FlowDirection.LeftToRight }; + punky.Name = "importPunky"; punky.Size = new Size(160, 160); mascot.Controls.Add(punky); + Configure(mute, "importMute", T("靜音", "Mute"), (s, e) => { punky.Muted = !punky.Muted; mute.Text = mute.AccessibleName = punky.Muted ? T("開啟聲音", "Unmute") : T("靜音", "Mute"); }); + mute.Enabled = punky.AudioAvailable; mute.AccessibleDescription = T("每次實際執行只在首輪播放一次;取消靜音不會重播。", "Sound plays once per execution job; unmuting never replays it."); mascot.Controls.Add(mute); + statsRow.Controls.Add(mascot, 1, 0); content.Controls.Add(resultText); + content.Controls.Add(L(T("錯誤紀錄 · 最近預覽與本機 log", "Error log · recent preview and local log"), UiKit.Subtitle)); + logPath.Name = "importLogPath"; logPath.ReadOnly = true; logPath.Multiline = true; logPath.WordWrap = true; logPath.ScrollBars = ScrollBars.Vertical; logPath.Height = 42; logPath.Width = 800; logPath.AccessibleName = T("完整 log 路徑", "Full log path"); content.Controls.Add(logPath); + errors.Name = "importErrorText"; errors.ReadOnly = true; errors.Multiline = true; errors.ScrollBars = ScrollBars.Both; errors.WordWrap = false; errors.Width = 800; errors.Height = 115; errors.MaxLength = 32768; + errors.AccessibleName = T("最近錯誤紀錄(唯讀)", "Recent errors (read only)"); content.Controls.Add(errors); + errorCount.Name = "importErrorCount"; errorCount.AutoSize = true; content.Controls.Add(errorCount); + var actions = new FlowLayoutPanel { AutoSize = true, WrapContents = true }; + Configure(openLog, "importOpenLog", T("開啟 log", "Open log"), (s, e) => OpenLogPath(false)); + Configure(openFolder, "importOpenFolder", T("開啟資料夾", "Open folder"), (s, e) => OpenLogPath(true)); + Configure(copyPath, "importCopyPath", T("複製路徑", "Copy path"), (s, e) => { + if (!HasLog()) return; + try { if (ClipboardWriter != null) ClipboardWriter(logPath.Text); else Clipboard.SetText(logPath.Text); status.Text = T("已複製 log 路徑。", "Log path copied."); } + catch (Exception) { status.Text = T("無法複製路徑,請選取上方完整路徑手動複製。", "Cannot copy; select the full path above and copy it manually."); } + }); actions.Controls.AddRange(new Control[] { openLog, openFolder, copyPath }); content.Controls.Add(actions); + } + private void ReadFileSize() + { + fileBytes = 0; try { if (File.Exists(source.Text)) fileBytes = new FileInfo(source.Text).Length; } catch (Exception) { } + facts.Text = fileBytes.ToString("N0") + T(" bytes;編碼與來源摘要待預檢。", " bytes; encoding and source summary await preflight."); UpdateButtons(); + } + private void Reflow() + { + foreach (var page in pages) + { + int width = Math.Max(180, page.ClientSize.Width - 28); + foreach (Control container in page.Controls) + { + container.MaximumSize = new Size(width, 0); + foreach (Control c in container.Controls) + { + if (c is Label || c is CheckBox) c.MaximumSize = new Size(Math.Max(160, width - 8), 0); + if (c is TextBox || c is ProgressBar) c.Width = Math.Max(160, width - 8); + if (c is FlowLayoutPanel) c.MaximumSize = new Size(width, 0); + if (c.Name == "importStatsLayout") { + c.Width = width; c.MaximumSize = new Size(width, 0); int statWidth = Math.Max(120, (int)(width * .69)); + foreach (var label in new[] { stage, coverage, position, elapsed, eta }) label.MaximumSize = new Size(statWidth, 0); + bar.Width = statWidth; int side = Math.Max(80, Math.Min(160, (int)(width * .23))); punky.Size = new Size(side, side); + } + } + } + } + } + private void ShowPage() + { + for (int i = 0; i < 4; i++) { pages[i].Visible = step == i; if (step == i) pages[i].BringToFront(); } + steps.Text = T("① 檔案與目標 → ② 匯入檢查 → ③ 確認執行 → ④ 執行與結果", "① File and target → ② Checks → ③ Confirm → ④ Run and result"); + if (step == 2) status.Text = T("請確認實際目標與錯誤政策後開始匯入;開始會修改目標資料庫。", "Confirm actual target and policy before starting; execution changes the database."); + confirmPunky.SetRunning(step == 2 && !closing); Reflow(); UpdateButtons(); + } + private void UpdateButtons() + { + if (disposing) return; + next.Visible = step < 3 || !busy && Result != null; + next.Enabled = !busy && !closing && (step == 0 ? !string.IsNullOrWhiteSpace(source.Text) && !string.IsNullOrWhiteSpace(database) : step == 1 ? prepared?.Passed == true : step == 2 ? prepared?.Passed == true && confirm.Checked : Result != null); + next.Text = next.AccessibleName = step == 2 ? T("開始匯入", "Start import") : step == 3 ? T("重新檢查匯入", "Recheck import") : T("下一步 →", "Next →"); + back.Visible = step > 0 && step < 3; back.Enabled = !busy && !closing; + recheck.Visible = step == 1 || step == 2; recheck.Enabled = !busy && !closing; + cancel.Enabled = !cancelRequested; cancel.Text = cancel.AccessibleName = busy ? cancelRequested ? T("正在停止…", "Stopping…") : T("取消匯入", "Cancel import") : T("關閉", "Close"); + browse.Enabled = source.Enabled = skip.Enabled = !busy && !closing; confirm.Enabled = !busy && prepared?.Passed == true; + bool hasLog = HasLog(); openLog.Enabled = openFolder.Enabled = copyPath.Enabled = hasLog; + } + private void InvalidatePrepared() + { + if (executing || disposing) return; + lock (progressGate) { generation++; latest = null; activeJob = Guid.Empty; } + if (busy) { cancelRequested = true; SignalCancellation(); } + DiscardPrepared(); confirm.Checked = false; Result = null; + checkSummary.Text = T("檔案或錯誤政策已變更,請重新檢查。", "File or policy changed; recheck required."); + confirmSummary.Text = T("舊預檢已失效。重新檢查後,請再確認實際目標。", "Previous preflight is invalid. Recheck and confirm the actual target."); UpdateButtons(); + } + private void DiscardPrepared() + { + var old = prepared; prepared = null; request = null; service = null; if (old == null) return; + try { old.Dispose(); } + catch (Exception) { if (!disposing) status.Text = T("預檢資源釋放時發生異常,請保留 log 並重新檢查。", "Preflight resource release failed; retain the log and recheck."); } + } + private void BeginOperation(MssqlImportRequest current, bool run) + { + busy = true; executing = run; terminalRendered = false; cancelRequested = false; cancellation = new CancellationTokenSource(); cancellationDispatch = null; clock.Restart(); + lock (progressGate) { acceptingProgress = true; latest = null; activeJob = current.JobId; confirmedBytes = 0; lastAdvance = Stopwatch.GetTimestamp(); } + timer.Start(); UpdateButtons(); + } + private void Publish(MssqlImportProgress progress, int version, Guid job) + { + if (progress == null || progress.JobId != job) return; + lock (progressGate) + { + if (disposing || !acceptingProgress || version != generation || activeJob != job) return; + if (progress.ProcessedBytes > confirmedBytes) { confirmedBytes = progress.ProcessedBytes; lastAdvance = Stopwatch.GetTimestamp(); } + latest = progress; + } + } + private bool Current(int version, Guid job) { lock (progressGate) return !disposing && !IsDisposed && version == generation && activeJob == job; } + private async Task CheckAsync() + { + if (busy || closing || disposing) return; + DiscardPrepared(); confirm.Checked = false; Result = null; errors.Text = ""; resultText.Text = ""; resultText.Visible = false; logPath.Text = ""; scanBar.Value = 0; + lock (progressGate) generation++; + var current = new MssqlImportRequest(Guid.NewGuid(), source.Text, database, skip.Checked); int version = generation; + step = 1; checkSummary.ForeColor = ThemeManager.TextColor; checkSummary.Text = T("正在串流分析檔案…", "Analyzing file by streaming…"); status.Text = checkSummary.Text; + ShowPage(); BeginOperation(current, false); var owner = cancellation; var token = owner.Token; + try + { + var value = await Task.Run(() => { + var worker = serviceFactory(); return Tuple.Create(worker, worker.Preflight(current, p => Publish(p, version, current.JobId), token)); + }); + if (!Current(version, current.JobId) || token.IsCancellationRequested || cancelRequested || closing) { + value.Item2.Dispose(); + if (Current(version, current.JobId) && !closing) status.Text = checkSummary.Text = T("檢查已停止,尚未執行 SQL。", "Check stopped; no source SQL executed."); + return; + } + service = value.Item1; prepared = value.Item2; request = current; RenderCheck(prepared); + } + catch (Exception) { if (Current(version, current.JobId)) status.Text = checkSummary.Text = token.IsCancellationRequested ? T("檢查已停止,尚未執行 SQL。", "Check stopped; no source SQL executed.") : T("無法完成預檢,請確認檔案與連線設定後重試。", "Preflight failed. Review file and connection settings and retry."); } + finally { await EndOperation(owner); } + } + private void RenderCheck(MssqlImportPreflight check) + { + string details = ""; + if (check.Scan != null) + { + var scan = check.Scan; fileBytes = scan.FileBytes; scanBar.Style = ProgressBarStyle.Continuous; scanBar.Value = 100; + details += source.Text + "\r\n" + scan.FileBytes.ToString("N0") + " bytes · " + scan.EncodingName + " · " + scan.BatchCount.ToString("N0") + T(" 批次", " batches") + "\r\n"; + details += T("最大批次:", "Largest batch: ") + scan.MaxBatchCodeUnits.ToString("N0") + " UTF-16 code units\r\n"; + details += ScriptSummary(scan) + "\r\n"; facts.Text = details; + if (string.Equals(scan.Script.SourceReadMode, "READ COMMITTED", StringComparison.OrdinalIgnoreCase)) details += T("來源 READ COMMITTED:各表可能來自不同時間點,匯入可能遇到外鍵衝突;完整摘要不保證跨表一致。", "Source READ COMMITTED: tables may reflect different moments; foreign keys may fail. A complete footer does not prove consistency.") + "\r\n"; + } + if (check.Target != null) + { + var t = check.Target; target.Text = t.ServerName + " / " + t.DatabaseName + " · ID " + t.DatabaseId + " · " + t.DatabaseCreateDate.ToString("yyyy-MM-dd HH:mm:ss", CultureInfo.InvariantCulture); + details += T("實際目標:", "Actual target: ") + target.Text + "\r\nSQL Server " + t.Version + " · " + (t.Writable ? "READ_WRITE" : "READ_ONLY") + "\r\n"; + details += T("可見使用者物件:", "Visible user objects: ") + (t.MetadataVisible && t.UserObjectCount.HasValue ? t.UserObjectCount.Value.ToString("N0") : T("無法完整確認", "not fully known")) + "\r\n"; + } + details += string.Join("\r\n", check.Issues.Select(i => (i.IsWarning ? T("提醒:", "Warning: ") : T("阻擋:", "Blocked: ")) + Explain(i.Code) + (i.BatchNumber > 0 ? " · " + T("批次 ", "batch ") + i.BatchNumber + T("/行 ", "/line ") + i.Line + " / byte " + i.ByteOffset : ""))); + checkSummary.Text = details; checkSummary.ForeColor = check.Passed ? ThemeManager.TextColor : ThemeManager.DangerColor; + status.Text = check.Passed ? T("檢查通過,尚未執行 SQL。請到下一步確認。", "Preflight passed; no source SQL executed. Continue to review.") : T("有未通過的項目,暫時無法開始匯入。", "Preflight has blocking issues; import cannot start."); + confirmSummary.Text = details + "\r\n\r\n" + T("完整 log:", "Full log: ") + check.LogPath + "\r\n\r\n" + PartialWarning(); + logPath.Text = check.LogPath ?? ""; UpdateButtons(); Reflow(); + } + private static string ScriptSummary(MssqlImportScan scan) + { + var script = scan.Script; + if (script.Kind == MssqlImportScriptKind.Generic) return T("一般 T-SQL/GO;無匯出筆數摘要。", "Generic T-SQL/GO; no export row summary."); + return (script.Kind == MssqlImportScriptKind.NativeSchema ? T("mySQLPunk 結構匯出", "mySQLPunk schema export") : T("mySQLPunk 資料型匯出", "mySQLPunk data-only export")) + " · " + (script.ExpectedTables?.ToString("N0") ?? "?") + T(" 表/", " tables / ") + (script.ExpectedRows?.ToString("N0") ?? "?") + T(" 筆(檔尾預期摘要,非內容證明)", " rows (footer expectations, not content proof)") + " · " + script.SourceReadMode; + } + private async Task RunAsync() + { + if (busy || closing || disposing || prepared?.Passed != true || !confirm.Checked || request == null || service == null) return; + var current = request; var check = prepared; var worker = service; int version = generation; + Result = null; errors.Text = resultText.Text = ""; resultText.Visible = false; step = 3; ShowPage(); BeginOperation(current, true); + var owner = cancellation; var token = owner.Token; stage.ForeColor = ThemeManager.TextColor; stage.Text = status.Text = T("正在執行匯入…", "Import executing…"); + coverage.Text = T("已處理檔案範圍:0 / ", "Processed file coverage: 0 / ") + check.Scan.FileBytes.ToString("N0") + " bytes"; + position.Text = T("目標:", "Target: ") + target.Text + "\r\n" + T("正在確認執行 session,尚未確認任何批次。", "Checking execution session; no batch acknowledged yet."); + elapsed.Text = T("已經過:00:00:00", "Elapsed: 00:00:00"); eta.Text = T("預估剩餘:未知/計算中", "Remaining: unknown / estimating"); + errorCount.Text = T("目前沒有錯誤;完整 log 路徑見上方。", "No errors yet; full log path above."); bar.Style = ProgressBarStyle.Continuous; bar.Value = 0; + punky.BeginExport(); // Only execution starts sound; scanning/navigation uses silent animation. + try + { + try + { + Result = await Task.Run(() => worker.Run(current, check, p => Publish(p, version, current.JobId), token)); + } + catch (Exception) + { + Result = new MssqlImportResult(current.JobId, MssqlImportStatus.CleanupUncertain, check.Scan.FileBytes, + logPath: check.LogPath, validationSummary: "UnexpectedWorkerFailure;CleanupUnconfirmed"); + } + // Rendering cannot replace an authoritative worker result with a synthetic outcome. + if (Current(version, current.JobId)) { RenderHeartbeat(); RenderResult(Result); } + } + finally + { + try { check.Dispose(); } + finally { prepared = null; request = null; service = null; await EndOperation(owner); } + } + } + private async Task EndOperation(CancellationTokenSource owner) + { + // Provider Cancel may wait synchronously. Keep the same CTS alive until its dispatch returns. + if (cancellationDispatch != null) { try { await cancellationDispatch; } catch (Exception) { /* Worker result remains authoritative. */ } } + lock (progressGate) { acceptingProgress = false; latest = null; activeJob = Guid.Empty; } + owner.Dispose(); if (ReferenceEquals(cancellation, owner)) cancellation = null; + busy = executing = false; cancelRequested = false; clock.Stop(); if (disposing || IsDisposed) return; + timer.Stop(); punky.EndExport(); UpdateButtons(); if (closing) Close(); + } + private void RequestCancel() + { + if (!busy || terminalRendered || cancelRequested || cancellation == null) return; + cancelRequested = true; status.Text = stage.Text = T("正在停止匯入,等待指令回應與清理完成…", "Stopping import; waiting for command response and cleanup…"); + eta.Text = T("預估剩餘:未知(正在停止)", "Remaining: unknown (stopping)"); RenderHeartbeat(); SignalCancellation(); UpdateButtons(); + } + private void SignalCancellation() + { + if (cancellation == null || cancellationDispatch != null) return; + var owner = cancellation; + cancellationDispatch = Task.Run(() => owner.Cancel()); + } + private void RenderHeartbeat() + { + if (disposing || IsDisposed || !busy || terminalRendered) return; + MssqlImportProgress p; long advance; lock (progressGate) { p = latest; advance = lastAdvance; } + if (p != null) + { + if (executing) + { + stage.Text = cancelRequested ? T("正在停止匯入,等待清理…", "Stopping import; awaiting cleanup…") : StageText(p); + coverage.Text = T("已處理檔案範圍:", "Processed file coverage: ") + p.ProcessedBytes.ToString("N0") + " / " + p.FileBytes.ToString("N0") + " bytes · " + T("成功批次 ", "successful batches ") + p.SuccessfulBatches.ToString("N0") + T(";失敗/略過 ", "; failed/skipped ") + p.FailedBatches.ToString("N0") + "\r\n" + T("成功範圍 ", "Successful coverage ") + p.SuccessfulBytes.ToString("N0") + T(" bytes;略過範圍 ", " bytes; skipped coverage ") + p.SkippedBytes.ToString("N0") + " bytes"; + position.Text = T("目標:", "Target: ") + target.Text + "\r\n" + (p.BatchNumber > 0 ? T("目前批次 ", "Current batch ") + p.BatchNumber.ToString("N0") + T(" · 行 ", " · lines ") + p.StartLine.ToString("N0") + "–" + p.EndLine.ToString("N0") + " · byte " + p.StartByte.ToString("N0") + "–" + p.EndByte.ToString("N0") : p.Stage == MssqlImportStage.Executing ? T("正在確認執行 session,尚未確認任何批次。", "Checking execution session; no batch acknowledged yet.") : T("正在核對或收尾;請以最終結果為準。", "Verifying or finishing; await the terminal result.")); + bar.Style = ProgressBarStyle.Continuous; bar.Value = Percent(p.ProcessedBytes, p.FileBytes); + if (errors.Text != p.RecentErrorText) errors.Text = p.RecentErrorText; + logPath.Text = p.LogPath ?? logPath.Text; + errorCount.Text = (p.LogTruncated ? T("已截段;僅保留最近最多 100 筆/64 KiB UTF-16 文字。", "Truncated; recent at most 100 events / 64 KiB UTF-16 text.") : T("最近錯誤紀錄。", "Recent error events.")) + T(" 共 ", " Total ") + p.TotalErrors.ToString("N0") + T(" 筆;完整 log 路徑見上方。", " events; full log path above."); + } + else { + checkSummary.Text = T("正在串流分析:", "Streaming analysis: ") + p.ScannedBytes.ToString("N0") + " / " + fileBytes.ToString("N0") + T(" bytes;尚未執行 SQL。", " bytes; no source SQL executed."); + scanBar.Style = ProgressBarStyle.Continuous; scanBar.Value = Percent(p.ScannedBytes, fileBytes); + } + } + elapsed.Text = T("已經過:", "Elapsed: ") + Duration(clock.Elapsed); + bool stale = (Stopwatch.GetTimestamp() - advance) / (double)Stopwatch.Frequency >= 5; + bool known = executing && !cancelRequested && !stale && p != null && p.Stage == MssqlImportStage.Executing && p.Phase != MssqlImportPhase.Indexes && p.Phase != MssqlImportPhase.Constraints && p.Remaining.HasValue && p.EstimatedFinish.HasValue; + eta.Text = known ? T("預估剩餘:約 ", "Remaining: about ") + Duration(p.Remaining.Value) + T(" · 預計完成 ", " · Estimated finish ") + p.EstimatedFinish.Value.ToLocalTime().ToString("HH:mm:ss", CultureInfo.InvariantCulture) : T("預估剩餘:未知/計算中(目前指令或核對中)", "Remaining: unknown / estimating (current command or verification)"); UpdateButtons(); + } + private void RenderResult(MssqlImportResult result) + { + bool truncated; + lock (progressGate) truncated = latest != null && latest.JobId == result.JobId && latest.LogTruncated; + terminalRendered = true; stage.Text = status.Text = StatusText(result.Status); + stage.ForeColor = result.Status == MssqlImportStatus.Completed ? ThemeManager.SuccessColor : result.Status == MssqlImportStatus.CompletedWithErrors || result.Status == MssqlImportStatus.Cancelled ? ThemeManager.WarningColor : ThemeManager.DangerColor; + elapsed.Text = T("已經過:", "Elapsed: ") + Duration(result.Elapsed); eta.Text = T("預估剩餘:未知(作業已結束)", "Remaining: unknown (job ended)"); logPath.Text = result.LogPath ?? logPath.Text; + resultText.Visible = true; resultText.Text = StatusText(result.Status) + "\r\n" + T("目標:", "Target: ") + target.Text + "\r\n" + T("成功批次:", "Successful batches: ") + result.SuccessfulBatches.ToString("N0") + T(";失敗/略過:", "; failed/skipped: ") + result.FailedBatches.ToString("N0") + "\r\n" + T("已處理檔案範圍:", "Processed file coverage: ") + result.ProcessedBytes.ToString("N0") + " / " + result.FileBytes.ToString("N0") + " bytes" + "\r\n" + T("最後確認批次:", "Last acknowledged batch: ") + result.LastAcknowledgedBatch.ToString("N0") + T(";未確認批次:", "; uncertain batch: ") + (result.UncertainBatch?.ToString("N0") ?? T("無", "none")) + "\r\n" + string.Join("\r\n", (result.ValidationSummary ?? "ImportFailed").Split(';').Select(Explain)) + "\r\n" + CleanupText(result.Cleanup) + "\r\n" + PartialWarning(); + coverage.Text = T("已處理檔案範圍:", "Processed file coverage: ") + result.ProcessedBytes.ToString("N0") + " / " + result.FileBytes.ToString("N0") + " bytes"; bar.Style = ProgressBarStyle.Continuous; bar.Value = Percent(result.ProcessedBytes, result.FileBytes); + errorCount.Text = (truncated ? T("已截段;最近最多 100 筆/64 KiB UTF-16。", "Truncated; recent at most 100 events / 64 KiB UTF-16.") : T("最近錯誤紀錄。", "Recent errors.")) + T(" 共 ", " Total ") + result.TotalErrors.ToString("N0") + T(" 筆;完整 log 路徑見上方。", " events; full log path above."); Reflow(); + } + private static int Percent(long bytes, long total) => total <= 0 ? 0 : (int)Math.Max(0, Math.Min(100, 100.0 * bytes / total)); + private static string Duration(TimeSpan value) => ((long)Math.Max(0, value.TotalHours)).ToString("00") + ":" + value.Minutes.ToString("00") + ":" + value.Seconds.ToString("00"); + private static string PartialWarning() => T("已執行的結構與資料可能保留,目前批次也可能部分生效;沒有整份檔案的自動回復或續跑。請先檢查目標與 log,再決定如何重新匯入。", "Applied schema/data and partial current-batch effects may remain. No whole-file undo or resume. Review the target and log before importing again."); + private static string StatusText(MssqlImportStatus status) + { + switch (status) + { + case MssqlImportStatus.Completed: return T("匯入執行完成", "Import execution completed"); + case MssqlImportStatus.CompletedWithErrors: return T("執行完成,但有錯誤", "Execution completed with errors"); + case MssqlImportStatus.Cancelled: return T("這次匯入已取消", "Import cancelled"); + case MssqlImportStatus.CleanupUncertain: return T("清理或執行結果尚未確認", "Cleanup or execution outcome is unconfirmed"); + default: return T("匯入已停止:發生錯誤", "Import stopped: error"); + } + } + private static string StageText(MssqlImportProgress p) + { + if (p.Stage == MssqlImportStage.Stopping) return T("正在停止匯入,等待清理…", "Stopping import; awaiting cleanup…"); + if (p.Stage == MssqlImportStage.Verifying) return T("正在核對 session、檔案與必要摘要…", "Verifying session, file and applicable summary…"); + if (p.Stage == MssqlImportStage.Finished) return T("正在整理執行結果…", "Preparing terminal result…"); + return p.Phase == MssqlImportPhase.Indexes ? T("正在建立索引…", "Creating indexes…") : p.Phase == MssqlImportPhase.Constraints ? T("正在建立約束…", "Creating constraints…") : p.Phase == MssqlImportPhase.Schema ? T("正在執行結構批次…", "Executing schema batches…") : T("正在依序執行 SQL 批次…", "Executing SQL batches in order…"); + } + private static string CleanupText(MssqlImportCleanup cleanup) + { + if (cleanup == null) return T("清理狀態:尚未取得確認。", "Cleanup: no confirmed receipt."); + return T("指令已結束:", "Command ended: ") + Yes(cleanup.CommandEnded) + T(";session 已釋放:", "; session released: ") + Yes(cleanup.SessionReleased) + T(";未提交交易清理已確認:", "; transaction cleanup confirmed: ") + Yes(cleanup.TransactionCleanupConfirmed) + (string.IsNullOrWhiteSpace(cleanup.IssueCode) ? "" : "\r\n" + string.Join("\r\n", cleanup.IssueCode.Split(';').Select(Explain))); + } + private static string Yes(bool value) => value ? T("是", "yes") : T("尚未確認", "unconfirmed"); + internal static string Explain(string code) + { + switch (code) + { + case "SourceRequired": return T("請先選擇 SQL 檔案。", "Choose a SQL file."); + case "TargetRequired": return T("請先選擇目標資料庫。", "Select a target database."); + case "TargetReadOnly": return T("目標資料庫為唯讀,無法匯入。", "Target is read only."); + case "TargetIdentityUnavailable": return T("無法確認實際伺服器與資料庫識別。", "Actual server/database identity unavailable."); + case "TargetVisibilityRequired": return T("無法完整確認目標物件;結構匯出須確認空庫。", "Complete visibility is required to prove an empty schema target."); + case "TargetNotEmpty": return T("結構匯出需要空目標;請另選資料庫,精靈不會清空物件。", "Schema import requires an empty target; select another database."); + case "CreateTableRequired": return T("無法確認建立資料表的必要權限。", "Required CREATE TABLE permission unconfirmed."); + case "CreateTableNotProven": case "CreateViewNotProven": return T("建立物件權限未能完整確認,實際 SQL 仍可能失敗。", "Creation permissions not fully known; SQL may still fail."); + case "TargetNotProvenEmpty": return T("目標未確認為空;一般或資料型腳本依自身 SQL 與權限執行。", "Target not proven empty; generic/data SQL uses existing permissions."); + case "SessionIntrinsicsMayChange": return T("批次間狀態查詢可能改變 @@ROWCOUNT/@@ERROR;需保留時請在同一 batch 先自行保存。", "Interbatch probes may change @@ROWCOUNT/@@ERROR; preserve them in the same batch if needed."); + case "PartialBatchWritesMayRemain": return PartialWarning(); + case "Cancelled": case "PreviouslyCommittedWritesMayRemain": return T("已停止排程;先前已提交的寫入可能保留。", "Scheduling stopped; previously committed writes may remain."); + case "LogIncomplete": return T("完整錯誤紀錄未確認存妥。請保留畫面內容與現有 log,後續批次已停止。", "Full log persistence unconfirmed. Retain preview and existing log; later batches stopped."); + case "SqlBatchFailed": return T("SQL 批次失敗;請查看下方錯誤與完整 log。", "SQL batch failed; inspect errors and full log."); + case "AllBatchesExecuted": return T("所有批次已完成執行。", "All batches executed."); + case "NoDataContentValidation": return T("一般/資料型腳本未提供可比較的完整摘要;未核對資料內容。", "Generic/data SQL has no comparable full manifest; data contents not verified."); + case "AllBatchesProcessedWithErrors": case "FullRestoreNotValidated": return T("已遍歷檔案但含失敗/略過批次,未通過完整還原核對。", "File processed with failed/skipped batches; full restore not validated."); + case "NativeSummaryCountsMatched": return T("native 摘要的資料表數與總筆數相符。", "Native footer table and total-row counts match."); + case "NotColumnContentValidation": return T("此核對僅限表數與總筆數,不代表逐欄資料內容相同。", "Count-only validation; column contents were not compared."); + case "TargetChanged": return T("目標識別已變更,作業停止。", "Target identity changed; job stopped."); + case "DigestMismatch": return T("檔案 digest 與預掃描不符,已執行部分無法自動撤回。", "File digest differs from preflight; applied changes cannot be undone."); + case "SummaryMismatch": return T("資料表數或總筆數與 native 摘要不符。", "Counts differ from native summary."); + case "EofTransactionOpen": return T("檔尾仍有未結交易,不能視為成功。", "An open transaction remains at EOF."); + case "SessionStateInvalid": return T("無法確認 session/交易狀態,作業停止。", "Session/transaction state invalid; stopped."); + case "LobStateInvalid": case "LobInitializationMissing": return T("native LOB 傳輸狀態不完整或不一致,不能略過。", "Native LOB state missing/inconsistent; cannot skip."); + case "PreflightInvalidOrConsumed": return T("預檢已失效或已使用,請重新檢查。", "Preflight invalid/consumed; recheck."); + case "EmbeddedNul": return T("v1 不支援 SQL 文字中的 U+0000;也請確認是否為無 BOM 的 UTF-16。", "v1 does not support U+0000 in SQL text; check for BOM-less UTF-16."); + case "BatchTooLarge": return T("單一 GO 批次超過 32 MiB 解碼上限,請調整來源匯出批次後重產檔案。", "GO batch exceeds 32 MiB decoded; regenerate smaller batches."); + case "UnsupportedGo": case "UnsupportedGoSyntax": return T("GO 語法不支援:不可重複次數或在分隔行使用 block comment。", "Unsupported GO repeat count or block comment delimiter."); + case "SqlCmdUnsupported": case "SqlcmdUnsupported": case "UnsupportedSqlCmd": case "UnsupportedSqlcmd": return T("不支援 SQLCMD 指令或變數替代。", "SQLCMD directives/substitution unsupported."); + case "InvalidEncoding": case "UnsupportedEncoding": return T("檔案編碼無效或不支援;需要 UTF-8 或帶 BOM 的 UTF-16。", "Invalid encoding; use UTF-8 or BOM-marked UTF-16."); + case "DatabaseChange": case "CrossDatabaseUse": case "UnsupportedUse": return T("不支援直接 USE 切換至另一資料庫。", "Direct USE to another database unsupported."); + case "CleanupStillActive": case "SessionUnavailable": case "TransactionCleanupUnconfirmed": case "TransactionCleanupFailed": case "CleanupUnconfirmed": case "CleanupFailed": case "SessionReleaseFailed": case "CommandStillActive": case "CleanupReceiptMissing": case "ExecutionOutcomeUnknown": case "UnexpectedWorkerFailure": return T("執行或清理結果尚未確認,請依 log 核對目標。", "Execution/cleanup unconfirmed; inspect target using log."); + case "NativeFormat": return T("mySQLPunk 匯出標記或 LOB job/完成摘要不完整或不一致。", "Native export markers, LOB job or completion summary are missing/inconsistent."); + case "UnclosedContext": return T("檔案結尾有未閉合的字串、識別字或註解。", "Unclosed string, identifier or comment at EOF."); + case "PreflightFailed": return T("無法完成預檢,請確認檔案可讀、連線設定與 log 目錄可寫。", "Preflight failed; check readable file, connection and writable log directory."); + case "ImportFailed": return T("匯入因檔案、連線或執行異常停止,請核對 log 與目標。", "Import stopped on file/connection/execution error; inspect log and target."); + case "InputReleaseFailed": return T("來源檔案 handle 釋放未確認。", "Source handle release unconfirmed."); + default: return T("檢查或執行未完成,診斷代碼:", "Check/execution incomplete; diagnostic code: ") + code; + } + } + private bool HasLog() => !string.IsNullOrWhiteSpace(logPath.Text) && File.Exists(logPath.Text); + private void OpenLogPath(bool folder) + { + if (!HasLog()) return; string path = folder ? Path.GetDirectoryName(logPath.Text) : logPath.Text; + try { if (PathOpener != null) PathOpener(path); else Process.Start(new ProcessStartInfo(path) { UseShellExecute = true }); } + catch (Exception) { status.Text = T("無法開啟路徑,請使用上方完整路徑。", "Cannot open path; use full path above."); } + } + protected override void Dispose(bool disposingManaged) + { + if (disposingManaged && !disposing) + { + lock (progressGate) { disposing = true; acceptingProgress = false; generation++; latest = null; activeJob = Guid.Empty; } + SignalCancellation(); + DiscardPrepared(); timer.Dispose(); + } + base.Dispose(disposingManaged); + } + } +} diff --git a/mySQLPunk/lib/ConnectionConfigurationService.cs b/mySQLPunk/lib/ConnectionConfigurationService.cs index 000902d..fd1130d 100644 --- a/mySQLPunk/lib/ConnectionConfigurationService.cs +++ b/mySQLPunk/lib/ConnectionConfigurationService.cs @@ -74,6 +74,17 @@ public static string BuildSqlServerConnectionString(Dictionary c return builder.ConnectionString; } + internal static string BuildMssqlImportConnectionString(Dictionary connection) + { + if (connection == null) throw new ArgumentNullException(nameof(connection)); + if (string.IsNullOrWhiteSpace(GetValue(connection, "initial_database"))) + throw new ArgumentException("MSSQL import requires an explicit target database.", nameof(connection)); + var builder = new SqlConnectionStringBuilder(BuildSqlServerConnectionString(connection)); + builder.ConnectRetryCount = 0; + builder.MultipleActiveResultSets = false; + return builder.ConnectionString; + } + public static string BuildMySqlConnectionString(Dictionary connection) { uint port = 3306; diff --git a/mySQLPunk/lib/ConnectionOpenService.cs b/mySQLPunk/lib/ConnectionOpenService.cs index fedfba0..eebb584 100644 --- a/mySQLPunk/lib/ConnectionOpenService.cs +++ b/mySQLPunk/lib/ConnectionOpenService.cs @@ -52,6 +52,11 @@ public static ConnectionOpenResult Open(Func databaseFactory, string } public static ConnectionOpenResult Open(Dictionary connection, bool loadDatabases = true) + { + return Open(connection, loadDatabases, false); + } + + private static ConnectionOpenResult Open(Dictionary connection, bool loadDatabases, bool importOnly) { if (connection == null) throw new ArgumentNullException("connection"); Dictionary effective = new Dictionary(connection); @@ -71,7 +76,9 @@ public static ConnectionOpenResult Open(Dictionary connection, b } string provider = ConnectionConfigurationService.NormalizeProvider(ConnectionConfigurationService.GetValue(effective, "db_kind")); - string connectionString = ConnectionConfigurationService.BuildConnectionString(effective); + string connectionString = importOnly + ? ConnectionConfigurationService.BuildMssqlImportConnectionString(effective) + : ConnectionConfigurationService.BuildConnectionString(effective); IDatabase rawDatabase = ConnectionConfigurationService.CreateDatabase(provider); database = tunnel == null ? rawDatabase : new SecuredDatabase(rawDatabase, tunnel); if (tunnel != null) tunnel = null; @@ -106,6 +113,26 @@ internal static MssqlOwnedSession OpenMssqlSession(Dictionary co } } + internal static MssqlOwnedSession OpenMssqlImportSession(Dictionary connection) + { + if (connection == null) throw new ArgumentNullException(nameof(connection)); + string provider = ConnectionConfigurationService.NormalizeProvider(ConnectionConfigurationService.GetValue(connection, "db_kind")); + if (provider != "mssql") throw new ArgumentException("MSSQL import requires a SQL Server connection", nameof(connection)); + // Validate the selected catalog before starting any SSH lease or opening a connection. + ConnectionConfigurationService.BuildMssqlImportConnectionString(connection); + ConnectionOpenResult result = Open(connection, false, true); + try + { + if (result.NativeMssqlConnection == null) throw new InvalidOperationException("SQL Server provider connection is unavailable"); + return new MssqlOwnedSession(result.Database, result.NativeMssqlConnection); + } + catch + { + try { result.Database.Dispose(); } catch { } + throw; + } + } + private static void ValidateSecurityCombination(Dictionary connection) { if (!ConnectionSecuritySettingsService.IsTrue(connection, "ssh_enabled")) return; diff --git a/mySQLPunk/lib/MssqlExport/MssqlValueCodec.cs b/mySQLPunk/lib/MssqlExport/MssqlValueCodec.cs index 5ce5bb2..cefa1be 100644 --- a/mySQLPunk/lib/MssqlExport/MssqlValueCodec.cs +++ b/mySQLPunk/lib/MssqlExport/MssqlValueCodec.cs @@ -87,7 +87,9 @@ internal static string FormatScalar(object value, MssqlColumnDefinition column) case "real": return ((float)value).ToString("R", Invariant); case "uniqueidentifier": return "'" + ((Guid)value).ToString("D") + "'"; case "date": return "'" + ((DateTime)value).ToString("yyyy-MM-dd", Invariant) + "'"; - case "datetime": case "datetime2": case "smalldatetime": return "'" + ((DateTime)value).ToString("yyyy-MM-dd'T'HH:mm:ss.fffffff", Invariant) + "'"; + // Legacy datetime parsers reject >3 fractional digits; datetime2 retains all seven. + case "datetime": case "smalldatetime": return "'" + ((DateTime)value).ToString("yyyy-MM-dd'T'HH:mm:ss.fff", Invariant) + "'"; + case "datetime2": return "'" + ((DateTime)value).ToString("yyyy-MM-dd'T'HH:mm:ss.fffffff", Invariant) + "'"; case "datetimeoffset": return "'" + ((DateTimeOffset)value).ToString("yyyy-MM-dd'T'HH:mm:ss.fffffffzzz", Invariant) + "'"; case "time": return "'" + ((TimeSpan)value).ToString(@"hh\:mm\:ss\.fffffff", Invariant) + "'"; default: throw new NotSupportedException("No scalar codec for " + column.NativeType + "."); diff --git a/mySQLPunk/lib/MssqlImport/MssqlGoBatchReader.cs b/mySQLPunk/lib/MssqlImport/MssqlGoBatchReader.cs new file mode 100644 index 0000000..43fc317 --- /dev/null +++ b/mySQLPunk/lib/MssqlImport/MssqlGoBatchReader.cs @@ -0,0 +1,268 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Security.Cryptography; +using System.Text; +using System.Threading; + +namespace mySQLPunk.lib.MssqlImport +{ + // A single bounded SQL buffer. The line candidate is a finite-state machine: + // a GO comment (including an arbitrarily long one) is never buffered. + internal sealed class MssqlGoBatchReader : IDisposable + { + internal const int MaxBatchCodeUnits = 16777216; + private enum Candidate { Leading, G, Go, Space, Dash, Comment, Sql } + private enum Context { Normal, Single, Double, Bracket, LineComment, BlockComment } + private readonly Stream stream; + private readonly int bufferSize; + private readonly Action progress; + private readonly Action observe; + private Context context; + private int blockDepth; + private bool skipPair, hasSql, reading; + private long number = 1, line = 1, byteOffset; + internal string EncodingName { get; private set; } + internal string Sha256 { get; private set; } + internal MssqlGoBatchReader(Stream stream, int bufferSize = 8192, Action progress = null, Action observe = null) + { + this.stream = stream ?? throw new ArgumentNullException(nameof(stream)); + if (bufferSize < 1 || bufferSize > 1048576) throw new ArgumentOutOfRangeException(nameof(bufferSize)); + this.bufferSize = bufferSize; this.progress = progress; this.observe = observe; + } + + internal IEnumerable ReadBatches(CancellationToken cancellation) + { + if (reading) throw new InvalidOperationException("Reader is single use."); + reading = true; + var sql = new StringBuilder(); + long startByte = 0, startLine = 1, endLine = 1; + int lineStart = 0; + bool overflow = false; + Candidate candidate = Candidate.Leading; + using (var chars = Decode(cancellation).GetEnumerator()) + { + bool more = chars.MoveNext(); + Unit current = more ? chars.Current : default(Unit); + while (more) + { + cancellation.ThrowIfCancellationRequested(); + // Decode failures are deferred through lookahead, so all preceding + // valid CR/LF and GO boundaries update the source coordinates first. + if (current.DecodeError != null) { byteOffset = current.Start; throw Error("InvalidEncoding", current.DecodeError); } + more = chars.MoveNext(); + Unit next = more ? chars.Current : default(Unit); + char c = current.Character; + byteOffset = current.Start; + if (c == '\0') throw Error("EmbeddedNul"); + if (c == '\r' || c == '\n') + { + string newline = c.ToString(); + long boundary = current.End; + if (c == '\r' && more && next.Character == '\n') + { + newline = "\r\n"; boundary = next.End; + more = chars.MoveNext(); next = more ? chars.Current : default(Unit); + } + if (IsGo(candidate)) + { + sql.Length = lineStart; + var batch = new MssqlImportBatch(number, startByte, boundary, startLine, Math.Max(startLine, line - 1), sql.ToString(), EncodingName); + if (sql.Length != 0) observe?.Invoke(batch); + if (hasSql) { number++; yield return batch; } + sql.Clear(); hasSql = false; startByte = boundary; startLine = line + 1; + } + else + { + if (candidate == Candidate.Dash) throw Error("UnsupportedGo"); + if (overflow) throw Error("BatchTooLarge"); + Append(sql, newline); + if (context == Context.LineComment) context = Context.Normal; + skipPair = false; + endLine = line; + } + line++; lineStart = sql.Length; overflow = false; + candidate = context == Context.Normal ? Candidate.Leading : Candidate.Sql; + current = next; + continue; + } + + if (candidate != Candidate.Sql) + { + Candidate before = candidate; + candidate = Advance(candidate, c); + if (candidate == Candidate.Sql) + { + if (overflow) throw Error("BatchTooLarge"); + // The deferred prefix contains only whitespace/G/O, so has no + // lexical side effects. G/O become SQL if this isn't a GO line. + if (before != Candidate.Leading) hasSql = true; + Append(sql, c); Lex(c, more ? next.Character : '\0'); + } + else if (candidate != Candidate.Comment) + { + if (sql.Length < MaxBatchCodeUnits) sql.Append(c); else overflow = true; + } + } + else { Append(sql, c); Lex(c, more ? next.Character : '\0'); } + endLine = line; + current = next; + } + } + if (IsGo(candidate)) + { + sql.Length = lineStart; + var batch = new MssqlImportBatch(number, startByte, stream.Position, startLine, Math.Max(startLine, line - 1), sql.ToString(), EncodingName); + if (sql.Length != 0) observe?.Invoke(batch); + if (hasSql) yield return batch; + } + else + { + if (candidate == Candidate.Dash) throw Error("UnsupportedGo"); + if (overflow) throw Error("BatchTooLarge"); + if (context != Context.Normal && context != Context.LineComment) throw Error("UnclosedContext"); + var batch = new MssqlImportBatch(number, startByte, stream.Position, startLine, endLine, sql.ToString(), EncodingName); + if (sql.Length != 0) observe?.Invoke(batch); + if (hasSql) yield return batch; + } + } + + private Candidate Advance(Candidate state, char c) + { + if (state == Candidate.Comment) return state; + if (state == Candidate.Leading) return char.IsWhiteSpace(c) ? state : c == 'g' || c == 'G' ? Candidate.G : Candidate.Sql; + if (state == Candidate.G) return c == 'o' || c == 'O' ? Candidate.Go : Candidate.Sql; + if (state == Candidate.Go && (char.IsLetterOrDigit(c) || c == '_' || c == '#' || c == '@' || c == '$')) return Candidate.Sql; + if (state == Candidate.Dash) + { + if (c == '-') return Candidate.Comment; + throw Error("UnsupportedGo"); + } + if (char.IsWhiteSpace(c)) return Candidate.Space; + if (c == '-') return Candidate.Dash; + throw Error("UnsupportedGo"); + } + private static bool IsGo(Candidate candidate) => candidate == Candidate.Go || candidate == Candidate.Space || candidate == Candidate.Comment; + private void Append(StringBuilder sql, char c) { if (sql.Length == MaxBatchCodeUnits) throw Error("BatchTooLarge"); sql.Append(c); } + private void Append(StringBuilder sql, string text) { if ((long)sql.Length + text.Length > MaxBatchCodeUnits) throw Error("BatchTooLarge"); sql.Append(text); } + private MssqlImportInputException Error(string code, Exception inner = null) => new MssqlImportInputException(code, number, line, byteOffset, inner); + private void Lex(char c, char next) + { + if (skipPair) { skipPair = false; return; } + if (context == Context.LineComment) return; + if (context == Context.BlockComment) + { + if (c == '/' && next == '*') { blockDepth++; skipPair = true; } + else if (c == '*' && next == '/') { if (--blockDepth == 0) context = Context.Normal; skipPair = true; } + return; + } + if (context == Context.Single || context == Context.Double || context == Context.Bracket) + { + char close = context == Context.Single ? '\'' : context == Context.Double ? '"' : ']'; + if (c == close) { if (next == close) skipPair = true; else context = Context.Normal; } + return; + } + if (c == '-' && next == '-') { context = Context.LineComment; skipPair = true; } + else if (c == '/' && next == '*') { context = Context.BlockComment; blockDepth = 1; skipPair = true; } + else if (c == '\'') { context = Context.Single; hasSql = true; } + else if (c == '"') { context = Context.Double; hasSql = true; } + else if (c == '[') { context = Context.Bracket; hasSql = true; } + else if (!char.IsWhiteSpace(c)) hasSql = true; + } + + private struct Unit + { + internal char Character; + internal long Start, End; + internal DecoderFallbackException DecodeError; + internal Unit(char character, long start, long end) { Character = character; Start = start; End = end; DecodeError = null; } + internal Unit(long offset, DecoderFallbackException error) { Character = '\0'; Start = End = offset; DecodeError = error; } + } + private IEnumerable Decode(CancellationToken cancellation) + { + var prefix = new byte[4]; + int prefixCount = 0; + while (prefixCount < prefix.Length) + { + cancellation.ThrowIfCancellationRequested(); + int n = stream.Read(prefix, prefixCount, Math.Min(bufferSize, prefix.Length - prefixCount)); + if (n == 0) break; + prefixCount += n; + } + int bom = 0; + Encoding encoding = new UTF8Encoding(false, true); + if (prefixCount >= 4 && ((prefix[0] == 0xff && prefix[1] == 0xfe && prefix[2] == 0 && prefix[3] == 0) || (prefix[0] == 0 && prefix[1] == 0 && prefix[2] == 0xfe && prefix[3] == 0xff))) throw Error("InvalidEncoding"); + if (prefixCount >= 3 && prefix[0] == 0xef && prefix[1] == 0xbb && prefix[2] == 0xbf) bom = 3; + else if (prefixCount >= 2 && prefix[0] == 0xff && prefix[1] == 0xfe) { encoding = new UnicodeEncoding(false, true, true); bom = 2; } + else if (prefixCount >= 2 && prefix[0] == 0xfe && prefix[1] == 0xff) { encoding = new UnicodeEncoding(true, true, true); bom = 2; } + EncodingName = encoding.CodePage == 65001 ? (bom == 0 ? "UTF-8" : "UTF-8 BOM") : encoding.CodePage == 1200 ? "UTF-16 LE BOM" : "UTF-16 BE BOM"; + Decoder decoder = encoding.GetDecoder(); + var buffer = new byte[bufferSize]; + var decoded = new char[Math.Max(4, bufferSize + 2)]; + long decodedByte = bom, chunkStart = bom; + int previousByte = -1; + using (var hash = SHA256.Create()) + { + hash.TransformBlock(prefix, 0, prefixCount, null, 0); + byte[] source = prefix; int offset = bom, count = prefixCount - bom; + bool eof = false; + while (true) + { + cancellation.ThrowIfCancellationRequested(); + int usedBytes = 0, usedChars = 0; bool complete; + // Decode only through the next encoded newline. A failed conversion + // can then hide no earlier line/GO transition in its valid prefix. + // Reads and buffers stay fixed-size; this is not byte-at-a-time decoding. + int span = DecodeSpan(source, offset, count, chunkStart, bom, encoding.CodePage, previousByte); + DecoderFallbackException failure = null; + try { decoder.Convert(source, offset, span, decoded, 0, decoded.Length, eof, out usedBytes, out usedChars, out complete); } + catch (DecoderFallbackException ex) { failure = ex; } + if (failure != null) + { + yield return new Unit(Math.Max(0, chunkStart + failure.Index), failure); + yield break; + } + for (int i = 0; i < usedChars; i++) + { + char c = decoded[i]; + int bytes = encoding.CodePage != 65001 ? 2 : char.IsHighSurrogate(c) ? 0 : char.IsLowSurrogate(c) ? 4 : c < 0x80 ? 1 : c < 0x800 ? 2 : 3; + yield return new Unit(c, decodedByte, decodedByte + bytes); decodedByte += bytes; + } + if (usedBytes != 0) previousByte = source[offset + usedBytes - 1]; + offset += usedBytes; count -= usedBytes; chunkStart += usedBytes; + if (count != 0) continue; + if (eof) + { + hash.TransformFinalBlock(new byte[0], 0, 0); + Sha256 = BitConverter.ToString(hash.Hash).Replace("-", "").ToLowerInvariant(); + progress?.Invoke(stream.Position); + yield break; + } + int read = stream.Read(buffer, 0, buffer.Length); + chunkStart = stream.Position - read; + if (read > 0) { hash.TransformBlock(buffer, 0, read, null, 0); progress?.Invoke(stream.Position); } + source = buffer; offset = 0; count = read; eof = read == 0; + } + } + } + private static int DecodeSpan(byte[] source, int offset, int count, long byteStart, int bom, int codePage, int previousByte) + { + for (int i = 0; i < count; i++) + { + byte value = source[offset + i]; + if (codePage == 65001) + { + if (value == 10 || value == 13) return i + 1; + } + else if (((byteStart + i - bom) & 1) != 0) + { + int prior = i == 0 ? previousByte : source[offset + i - 1]; + if (codePage == 1200 ? value == 0 && (prior == 10 || prior == 13) : prior == 0 && (value == 10 || value == 13)) return i + 1; + } + } + return count; + } + // The caller owns the same file handle across scan and execution. + public void Dispose() { } + } +} diff --git a/mySQLPunk/lib/MssqlImport/MssqlImportInput.cs b/mySQLPunk/lib/MssqlImport/MssqlImportInput.cs new file mode 100644 index 0000000..ff40c14 --- /dev/null +++ b/mySQLPunk/lib/MssqlImport/MssqlImportInput.cs @@ -0,0 +1,58 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Threading; + +namespace mySQLPunk.lib.MssqlImport +{ + internal sealed class MssqlImportInput : IDisposable + { + private readonly FileStream file; + private readonly string databaseName; + private bool disposed, active; + internal string ExecutionSha256 { get; private set; } + private MssqlImportInput(string path, string databaseName) + { + file = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 8192, FileOptions.SequentialScan); + this.databaseName = databaseName; + } + internal static MssqlImportInput Open(string path, string databaseName = null) => new MssqlImportInput(path, databaseName); + internal MssqlImportScan Scan(CancellationToken cancellation, Action progress) + { + Enter(); + try + { + file.Position = 0; ExecutionSha256 = null; + var inspector = new MssqlImportScriptInspector(databaseName); + long count = 0, max = 0; + using (var reader = new MssqlGoBatchReader(file, 8192, progress, inspector.Observe)) + { + foreach (var batch in reader.ReadBatches(cancellation)) { count++; max = Math.Max(max, (long)batch.Text.Length); } + return new MssqlImportScan(file.Length, count, max, reader.EncodingName, reader.Sha256, inspector.Complete()); + } + } + finally { active = false; } + } + internal IEnumerable ReadBatches(CancellationToken cancellation) + { + Enter(); + try + { + file.Position = 0; ExecutionSha256 = null; + using (var reader = new MssqlGoBatchReader(file)) + { + foreach (var batch in reader.ReadBatches(cancellation)) yield return batch; + ExecutionSha256 = reader.Sha256; + } + } + finally { active = false; } + } + private void Enter() + { + if (disposed) throw new ObjectDisposedException(nameof(MssqlImportInput)); + if (active) throw new InvalidOperationException("Input already has an active reader."); + active = true; + } + public void Dispose() { if (disposed) return; disposed = true; file.Dispose(); } + } +} diff --git a/mySQLPunk/lib/MssqlImport/MssqlImportLog.cs b/mySQLPunk/lib/MssqlImport/MssqlImportLog.cs new file mode 100644 index 0000000..8117034 --- /dev/null +++ b/mySQLPunk/lib/MssqlImport/MssqlImportLog.cs @@ -0,0 +1,150 @@ +using System; +using System.Collections.Generic; +using System.Globalization; +using System.IO; +using System.Text; + +namespace mySQLPunk.lib.MssqlImport +{ + internal interface IMssqlImportLog : IDisposable + { + string Path { get; } + // Worker snapshots this tail before publishing immutable UI progress; UI never reads the mutable tail. + MssqlImportLogTail Tail { get; } + void WriteStart(MssqlImportRequest request, MssqlImportScan scan, MssqlImportTarget target); + void WriteError(MssqlImportBatch batch, MssqlImportBatchException error, string action); + void Flush(); + void WriteEnd(MssqlImportResult result); + } + internal sealed class MssqlImportLogTail + { + private readonly Queue entries = new Queue(); + private readonly object gate = new object(); + private int characters; + private long totalErrors; + private bool truncated; + public long TotalErrors { get { lock (gate) return totalErrors; } } + public bool Truncated { get { lock (gate) return truncated; } } + public void Add(string text) + { + lock (gate) + { + totalErrors = checked(totalErrors + 1); + text = text ?? string.Empty; + if (text.Length > 32768) { text = text.Substring(text.Length - 32768); truncated = true; } + entries.Enqueue(text); characters += text.Length; + while (entries.Count > 100 || characters > 32768) + { characters -= entries.Dequeue().Length; truncated = true; } + } + } + public string Snapshot() { lock (gate) return string.Concat(entries); } + } + internal sealed class MssqlImportLog : IMssqlImportLog + { + private readonly Stream stream; + private readonly StreamWriter writer; + private readonly Func utcNow; + private bool disposed; + public string Path { get; } + public MssqlImportLogTail Tail { get; } = new MssqlImportLogTail(); + internal MssqlImportLog(Guid jobId, string directory = null, Func streamFactory = null, Func utcNow = null) + { + if (jobId == Guid.Empty) throw new ArgumentException("A job identity is required.", nameof(jobId)); + this.utcNow = utcNow ?? (() => DateTime.UtcNow); + directory = directory ?? System.IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "mySQLPunk", "ImportLogs"); + directory = System.IO.Path.GetFullPath(directory); + Directory.CreateDirectory(directory); + string stem = "import-" + this.utcNow().ToString("yyyyMMdd-HHmmss-fffffff", CultureInfo.InvariantCulture) + "-" + jobId.ToString("N"); + for (int collision = 0; ; collision++) + { + string suffix = collision == 0 ? string.Empty : "-" + collision.ToString(CultureInfo.InvariantCulture); + string candidate = System.IO.Path.Combine(directory, stem + suffix + ".log"); + try + { + stream = streamFactory != null ? streamFactory(candidate) : new FileStream(candidate, FileMode.CreateNew, FileAccess.Write, FileShare.Read); + Path = candidate; + break; + } + catch (IOException) when (streamFactory == null && File.Exists(candidate)) { } + } + try { writer = new StreamWriter(stream, new UTF8Encoding(false, true), 4096, true) { NewLine = "\r\n" }; } + catch { stream.Dispose(); throw; } + } + public void WriteStart(MssqlImportRequest request, MssqlImportScan scan, MssqlImportTarget target) + { + EnsureOpen(); + Field("StartUtc", utcNow().ToString("O", CultureInfo.InvariantCulture)); + Field("Job", request.JobId); Field("LogPath", Path); Field("SourcePath", request.SourcePath); + Field("FileBytes", scan.FileBytes); Field("Sha256", scan.Sha256); Field("Encoding", scan.EncodingName); + Field("SkipSqlErrors", request.SkipSqlErrors); Field("ServerName", target.ServerName); Field("DatabaseName", target.DatabaseName); + Field("DatabaseId", target.DatabaseId); Field("DatabaseCreateDate", target.DatabaseCreateDate.ToString("O", CultureInfo.InvariantCulture)); + Field("Version", target.Version); Field("Writable", target.Writable); Field("MetadataVisible", target.MetadataVisible); + Field("UserObjectCount", target.UserObjectCount); Field("CanCreateTable", target.CanCreateTable); Field("CanCreateView", target.CanCreateView); + Flush(); // The caller must not start DB writes until this durable header succeeds. + } + public void WriteError(MssqlImportBatch batch, MssqlImportBatchException error, string action) + { + EnsureOpen(); + string timestamp = utcNow().ToString("O", CultureInfo.InvariantCulture); + // Build only a bounded UI suffix. The original provider message is streamed independently. + var preview = new StringBuilder(32769); + foreach (string part in ErrorParts(batch, error, action, timestamp)) + { + int keep = Math.Min(part.Length, 32769); + if (preview.Length + keep > 32769) preview.Remove(0, preview.Length + keep - 32769); + preview.Append(part, part.Length - keep, keep); + } + Tail.Add(preview.ToString()); + foreach (string part in ErrorParts(batch, error, action, timestamp)) writer.Write(part); + } + private static IEnumerable ErrorParts(MssqlImportBatch batch, MssqlImportBatchException error, string action, string timestamp) + { + yield return "ErrorUtc=" + timestamp + " Batch=" + Number(batch.Number) + " StartLine=" + Number(batch.StartLine) + + " EndLine=" + Number(batch.EndLine) + " StartByte=" + Number(batch.StartByte) + " EndByte=" + Number(batch.EndByte) + + " OutcomeUncertain=" + error.OutcomeUncertain + " Action=" + action + "\r\n"; + foreach (var detail in error.Errors) + { + long? sourceLine = batch.StartLine > 0 && detail.RelativeLine > 0 && batch.StartLine <= long.MaxValue - detail.RelativeLine + 1 + ? (long?)(batch.StartLine + ((long)detail.RelativeLine - 1)) : null; + yield return "Number=" + detail.Number.ToString(CultureInfo.InvariantCulture) + " State=" + detail.State.ToString(CultureInfo.InvariantCulture) + + " Class=" + detail.Class.ToString(CultureInfo.InvariantCulture) + " RelativeLine=" + detail.RelativeLine.ToString(CultureInfo.InvariantCulture) + + " SourceLine=" + (sourceLine.HasValue ? Number(sourceLine.Value) : "unknown") + " Message="; + yield return detail.Message; + yield return "\r\n"; + } + } + public void Flush() + { + EnsureOpen(); writer.Flush(); + var file = stream as FileStream; + if (file != null) file.Flush(true); else stream.Flush(); + } + public void WriteEnd(MssqlImportResult result) + { + EnsureOpen(); + Field("EndUtc", utcNow().ToString("O", CultureInfo.InvariantCulture)); Field("Job", result.JobId); Field("Status", result.Status); + Field("FileBytes", result.FileBytes); Field("ProcessedBytes", result.ProcessedBytes); Field("SuccessfulBytes", result.SuccessfulBytes); + Field("SkippedBytes", result.SkippedBytes); Field("SuccessfulBatches", result.SuccessfulBatches); Field("FailedBatches", result.FailedBatches); + Field("LastAcknowledgedBatch", result.LastAcknowledgedBatch); Field("UncertainBatch", result.UncertainBatch); Field("TotalErrors", result.TotalErrors); + Field("LogPath", Path); Field("Elapsed", result.Elapsed); Field("ValidationSummary", result.ValidationSummary); + if (result.Cleanup != null) + { + Field("CommandEnded", result.Cleanup.CommandEnded); Field("SessionReleased", result.Cleanup.SessionReleased); + Field("TransactionCleanupConfirmed", result.Cleanup.TransactionCleanupConfirmed); Field("CleanupIssueCode", result.Cleanup.IssueCode); + } + else Field("Cleanup", "unknown"); + Flush(); + } + private static string Number(long value) { return value.ToString(CultureInfo.InvariantCulture); } + private void Field(string name, object value) + { writer.Write(name); writer.Write('='); writer.WriteLine(value == null ? "unknown" : Convert.ToString(value, CultureInfo.InvariantCulture)); } + private void EnsureOpen() { if (disposed) throw new ObjectDisposedException(nameof(MssqlImportLog)); } + public void Dispose() + { + if (disposed) return; + disposed = true; + try { writer.Dispose(); } + finally { stream.Dispose(); } + } + } +} diff --git a/mySQLPunk/lib/MssqlImport/MssqlImportModels.cs b/mySQLPunk/lib/MssqlImport/MssqlImportModels.cs new file mode 100644 index 0000000..5347ec7 --- /dev/null +++ b/mySQLPunk/lib/MssqlImport/MssqlImportModels.cs @@ -0,0 +1,237 @@ +using System; +using System.Collections.Generic; +using System.Linq; + +namespace mySQLPunk.lib.MssqlImport +{ + internal enum MssqlImportScriptKind { Generic, NativeSchema, NativeData } + internal enum MssqlImportStage { Scanning, Executing, Verifying, Stopping, Finished } + internal enum MssqlImportPhase { None, Schema, Data, Indexes, Constraints } + internal enum MssqlImportStatus { Completed, CompletedWithErrors, Failed, Cancelled, CleanupUncertain } + + internal sealed class MssqlImportRequest + { + public Guid JobId { get; } + public string SourcePath { get; } + public string DatabaseName { get; } + public bool SkipSqlErrors { get; } + internal MssqlImportRequest(Guid jobId, string sourcePath, string databaseName, bool skipSqlErrors = false) + { + if (jobId == Guid.Empty) throw new ArgumentException("A job identity is required.", nameof(jobId)); + JobId = jobId; SourcePath = sourcePath ?? string.Empty; DatabaseName = databaseName; SkipSqlErrors = skipSqlErrors; + } + } + + internal sealed class MssqlImportTarget + { + public string ServerName { get; } + public string DatabaseName { get; } + public int DatabaseId { get; } + public DateTime DatabaseCreateDate { get; } + public string Version { get; } + public bool Writable { get; } + public bool MetadataVisible { get; } + public long? UserObjectCount { get; } + public bool? CanCreateTable { get; } + public bool? CanCreateView { get; } + internal MssqlImportTarget(string serverName, string databaseName, int databaseId, DateTime databaseCreateDate, + string version, bool writable, bool metadataVisible, long? userObjectCount, bool? canCreateTable = null, bool? canCreateView = null) + { + ServerName = serverName; DatabaseName = databaseName; DatabaseId = databaseId; DatabaseCreateDate = databaseCreateDate; + Version = version; Writable = writable; MetadataVisible = metadataVisible; UserObjectCount = userObjectCount; + CanCreateTable = canCreateTable; CanCreateView = canCreateView; + } + } + + internal sealed class MssqlImportSessionState + { + public bool IsOpen { get; } + public int DatabaseId { get; } + public int TransactionCount { get; } + public int XactState { get; } + internal MssqlImportSessionState(bool isOpen, int databaseId, int transactionCount, int xactState) + { IsOpen = isOpen; DatabaseId = databaseId; TransactionCount = transactionCount; XactState = xactState; } + } + + internal sealed class MssqlImportLobState + { + public bool Exists { get; } + public long RowCount { get; } + public long Sequence { get; } + public long ConsumedThrough { get; } + public bool Consumed { get; } + public bool Failed { get; } + internal MssqlImportLobState(bool exists, long rowCount, long sequence, long consumedThrough, bool consumed, bool failed) + { Exists = exists; RowCount = rowCount; Sequence = sequence; ConsumedThrough = consumedThrough; Consumed = consumed; Failed = failed; } + } + + internal sealed class MssqlImportCounts + { + public long Tables { get; } + public long Rows { get; } + internal MssqlImportCounts(long tables, long rows) { Tables = tables; Rows = rows; } + } + + internal sealed class MssqlImportCleanup + { + public bool CommandEnded { get; } + public bool SessionReleased { get; } + public bool TransactionCleanupConfirmed { get; } + public string IssueCode { get; } + public IReadOnlyList Errors { get; } + internal MssqlImportCleanup(bool commandEnded, bool sessionReleased, bool transactionCleanupConfirmed, string issueCode, IEnumerable errors = null) + { + CommandEnded = commandEnded; SessionReleased = sessionReleased; TransactionCleanupConfirmed = transactionCleanupConfirmed; IssueCode = issueCode; + Errors = Array.AsReadOnly((errors ?? Enumerable.Empty()).ToArray()); + } + } + + internal sealed class MssqlImportSqlError + { + public int Number { get; } + public int State { get; } + public int Class { get; } + public int RelativeLine { get; } + public string Message { get; } + internal MssqlImportSqlError(int number, int state, int @class, int relativeLine, string message) + { Number = number; State = state; Class = @class; RelativeLine = relativeLine; Message = message ?? string.Empty; } + } + + internal sealed class MssqlImportBatchException : Exception + { + public IReadOnlyList Errors { get; } + public bool OutcomeUncertain { get; } + internal MssqlImportBatchException(IEnumerable errors, bool outcomeUncertain = false) + : base("MSSQL batch execution failed.") + { Errors = Array.AsReadOnly((errors ?? Enumerable.Empty()).ToArray()); OutcomeUncertain = outcomeUncertain; } + } + + internal sealed class MssqlImportProgress + { + public Guid JobId { get; } + public long FileBytes { get; } + public MssqlImportStage Stage { get; } + public MssqlImportPhase Phase { get; } + public long ScannedBytes { get; } + public long ProcessedBytes { get; } + public long SuccessfulBytes { get; } + public long SkippedBytes { get; } + public long SuccessfulBatches { get; } + public long FailedBatches { get; } + public long BatchNumber { get; } + public long StartLine { get; } + public long EndLine { get; } + public long StartByte { get; } + public long EndByte { get; } + public string TableName { get; } + public TimeSpan Elapsed { get; } + public TimeSpan? Remaining { get; } + public DateTime? EstimatedFinish { get; } + public string RecentErrorText { get; } + public string LogPath { get; } + public long TotalErrors { get; } + public bool LogTruncated { get; } + internal MssqlImportProgress(Guid jobId, long fileBytes, MssqlImportStage stage, long scannedBytes = 0, + long successfulBytes = 0, long skippedBytes = 0, long successfulBatches = 0, long failedBatches = 0, + long batchNumber = 0, long startLine = 0, long endLine = 0, long startByte = 0, long endByte = 0, + MssqlImportPhase phase = MssqlImportPhase.None, string tableName = null, TimeSpan elapsed = default(TimeSpan), + TimeSpan? remaining = null, DateTime? estimatedFinish = null, string recentErrorText = null, + string logPath = null, long totalErrors = 0, bool logTruncated = false) + { + JobId = jobId; FileBytes = fileBytes; Stage = stage; ScannedBytes = scannedBytes; + SuccessfulBytes = successfulBytes; SkippedBytes = skippedBytes; ProcessedBytes = checked(successfulBytes + skippedBytes); + SuccessfulBatches = successfulBatches; FailedBatches = failedBatches; BatchNumber = batchNumber; + StartLine = startLine; EndLine = endLine; StartByte = startByte; EndByte = endByte; Phase = phase; TableName = tableName; + Elapsed = elapsed; Remaining = remaining; EstimatedFinish = estimatedFinish; LogPath = logPath; TotalErrors = totalErrors; + string text = recentErrorText ?? string.Empty; + RecentErrorText = text.Length <= 32768 ? text : text.Substring(text.Length - 32768); + LogTruncated = logTruncated || text.Length > 32768; + } + } + + internal sealed class MssqlImportResult + { + public Guid JobId { get; } + public MssqlImportStatus Status { get; } + public long FileBytes { get; } + public long ProcessedBytes { get; } + public long SuccessfulBytes { get; } + public long SkippedBytes { get; } + public long SuccessfulBatches { get; } + public long FailedBatches { get; } + public long LastAcknowledgedBatch { get; } + public long? UncertainBatch { get; } + public string LogPath { get; } + public MssqlImportCleanup Cleanup { get; } + public string ValidationSummary { get; } + public long TotalErrors { get; } + public TimeSpan Elapsed { get; } + internal MssqlImportResult(Guid jobId, MssqlImportStatus status, long fileBytes, long successfulBytes = 0, long skippedBytes = 0, + long successfulBatches = 0, long failedBatches = 0, long lastAcknowledgedBatch = 0, long? uncertainBatch = null, + string logPath = null, MssqlImportCleanup cleanup = null, string validationSummary = null, long totalErrors = 0, + TimeSpan elapsed = default(TimeSpan)) + { + JobId = jobId; Status = status; FileBytes = fileBytes; SuccessfulBytes = successfulBytes; SkippedBytes = skippedBytes; + ProcessedBytes = checked(successfulBytes + skippedBytes); SuccessfulBatches = successfulBatches; FailedBatches = failedBatches; + LastAcknowledgedBatch = lastAcknowledgedBatch; UncertainBatch = uncertainBatch; LogPath = logPath; Cleanup = cleanup; + ValidationSummary = validationSummary; TotalErrors = totalErrors; Elapsed = elapsed; + } + } + + internal sealed class MssqlImportEstimate + { + public TimeSpan? Remaining { get; } + public DateTime? EstimatedFinish { get; } + public bool IsReliable { get; } + internal MssqlImportEstimate(TimeSpan? remaining, DateTime? estimatedFinish, bool isReliable) + { Remaining = remaining; EstimatedFinish = estimatedFinish; IsReliable = isReliable; } + } + + internal sealed class MssqlImportBatch + { + public long Number { get; } + public long StartByte { get; } + public long EndByte { get; } + public long StartLine { get; } + public long EndLine { get; } + public string Text { get; } + internal string SourceEncodingName { get; } + internal MssqlImportBatch(long number, long startByte, long endByte, long startLine, long endLine, string text, string sourceEncodingName = "UTF-8") + { Number = number; StartByte = startByte; EndByte = endByte; StartLine = startLine; EndLine = endLine; Text = text; SourceEncodingName = sourceEncodingName; } + } + + internal sealed class MssqlImportInputException : Exception + { + public string Code { get; } + public long BatchNumber { get; } + public long Line { get; } + public long ByteOffset { get; } + internal MssqlImportInputException(string code, long batchNumber, long line, long byteOffset, Exception inner = null) + : base(code + (code == "EmbeddedNul" ? " (unsupported embedded NUL or possible BOM-less UTF-16)" : "") + " at batch " + batchNumber + ", line " + line + ", byte " + byteOffset + ".", inner) + { Code = code; BatchNumber = batchNumber; Line = line; ByteOffset = byteOffset; } + } + + internal sealed class MssqlImportScriptInfo + { + public MssqlImportScriptKind Kind { get; } + public Guid? NativeJobId { get; } + public long? ExpectedTables { get; } + public long? ExpectedRows { get; } + public string SourceReadMode { get; } + public bool HasLobProtocol { get; } + internal MssqlImportScriptInfo(MssqlImportScriptKind kind, Guid? job, long? tables, long? rows, string mode, bool lob) + { Kind = kind; NativeJobId = job; ExpectedTables = tables; ExpectedRows = rows; SourceReadMode = mode; HasLobProtocol = lob; } + } + + internal sealed class MssqlImportScan + { + public long FileBytes { get; } + public long BatchCount { get; } + public long MaxBatchCodeUnits { get; } + public string EncodingName { get; } + public string Sha256 { get; } + public MssqlImportScriptInfo Script { get; } + internal MssqlImportScan(long bytes, long batches, long max, string encoding, string sha256, MssqlImportScriptInfo script) + { FileBytes = bytes; BatchCount = batches; MaxBatchCodeUnits = max; EncodingName = encoding; Sha256 = sha256; Script = script; } + } +} diff --git a/mySQLPunk/lib/MssqlImport/MssqlImportProgressEstimator.cs b/mySQLPunk/lib/MssqlImport/MssqlImportProgressEstimator.cs new file mode 100644 index 0000000..556433b --- /dev/null +++ b/mySQLPunk/lib/MssqlImport/MssqlImportProgressEstimator.cs @@ -0,0 +1,93 @@ +using System; +using System.Collections.Generic; + +namespace mySQLPunk.lib.MssqlImport +{ + // Worker-only, confirmed byte boundaries; no file-position, row-count or SQL inference. + internal sealed class MssqlImportProgressEstimator + { + private sealed class Point + { + internal readonly DateTime Time; + internal readonly long Bytes; + internal Point(DateTime time, long bytes) { Time = time; Bytes = bytes; } + } + private readonly List samples = new List(64); + private MssqlImportProgress latest; + private DateTime observedAt; + private DateTime? lastAdvance; + private bool reported; + private Guid reportedJob; + private MssqlImportStage reportedStage; + private DateTime reportedAt; + + internal void Observe(MssqlImportProgress progress, DateTime utcNow) + { + if (progress == null) throw new ArgumentNullException(nameof(progress)); + bool reset = latest == null || latest.JobId != progress.JobId || latest.FileBytes != progress.FileBytes || + utcNow < observedAt || progress.ProcessedBytes < latest.ProcessedBytes; + long previous = reset ? 0 : latest.ProcessedBytes; + if (reset) { samples.Clear(); lastAdvance = null; } + latest = progress; observedAt = utcNow; + if (!CanEstimate(progress)) { samples.Clear(); lastAdvance = null; return; } + if (progress.ProcessedBytes > previous) + { + // Update confirmed counters/timestamp even when a UI report or rate sample is throttled. + lastAdvance = utcNow; + if (samples.Count == 0 || utcNow - samples[samples.Count - 1].Time >= TimeSpan.FromMilliseconds(200)) + samples.Add(new Point(utcNow, progress.ProcessedBytes)); + } + Trim(utcNow); + } + private void Trim(DateTime utcNow) + { + DateTime cutoff = utcNow > DateTime.MinValue.AddSeconds(10) ? utcNow.AddSeconds(-10) : DateTime.MinValue; + // Keep one sample at/before the window boundary so an interval remains measurable. + while (samples.Count > 1 && samples[1].Time <= cutoff) samples.RemoveAt(0); + while (samples.Count > 64) samples.RemoveAt(0); + } + private static bool CanEstimate(MssqlImportProgress progress) + { + return progress.Stage == MssqlImportStage.Executing && progress.Phase != MssqlImportPhase.Indexes && + progress.Phase != MssqlImportPhase.Constraints; + } + private static MssqlImportEstimate Unknown() { return new MssqlImportEstimate(null, null, false); } + internal MssqlImportEstimate Estimate(DateTime utcNow) + { + if (latest == null || !CanEstimate(latest) || !lastAdvance.HasValue || utcNow < observedAt || + utcNow - lastAdvance.Value >= TimeSpan.FromSeconds(5) || latest.ProcessedBytes < 0 || + latest.FileBytes <= latest.ProcessedBytes) return Unknown(); + Trim(utcNow); + if (samples.Count < 3) return Unknown(); + Point first = samples[0], last = samples[samples.Count - 1]; + double span = (last.Time - first.Time).TotalSeconds; + if (span < 2) return Unknown(); + double rateSum = 0, rateSquares = 0; + int rates = 0; + for (int i = 1; i < samples.Count; i++) + { + double rate = (samples[i].Bytes - samples[i - 1].Bytes) / (samples[i].Time - samples[i - 1].Time).TotalSeconds; + if (rate <= 0) continue; + rateSum += rate; rateSquares += rate * rate; rates++; + } + if (rates == 0) return Unknown(); + double mean = rateSum / rates; + double deviation = Math.Sqrt(Math.Max(0, rateSquares / rates - mean * mean)); + if (deviation / mean > 1) return Unknown(); + double speed = (last.Bytes - first.Bytes) / span; + double seconds = (latest.FileBytes - latest.ProcessedBytes) / speed; + if (speed <= 0 || double.IsNaN(seconds) || double.IsInfinity(seconds) || seconds < 0 || + seconds >= TimeSpan.MaxValue.TotalSeconds || seconds >= (DateTime.MaxValue - utcNow).TotalSeconds) return Unknown(); + return new MssqlImportEstimate(TimeSpan.FromSeconds(seconds), utcNow.AddSeconds(seconds), true); + } + internal bool ShouldReport(MssqlImportProgress progress, DateTime utcNow) + { + if (progress == null) throw new ArgumentNullException(nameof(progress)); + bool force = !reported || reportedJob != progress.JobId || reportedStage != progress.Stage || + progress.Stage == MssqlImportStage.Finished || utcNow < reportedAt; + if (!force && utcNow - reportedAt < TimeSpan.FromMilliseconds(200)) return false; + reported = true; reportedJob = progress.JobId; reportedStage = progress.Stage; reportedAt = utcNow; + return true; + } + } +} diff --git a/mySQLPunk/lib/MssqlImport/MssqlImportScriptInspector.cs b/mySQLPunk/lib/MssqlImport/MssqlImportScriptInspector.cs new file mode 100644 index 0000000..ad6bd4d --- /dev/null +++ b/mySQLPunk/lib/MssqlImport/MssqlImportScriptInspector.cs @@ -0,0 +1,273 @@ +using System; +using System.Globalization; +using System.Text; +using System.Text.RegularExpressions; + +namespace mySQLPunk.lib.MssqlImport +{ + internal sealed class MssqlImportScriptInspector + { + private const string Header = "Source read mode:"; + private const string Hint = "Restore into an empty, compatible target selected by the caller; existing objects fail."; + private const string Footer = "mysqlpunk-export-complete"; + private readonly string databaseName; + private bool candidate, header, hint, footer, schema, data, lob, usePending; + private string mode, createKind; + private bool foreignKeyword, foreignKey, references; + private int referentialAction, parenthesisDepth, foreignKeyDepth; + private Guid? job, lobJob; + private long? tables, rows; + private MssqlImportBatch current; + private int currentIndex; + private bool complete; + internal bool UsesNativeLob { get; private set; } + internal bool InitializesNativeLob { get; private set; } + internal MssqlImportPhase Phase { get; private set; } + internal MssqlImportScriptInspector(string databaseName = null) { this.databaseName = databaseName; } + + internal void Observe(MssqlImportBatch batch) + { + if (complete) throw new InvalidOperationException("Inspector is complete."); + current = batch ?? throw new ArgumentNullException(nameof(batch)); + UsesNativeLob = false; InitializesNativeLob = false; Phase = MssqlImportPhase.None; + ResetReferentialContext(); parenthesisDepth = 0; + string text = batch.Text; + bool lineHasCode = false; + int i = 0; + while (i < text.Length) + { + currentIndex = i; + char c = text[i]; + if (c == '\r' || c == '\n') { lineHasCode = false; i++; continue; } + if (char.IsWhiteSpace(c)) { i++; continue; } + if (c == '-' && i + 1 < text.Length && text[i + 1] == '-') + { + int start = i + 2, end = start; + while (end < text.Length && text[end] != '\r' && text[end] != '\n') end++; + // Only actual line comments can carry native markers. Capture at most + // 512 characters, never the whole comment or all batch metadata. + if (!lineHasCode) Comment(text, start, end); + i = end; continue; + } + if (c == '/' && i + 1 < text.Length && text[i + 1] == '*') + { + int depth = 1; i += 2; + while (i < text.Length && depth != 0) + { + if (text[i] == '\r' || text[i] == '\n') lineHasCode = false; + if (i + 1 < text.Length && text[i] == '/' && text[i + 1] == '*') { depth++; i += 2; } + else if (i + 1 < text.Length && text[i] == '*' && text[i + 1] == '/') { depth--; i += 2; } + else i++; + } + continue; + } + if ((!lineHasCode && (c == ':' || (c == '!' && i + 1 < text.Length && text[i + 1] == '!'))) || (c == '$' && i + 1 < text.Length && text[i + 1] == '(')) throw Error("SqlCmdUnsupported"); + if (footer) throw Error("NativeFormat"); // Footer is an EOF marker, not an in-band hint. + bool firstCode = !lineHasCode; + lineHasCode = true; + if (c == '\'') + { + if (usePending) throw Error("DatabaseChange"); + SkipQuoted(text, ref i, '\'', null); + continue; + } + if (c == '[' || c == '"') + { + var token = new StringBuilder(); + SkipQuoted(text, ref i, c == '[' ? ']' : '"', token); + Token(token.ToString(), true); + continue; + } + if (IsWord(c)) + { + int start = i; + while (i < text.Length && IsWord(text[i])) + { + if (text[i] == '$' && i + 1 < text.Length && text[i + 1] == '(') { currentIndex = i; throw Error("SqlCmdUnsupported"); } + i++; + } + int length = i - start; + if (firstCode && length == 2 && string.Compare(text, start, "GO", 0, 2, StringComparison.OrdinalIgnoreCase) == 0) throw Error("UnsupportedGo"); + // Long identifiers are irrelevant unless they claim our reserved native + // namespace or are the pending USE target; do not allocate the token. + if (length > 512) + { + if (usePending) throw Error("DatabaseChange"); + if (text[start] == '#' && length >= 4 && string.Compare(text, start, "#mp_", 0, 4, StringComparison.OrdinalIgnoreCase) == 0) throw Error("NativeFormat"); + createKind = null; + } + else Token(text.Substring(start, length), false); + continue; + } + if (usePending) throw Error("DatabaseChange"); + createKind = null; + if (c == '(') parenthesisDepth++; + else if (c == ')') + { + if (parenthesisDepth > 0) parenthesisDepth--; + if (foreignKey && parenthesisDepth < foreignKeyDepth) ResetReferentialContext(); + } + else if (c == ';' || (c == ',' && (!foreignKey || parenthesisDepth <= foreignKeyDepth))) ResetReferentialContext(); + i++; + } + if (usePending) throw Error("DatabaseChange"); + ResetReferentialContext(); + // Keep only immutable source coordinates, not the previous SQL payload. + current = new MssqlImportBatch(batch.Number, batch.StartByte, batch.EndByte, batch.StartLine, batch.EndLine, string.Empty); + currentIndex = 0; + } + + private static bool IsWord(char c) => char.IsLetterOrDigit(c) || c == '_' || c == '#' || c == '@' || c == '$'; + private void SkipQuoted(string text, ref int i, char close, StringBuilder value) + { + i++; + while (i < text.Length) + { + char c = text[i++]; + if (c == close) + { + if (i < text.Length && text[i] == close) i++; + else return; + } + if (value != null) + { + if (value.Length == 512) + { + if (usePending) throw Error("DatabaseChange"); + value = null; + } + else value.Append(c); + } + } + } + private void Token(string token, bool quoted) + { + bool dataToken = !quoted && (token.Equals("INSERT", StringComparison.OrdinalIgnoreCase) || token.Equals("UPDATE", StringComparison.OrdinalIgnoreCase) || token.Equals("DELETE", StringComparison.OrdinalIgnoreCase)); + if (ReferentialKeyword(token, quoted)) dataToken = false; + if (usePending) + { + usePending = false; + if (databaseName == null || !string.Equals(token, databaseName, StringComparison.Ordinal)) throw Error("DatabaseChange"); + createKind = null; + return; + } + if (!quoted && token.Equals("USE", StringComparison.OrdinalIgnoreCase)) { usePending = true; return; } + if ((!quoted || header) && token.StartsWith("#mp_", StringComparison.OrdinalIgnoreCase)) + { + candidate = true; lob = true; + Guid parsed; + if (token.Length != 38 || token[36] != '_' || (token[37] != 's' && token[37] != 'v' && token[37] != 'c') || !Guid.TryParseExact(token.Substring(4, 32), "N", out parsed) || parsed == Guid.Empty) throw Error("NativeFormat"); + if (lobJob.HasValue && lobJob != parsed) throw Error("NativeFormat"); + lobJob = parsed; + UsesNativeLob = true; + if (createKind == "object" && token[37] == 's') InitializesNativeLob = true; + createKind = null; + return; + } + if (!quoted) + { + if (token.Equals("CONSTRAINT", StringComparison.OrdinalIgnoreCase) || token.Equals("FOREIGN", StringComparison.OrdinalIgnoreCase)) Phase = MssqlImportPhase.Constraints; + else if (createKind == "create" && token.Equals("INDEX", StringComparison.OrdinalIgnoreCase) && Phase != MssqlImportPhase.Constraints) Phase = MssqlImportPhase.Indexes; + else if (Phase == MssqlImportPhase.None && token.Equals("CREATE", StringComparison.OrdinalIgnoreCase)) Phase = MssqlImportPhase.Schema; + else if ((Phase == MssqlImportPhase.None || Phase == MssqlImportPhase.Schema) && dataToken) Phase = MssqlImportPhase.Data; + if (createKind == "create" && (token.Equals("UNIQUE", StringComparison.OrdinalIgnoreCase) || token.Equals("CLUSTERED", StringComparison.OrdinalIgnoreCase) || token.Equals("NONCLUSTERED", StringComparison.OrdinalIgnoreCase))) return; + } + if (createKind == "object") + { + if (!token.StartsWith("#", StringComparison.Ordinal)) schema = true; + createKind = null; + } + else if (createKind == "create" && !quoted && (token.Equals("TABLE", StringComparison.OrdinalIgnoreCase) || token.Equals("VIEW", StringComparison.OrdinalIgnoreCase) || token.Equals("PROCEDURE", StringComparison.OrdinalIgnoreCase) || token.Equals("FUNCTION", StringComparison.OrdinalIgnoreCase))) createKind = "object"; + else createKind = !quoted && token.Equals("CREATE", StringComparison.OrdinalIgnoreCase) ? "create" : null; + if (dataToken) data = true; + } + // This recognizes only actual FOREIGN KEY ... REFERENCES ... ON action clauses, + // not arbitrary previous-ON tokens such as SET ANSI_NULLS ON DELETE ... . + private bool ReferentialKeyword(string token, bool quoted) + { + string word = quoted ? string.Empty : token.ToUpperInvariant(); + if (referentialAction == 4) + { + referentialAction = 0; + if (word == "DELETE" || word == "UPDATE") { referentialAction = 1; return true; } + references = false; + } + else if (referentialAction != 0) + { + int expected = referentialAction; + if (expected == 1 && word == "NO") { referentialAction = 2; return false; } + if (expected == 1 && word == "SET") { referentialAction = 3; return false; } + if ((expected == 1 && word == "CASCADE") || (expected == 2 && word == "ACTION") || (expected == 3 && (word == "NULL" || word == "DEFAULT"))) { referentialAction = 0; return false; } + data = true; referentialAction = 0; references = false; + } + if (word == "INSERT" || word == "UPDATE" || word == "DELETE" || word == "SET" || word == "ALTER" || word == "CREATE" || word == "SELECT") + { references = false; foreignKey = false; } + if (references && word == "ON") referentialAction = 4; + if (foreignKeyword && word == "KEY") foreignKey = true; + if (word == "FOREIGN") foreignKeyDepth = parenthesisDepth; + foreignKeyword = word == "FOREIGN"; + if (foreignKey && word == "REFERENCES") references = true; + return false; + } + private void ResetReferentialContext() + { + if (referentialAction > 0 && referentialAction < 4) data = true; // Incomplete action must never hide real data syntax. + referentialAction = 0; foreignKeyword = false; foreignKey = false; references = false; foreignKeyDepth = 0; + } + private void Comment(string text, int start, int end) + { + while (start < end && char.IsWhiteSpace(text[start])) start++; + int count = Math.Min(end - start, 512); + string comment = text.Substring(start, count).TrimEnd(); + bool isHeader = comment.StartsWith("Source read mode", StringComparison.OrdinalIgnoreCase); + bool isHint = comment.StartsWith("Restore into an empty, compatible target", StringComparison.OrdinalIgnoreCase); + bool isFooter = comment.StartsWith(Footer, StringComparison.OrdinalIgnoreCase); + if (!isHeader && !isHint && !isFooter) return; + candidate = true; + if (end - start > 512 || footer) throw Error("NativeFormat"); + if (isHeader) + { + if (header || schema || data || lob || !comment.StartsWith(Header, StringComparison.Ordinal)) throw Error("NativeFormat"); + header = true; mode = comment.Substring(Header.Length).Trim(); + if (mode != "READ COMMITTED" && mode != "SNAPSHOT" && mode != "Schema only (no data transaction)") throw Error("NativeFormat"); + } + else if (isHint) + { + if (hint || !header || schema || data || lob || comment != Hint) throw Error("NativeFormat"); + hint = true; + } + else + { + var match = Regex.Match(comment, @"\Amysqlpunk-export-complete job=([0-9a-fA-F]{32}) tables=([0-9]+) rows=([0-9]+)\z", RegexOptions.CultureInvariant); + Guid parsed; long tableCount, rowCount; + if (!match.Success || !Guid.TryParseExact(match.Groups[1].Value, "N", out parsed) || parsed == Guid.Empty || !long.TryParse(match.Groups[2].Value, NumberStyles.None, CultureInfo.InvariantCulture, out tableCount) || !long.TryParse(match.Groups[3].Value, NumberStyles.None, CultureInfo.InvariantCulture, out rowCount)) throw Error("NativeFormat"); + footer = true; job = parsed; tables = tableCount; rows = rowCount; + } + } + internal MssqlImportScriptInfo Complete() + { + if (complete) throw new InvalidOperationException("Inspector is complete."); + complete = true; + if (!candidate) return new MssqlImportScriptInfo(MssqlImportScriptKind.Generic, null, null, null, null, false); + if (!header || !footer || (hint && !schema && (tables != 0 || rows != 0)) || (schema && !hint) || (lobJob.HasValue && lobJob != job) || (mode == "Schema only (no data transaction)" && (!hint || data || lob || rows != 0))) throw Error("NativeFormat"); + return new MssqlImportScriptInfo(hint ? MssqlImportScriptKind.NativeSchema : MssqlImportScriptKind.NativeData, job, tables, rows, mode, lob); + } + private MssqlImportInputException Error(string code) + { + if (current == null) return new MssqlImportInputException(code, 1, 1, 0); + long errorLine = current.StartLine; + for (int i = 0; i < currentIndex; i++) if (current.Text[i] == '\n' || (current.Text[i] == '\r' && (i + 1 == current.Text.Length || current.Text[i + 1] != '\n'))) errorLine++; + // Reader supplies exact byte encoding for variable-width source diagnostics. + string name = current.SourceEncodingName; + long prefixBytes = 0; + for (int i = 0; i < currentIndex; i++) + { + char c = current.Text[i]; + prefixBytes += name.StartsWith("UTF-16", StringComparison.Ordinal) ? 2 : char.IsHighSurrogate(c) ? 0 : char.IsLowSurrogate(c) ? 4 : c < 128 ? 1 : c < 2048 ? 2 : 3; + } + long bom = current.StartByte == 0 ? (name == "UTF-8 BOM" ? 3 : name.StartsWith("UTF-16", StringComparison.Ordinal) ? 2 : 0) : 0; + long offset = current.StartByte + bom + prefixBytes; + return new MssqlImportInputException(code, current.Number, errorLine, offset); + } + } +} diff --git a/mySQLPunk/lib/MssqlImport/MssqlImportService.cs b/mySQLPunk/lib/MssqlImport/MssqlImportService.cs new file mode 100644 index 0000000..75ac037 --- /dev/null +++ b/mySQLPunk/lib/MssqlImport/MssqlImportService.cs @@ -0,0 +1,379 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.Globalization; +using System.IO; +using System.Linq; +using System.Threading; + +namespace mySQLPunk.lib.MssqlImport +{ + internal sealed class MssqlImportIssue + { + public string Code { get; } + public string Message { get; } + public bool IsWarning { get; } + public long BatchNumber { get; } + public long Line { get; } + public long ByteOffset { get; } + internal MssqlImportIssue(string code, bool warning = false, long batch = 0, long line = 0, long offset = 0) + { Code = code; Message = code; IsWarning = warning; BatchNumber = batch; Line = line; ByteOffset = offset; } + } + + // Resource ownership transfers exactly once; a stale wizard Dispose cannot close a running input/log. + internal sealed class MssqlImportPreflight : IDisposable + { + private readonly object gate = new object(); + private readonly MssqlImportService service; + private readonly MssqlImportRequest request; + private readonly string source; + private MssqlImportInput input; + private IMssqlImportLog log; + private bool claimed, disposed; + public bool Passed { get; } + public MssqlImportScan Scan { get; } + public MssqlImportTarget Target { get; } + public string LogPath { get; } + public IReadOnlyList Issues { get; } + internal MssqlImportPreflight(MssqlImportService service, MssqlImportRequest request, string source, MssqlImportInput input, + MssqlImportScan scan, MssqlImportTarget target, IMssqlImportLog log, IEnumerable issues) + { + this.service = service; this.request = request; this.source = source; this.input = input; this.log = log; + Scan = scan; Target = target; LogPath = log == null ? null : log.Path; + Issues = Array.AsReadOnly(issues.ToArray()); Passed = !Issues.Any(i => !i.IsWarning); + } + internal bool Claim(MssqlImportService caller, MssqlImportRequest current, out MssqlImportInput ownedInput, out IMssqlImportLog ownedLog) + { + ownedInput = null; ownedLog = null; + lock (gate) + { + if (disposed || claimed || !Passed || caller != service || current == null || request == null || + current.JobId != request.JobId || current.SkipSqlErrors != request.SkipSqlErrors || + !string.Equals(current.DatabaseName, request.DatabaseName, StringComparison.Ordinal)) return false; + string path; try { path = Path.GetFullPath(current.SourcePath); } catch { return false; } + if (!string.Equals(path, source, StringComparison.OrdinalIgnoreCase)) return false; + claimed = true; ownedInput = input; ownedLog = log; input = null; log = null; return true; + } + } + public void Dispose() + { + MssqlImportInput ownedInput; IMssqlImportLog ownedLog; + lock (gate) { if (disposed) return; disposed = true; ownedInput = input; ownedLog = log; input = null; log = null; } + try { if (ownedInput != null) ownedInput.Dispose(); } finally { if (ownedLog != null) ownedLog.Dispose(); } + } + } + + internal sealed class MssqlImportService + { + private readonly Func openSession; + private readonly Func openLog; + internal MssqlImportService(Func openSession, Func openLog) + { this.openSession = openSession ?? throw new ArgumentNullException(nameof(openSession)); this.openLog = openLog ?? throw new ArgumentNullException(nameof(openLog)); } + + internal MssqlImportPreflight Preflight(MssqlImportRequest request, Action progress, CancellationToken cancellation) + { + var issues = new List(); + MssqlImportInput input = null; MssqlImportScan scan = null; MssqlImportTarget target = null; IMssqlImportLog log = null; + string source = null; + try + { + if (request == null || string.IsNullOrWhiteSpace(request.SourcePath)) throw new InvalidOperationException("SourceRequired"); + if (string.IsNullOrWhiteSpace(request.DatabaseName)) throw new InvalidOperationException("TargetRequired"); + cancellation.ThrowIfCancellationRequested(); source = Path.GetFullPath(request.SourcePath); + input = MssqlImportInput.Open(source, request.DatabaseName); + var clock = Stopwatch.StartNew(); var estimator = new MssqlImportProgressEstimator(); + scan = input.Scan(cancellation, bytes => { + var p = new MssqlImportProgress(request.JobId, 0, MssqlImportStage.Scanning, scannedBytes: bytes, elapsed: clock.Elapsed); + if (estimator.ShouldReport(p, DateTime.UtcNow)) Notify(progress, p); + }); + cancellation.ThrowIfCancellationRequested(); + using (var session = openSession()) target = session.ReadTarget(cancellation); + string gate = TargetGate(request, scan, target); + if (gate != null) throw new InvalidOperationException(gate); + if (scan.Script.Kind == MssqlImportScriptKind.Generic) issues.Add(new MssqlImportIssue("SessionIntrinsicsMayChange", true)); + if (scan.Script.Kind != MssqlImportScriptKind.NativeSchema && (!target.MetadataVisible || target.UserObjectCount != 0)) issues.Add(new MssqlImportIssue("TargetNotProvenEmpty", true)); + if (target.CanCreateView != true) issues.Add(new MssqlImportIssue("CreateViewNotProven", true)); + if (scan.Script.Kind != MssqlImportScriptKind.NativeSchema && target.CanCreateTable != true) issues.Add(new MssqlImportIssue("CreateTableNotProven", true)); + issues.Add(new MssqlImportIssue("PartialBatchWritesMayRemain", true)); + log = openLog(request.JobId); log.WriteStart(request, scan, target); + cancellation.ThrowIfCancellationRequested(); + } + catch (Exception error) + { + var parser = error as MssqlImportInputException; + string code = parser != null ? parser.Code : cancellation.IsCancellationRequested ? "Cancelled" : SafeCode(error, "PreflightFailed"); + issues.Add(new MssqlImportIssue(code, false, parser == null ? 0 : parser.BatchNumber, parser == null ? 0 : parser.Line, parser == null ? 0 : parser.ByteOffset)); + try { if (input != null) input.Dispose(); } catch { issues.Add(new MssqlImportIssue("InputReleaseFailed")); } + try { if (log != null) log.Dispose(); } catch { issues.Add(new MssqlImportIssue("LogIncomplete")); } + input = null; log = null; + } + return new MssqlImportPreflight(this, request, source, input, scan, target, log, issues); + } + + internal MssqlImportResult Run(MssqlImportRequest request, MssqlImportPreflight preflight, Action progress, CancellationToken cancellation) + { + MssqlImportInput input; IMssqlImportLog log; + if (preflight == null || !preflight.Claim(this, request, out input, out log)) + return new MssqlImportResult(request == null ? Guid.Empty : request.JobId, MssqlImportStatus.Failed, + preflight == null || preflight.Scan == null ? 0 : preflight.Scan.FileBytes, validationSummary: "PreflightInvalidOrConsumed"); + return new Worker(this, request, preflight, input, log, progress, cancellation).Run(); + } + + private static string TargetGate(MssqlImportRequest request, MssqlImportScan scan, MssqlImportTarget target) + { + if (target == null || string.IsNullOrWhiteSpace(target.ServerName) || string.IsNullOrWhiteSpace(target.Version) || + target.DatabaseId <= 0 || target.DatabaseCreateDate == default(DateTime) || !string.Equals(target.DatabaseName, request.DatabaseName, StringComparison.Ordinal)) return "TargetIdentityUnavailable"; + if (!target.Writable) return "TargetReadOnly"; + if (scan.Script.Kind == MssqlImportScriptKind.NativeSchema) + { + if (!target.MetadataVisible || !target.UserObjectCount.HasValue) return "TargetVisibilityRequired"; + if (target.UserObjectCount != 0) return "TargetNotEmpty"; + if (scan.Script.ExpectedTables > 0 && target.CanCreateTable != true) return "CreateTableRequired"; + } + return null; + } + private static bool SameTarget(MssqlImportTarget a, MssqlImportTarget b) + { return a != null && b != null && a.DatabaseId == b.DatabaseId && a.DatabaseCreateDate == b.DatabaseCreateDate && string.Equals(a.ServerName, b.ServerName, StringComparison.Ordinal) && string.Equals(a.DatabaseName, b.DatabaseName, StringComparison.Ordinal); } + private static void Notify(Action action, MssqlImportProgress progress) + { if (action != null) { try { action(progress); } catch { /* An observer cannot change import truth. */ } } } + private static string SafeCode(Exception error, string fallback) + { + // Only service-owned fixed codes may cross the UI boundary; provider/IO messages may contain secrets. + if (error is InvalidOperationException) + switch (error.Message) + { + case "SourceRequired": case "TargetRequired": case "TargetIdentityUnavailable": case "TargetReadOnly": + case "TargetVisibilityRequired": case "TargetNotEmpty": case "CreateTableRequired": + case "TargetChanged": case "SessionStateInvalid": case "LobStateInvalid": case "LobInitializationMissing": + case "EofTransactionOpen": case "DigestMismatch": case "SummaryMismatch": return error.Message; + } + return fallback; + } + + private sealed class Worker + { + private readonly MssqlImportService service; + private readonly MssqlImportRequest request; + private readonly MssqlImportPreflight preflight; + private readonly MssqlImportInput input; + private readonly IMssqlImportLog log; + private readonly Action observer; + private readonly CancellationToken cancellation; + private readonly Stopwatch clock = Stopwatch.StartNew(); + private readonly MssqlImportProgressEstimator estimator = new MssqlImportProgressEstimator(); + private readonly object reportGate = new object(); + private MssqlImportProgress latest; + private IMssqlImportSession session; + private MssqlImportBatch batch; + private MssqlImportPhase phase; + private long successfulBytes, skippedBytes, successes, failures, acknowledged, boundary; + private long? uncertain; + private bool lobInitialized, unacknowledgedExecution, finished, logIncomplete; + private MssqlImportCleanup cleanup; + private MssqlImportStatus status = MssqlImportStatus.Failed; + private string summary = "ExecutionNotStarted"; + internal Worker(MssqlImportService service, MssqlImportRequest request, MssqlImportPreflight preflight, MssqlImportInput input, + IMssqlImportLog log, Action observer, CancellationToken cancellation) + { this.service = service; this.request = request; this.preflight = preflight; this.input = input; this.log = log; this.observer = observer; this.cancellation = cancellation; } + + internal MssqlImportResult Run() + { + Publish(MssqlImportStage.Executing, true); + using (cancellation.Register(Stopping)) + { + try + { + cancellation.ThrowIfCancellationRequested(); session = service.openSession(); + var target = session.ReadTarget(cancellation); + if (!SameTarget(preflight.Target, target)) throw new InvalidOperationException("TargetChanged"); + string targetGate = TargetGate(request, preflight.Scan, target); + if (targetGate != null) throw new InvalidOperationException(targetGate); + var inspector = new MssqlImportScriptInspector(request.DatabaseName); + foreach (var next in input.ReadBatches(cancellation)) + { + cancellation.ThrowIfCancellationRequested(); batch = next; inspector.Observe(batch); + var previousPhase = phase; + phase = preflight.Scan.Script.Kind == MssqlImportScriptKind.Generic ? MssqlImportPhase.None : inspector.Phase; + Publish(MssqlImportStage.Executing, phase != previousPhase); + cancellation.ThrowIfCancellationRequested(); + var before = session.ReadState(cancellation); RequireState(before); + if (inspector.UsesNativeLob && !lobInitialized && !inspector.InitializesNativeLob) throw new InvalidOperationException("LobInitializationMissing"); + if (inspector.InitializesNativeLob) lobInitialized = true; + try + { + cancellation.ThrowIfCancellationRequested(); unacknowledgedExecution = true; + session.ExecuteBatch(batch, cancellation); + } + catch (MssqlImportBatchException error) + { + unacknowledgedExecution = false; + failures = checked(failures + 1); if (error.OutcomeUncertain) uncertain = batch.Number; + HandleSqlError(error, before, inspector.UsesNativeLob); + continue; + } + cancellation.ThrowIfCancellationRequested(); + var after = session.ReadState(cancellation); RequireState(after); + if (lobInitialized) RequireLob(false); + successfulBytes = checked(successfulBytes + batch.EndByte - boundary); boundary = batch.EndByte; + successes = checked(successes + 1); acknowledged = batch.Number; unacknowledgedExecution = false; + Publish(MssqlImportStage.Executing); + } + batch = null; phase = MssqlImportPhase.None; Publish(MssqlImportStage.Verifying, true); + cancellation.ThrowIfCancellationRequested(); + if (input.ExecutionSha256 == null || !string.Equals(input.ExecutionSha256, preflight.Scan.Sha256, StringComparison.Ordinal)) throw new InvalidOperationException("DigestMismatch"); + var finalState = session.ReadState(cancellation); RequireState(finalState); + if (finalState.TransactionCount != 0 || finalState.XactState != 0) throw new InvalidOperationException("EofTransactionOpen"); + if (!SameTarget(preflight.Target, session.ReadTarget(cancellation))) throw new InvalidOperationException("TargetChanged"); + if (lobInitialized) RequireLob(true); + summary = failures == 0 ? "AllBatchesExecuted;NoDataContentValidation" : "AllBatchesProcessedWithErrors;FullRestoreNotValidated"; + if (failures == 0 && preflight.Scan.Script.Kind == MssqlImportScriptKind.NativeSchema) + { + var counts = session.CountTargetRows((tables, rows) => { Publish(MssqlImportStage.Verifying); cancellation.ThrowIfCancellationRequested(); }, cancellation); + if (counts.Tables != preflight.Scan.Script.ExpectedTables || counts.Rows != preflight.Scan.Script.ExpectedRows) throw new InvalidOperationException("SummaryMismatch"); + summary = "NativeSummaryCountsMatched;NotColumnContentValidation"; + } + cancellation.ThrowIfCancellationRequested(); + successfulBytes = checked(successfulBytes + preflight.Scan.FileBytes - boundary); boundary = preflight.Scan.FileBytes; + status = failures == 0 ? MssqlImportStatus.Completed : MssqlImportStatus.CompletedWithErrors; + } + catch (OperationCanceledException) + { + status = cancellation.IsCancellationRequested ? MssqlImportStatus.Cancelled : MssqlImportStatus.Failed; + summary = cancellation.IsCancellationRequested ? "Cancelled;PreviouslyCommittedWritesMayRemain" : "ExecutionOutcomeUnknown"; + if (batch != null && unacknowledgedExecution) uncertain = batch.Number; + } + catch (Exception error) + { + status = MssqlImportStatus.Failed; + if (!summary.StartsWith("SqlBatchFailed", StringComparison.Ordinal) && !logIncomplete) summary = SafeCode(error, "ImportFailed"); + if (batch != null && unacknowledgedExecution) uncertain = batch.Number; + var sql = error as MssqlImportBatchException; + if (sql != null) + { + try { log.WriteError(new MssqlImportBatch(0, 0, 0, 0, 0, string.Empty), sql, "Stop;Origin=Probe;DuringBatch=" + (batch == null ? "0" : batch.Number.ToString(CultureInfo.InvariantCulture))); log.Flush(); } + catch { MarkLogIncomplete(); } + if (cancellation.IsCancellationRequested && !logIncomplete) + { status = MssqlImportStatus.Cancelled; summary = "Cancelled;PreviouslyCommittedWritesMayRemain"; } + } + } + finally + { + try + { + if (session != null) cleanup = session.Cleanup() ?? new MssqlImportCleanup(false, false, false, "CleanupReceiptMissing"); + else cleanup = new MssqlImportCleanup(true, true, true, null); + } + catch { cleanup = new MssqlImportCleanup(false, false, false, "CleanupFailed"); } + try { if (session != null) session.Dispose(); } + catch { cleanup = new MssqlImportCleanup(cleanup.CommandEnded, false, cleanup.TransactionCleanupConfirmed, "SessionReleaseFailed", cleanup.Errors); } + try { input.Dispose(); } catch { summary += ";InputReleaseFailed"; if (status != MssqlImportStatus.Failed) status = MssqlImportStatus.CleanupUncertain; } + if (cleanup == null || !cleanup.CommandEnded || !cleanup.SessionReleased || !cleanup.TransactionCleanupConfirmed) + { + summary += ";CleanupUnconfirmed"; + if (status != MssqlImportStatus.Failed) status = MssqlImportStatus.CleanupUncertain; + } + if (cleanup != null && cleanup.Errors.Count != 0) + { + try + { + log.WriteError(new MssqlImportBatch(0, 0, 0, 0, 0, string.Empty), new MssqlImportBatchException(cleanup.Errors, true), + "Stop;Origin=Cleanup;DuringBatch=" + (batch == null ? "0" : batch.Number.ToString(CultureInfo.InvariantCulture))); + log.Flush(); + } + catch { MarkLogIncomplete(); } + } + try { log.WriteEnd(Result()); } catch { MarkLogIncomplete(); } + try { log.Dispose(); } catch { MarkLogIncomplete(); } + Publish(MssqlImportStage.Finished, true); + } + } + return Result(); + } + + private void HandleSqlError(MssqlImportBatchException error, MssqlImportSessionState before, bool protocol) + { + bool skip = false; string stateSummary = string.Empty; MssqlImportBatchException probeFailure = null; + // Cancellation with SQL details must still log the original complete detail list. + if (!cancellation.IsCancellationRequested) + { + try + { + var after = session.ReadState(cancellation); RequireState(after); + if (lobInitialized) { RequireLob(true); } + skip = request.SkipSqlErrors && !protocol && !error.OutcomeUncertain && error.Errors.Count != 0 && + error.Errors.All(e => e.Class >= 11 && e.Class <= 16 && !UnsafeErrorNumber(e.Number)) && + before.TransactionCount == 0 && before.XactState == 0 && after.TransactionCount == 0 && after.XactState == 0; + stateSummary = ";TransactionCount=" + after.TransactionCount.ToString(CultureInfo.InvariantCulture) + ";XactState=" + after.XactState.ToString(CultureInfo.InvariantCulture) + ";DatabaseId=" + after.DatabaseId.ToString(CultureInfo.InvariantCulture); + } + catch (Exception stateError) + { + skip = false; stateSummary = ";StateCheck=" + SafeCode(stateError, "StateUnconfirmed"); + probeFailure = stateError as MssqlImportBatchException; + } + } + if (protocol) stateSummary += ";NativeLobBatch"; + if (!skip) summary = "SqlBatchFailed"; + try + { + log.WriteError(batch, error, (cancellation.IsCancellationRequested ? "Cancel" : skip ? "Skip" : "Stop") + ";Origin=Source" + stateSummary); + if (probeFailure != null) log.WriteError(new MssqlImportBatch(0, 0, 0, 0, 0, string.Empty), probeFailure, "Stop;Origin=Probe;DuringBatch=" + batch.Number.ToString(CultureInfo.InvariantCulture)); + log.Flush(); + } + catch { MarkLogIncomplete(); throw; } + cancellation.ThrowIfCancellationRequested(); + if (!skip) { summary = "SqlBatchFailed"; throw new InvalidOperationException("SqlBatchFailed"); } + skippedBytes = checked(skippedBytes + batch.EndByte - boundary); boundary = batch.EndByte; acknowledged = batch.Number; + Publish(MssqlImportStage.Executing); + } + private static bool UnsafeErrorNumber(int number) + { switch (number) { case -2: case 0: case 11: case 20: case 64: case 233: case 596: case 10053: case 10054: case 10060: return true; default: return false; } } + private void RequireState(MssqlImportSessionState state) + { if (state == null || !state.IsOpen || state.DatabaseId != preflight.Target.DatabaseId || state.TransactionCount < 0 || state.XactState < -1 || state.XactState > 1) throw new InvalidOperationException("SessionStateInvalid"); } + private void RequireLob(bool terminal) + { + if (!preflight.Scan.Script.NativeJobId.HasValue) throw new InvalidOperationException("LobStateInvalid"); + var state = session.ReadLobState(preflight.Scan.Script.NativeJobId.Value, cancellation); + if (state == null || !state.Exists || state.RowCount != 1 || state.Failed || state.Sequence < 0 || state.ConsumedThrough < 0 || + (state.Consumed ? state.Sequence != state.ConsumedThrough : state.Sequence <= 0 || state.ConsumedThrough != state.Sequence - 1) || terminal && !state.Consumed) + throw new InvalidOperationException("LobStateInvalid"); + } + private void MarkLogIncomplete() + { logIncomplete = true; if (!summary.Contains("LogIncomplete")) summary += ";LogIncomplete"; if (status != MssqlImportStatus.Failed) status = MssqlImportStatus.Failed; } + private MssqlImportResult Result() + { return new MssqlImportResult(request.JobId, status, preflight.Scan.FileBytes, successfulBytes, skippedBytes, successes, failures, acknowledged, uncertain, log.Path, cleanup, summary, log.Tail.TotalErrors, clock.Elapsed); } + private void Stopping() + { + lock (reportGate) + { + if (finished || latest == null || latest.Stage == MssqlImportStage.Stopping) return; + latest = Snapshot(MssqlImportStage.Stopping, latest, null); Notify(observer, latest); + } + } + private MssqlImportProgress Snapshot(MssqlImportStage stage, MssqlImportProgress previous, MssqlImportEstimate estimate) + { + // Cancellation thread only copies an already immutable worker snapshot. + if (previous != null) return new MssqlImportProgress(request.JobId, previous.FileBytes, stage, previous.ScannedBytes, previous.SuccessfulBytes, previous.SkippedBytes, + previous.SuccessfulBatches, previous.FailedBatches, previous.BatchNumber, previous.StartLine, previous.EndLine, previous.StartByte, previous.EndByte, + previous.Phase, elapsed: clock.Elapsed, recentErrorText: previous.RecentErrorText, logPath: previous.LogPath, totalErrors: previous.TotalErrors, logTruncated: previous.LogTruncated); + return new MssqlImportProgress(request.JobId, preflight.Scan.FileBytes, stage, successfulBytes: successfulBytes, skippedBytes: skippedBytes, + successfulBatches: successes, failedBatches: failures, batchNumber: batch == null ? 0 : batch.Number, startLine: batch == null ? 0 : batch.StartLine, + endLine: batch == null ? 0 : batch.EndLine, startByte: batch == null ? 0 : batch.StartByte, endByte: batch == null ? 0 : batch.EndByte, phase: phase, + elapsed: clock.Elapsed, remaining: estimate == null ? null : estimate.Remaining, estimatedFinish: estimate == null ? null : estimate.EstimatedFinish, + recentErrorText: log.Tail.Snapshot() + (logIncomplete ? "\r\nLogIncomplete" : ""), logPath: log.Path, totalErrors: log.Tail.TotalErrors, logTruncated: log.Tail.Truncated); + } + private void Publish(MssqlImportStage stage, bool force = false) + { + lock (reportGate) + { + if (finished) return; + if (cancellation.IsCancellationRequested && stage != MssqlImportStage.Finished) stage = MssqlImportStage.Stopping; + var now = DateTime.UtcNow; var p = Snapshot(stage, null, null); estimator.Observe(p, now); + latest = Snapshot(stage, null, estimator.Estimate(now)); + bool shouldReport = estimator.ShouldReport(latest, now); + if (stage == MssqlImportStage.Finished) finished = true; + if (force || shouldReport) Notify(observer, latest); + } + } + } + } +} \ No newline at end of file diff --git a/mySQLPunk/lib/MssqlImport/MssqlImportSession.cs b/mySQLPunk/lib/MssqlImport/MssqlImportSession.cs new file mode 100644 index 0000000..d8706f0 --- /dev/null +++ b/mySQLPunk/lib/MssqlImport/MssqlImportSession.cs @@ -0,0 +1,382 @@ +using System; +using System.Collections.Generic; +using System.Data; +using System.Data.Common; +using System.Data.SqlClient; +using System.Globalization; +using System.Diagnostics; +using System.Runtime.ExceptionServices; +using System.Threading; +using mySQLPunk.lib.MssqlExport; + +namespace mySQLPunk.lib.MssqlImport +{ + internal interface IMssqlImportSession : IDisposable + { + MssqlImportTarget ReadTarget(CancellationToken cancellation); + MssqlImportSessionState ReadState(CancellationToken cancellation); + MssqlImportLobState ReadLobState(Guid jobId, CancellationToken cancellation); + void ExecuteBatch(MssqlImportBatch batch, CancellationToken cancellation); + MssqlImportCounts CountTargetRows(Action progress, CancellationToken cancellation); + void CancelActiveCommand(); + MssqlImportCleanup Cleanup(); + } + + internal sealed class MssqlImportSqlSession : IMssqlImportSession + { + private readonly object gate = new object(); + private readonly Func createCommand; + private readonly Func isOpen; + private readonly int cancelMilliseconds, cleanupReleaseWaitMilliseconds; + private IDisposable owner; + private ActiveCommand active; + private bool releaseStarted, releaseCompleted, sessionReleased, cleanupRunning; + private MssqlImportCleanup cleanup; + private IReadOnlyList releaseErrors; + + private sealed class ActiveCommand + { + internal readonly DbCommand Command; + internal CancellationTokenRegistration Registration; + internal Timer Watchdog; + internal bool CancellationRequested; + internal ActiveCommand(DbCommand command) { Command = command; } + } + + internal MssqlImportSqlSession(MssqlOwnedSession owner) + : this(owner, () => owner.Connection.CreateCommand(), () => owner.Connection.State == ConnectionState.Open) { } + + // The test seam substitutes only this connection's command factory and lifetime, never a shared provider. + internal MssqlImportSqlSession(IDisposable owner, Func createCommand, Func isOpen, int cancelMilliseconds = 10000, int cleanupReleaseWaitMilliseconds = 10000) + { + try + { + if (owner == null) throw new ArgumentNullException(nameof(owner)); + if (createCommand == null) throw new ArgumentNullException(nameof(createCommand)); + if (isOpen == null) throw new ArgumentNullException(nameof(isOpen)); + if (cancelMilliseconds <= 0) throw new ArgumentOutOfRangeException(nameof(cancelMilliseconds)); + if (cleanupReleaseWaitMilliseconds <= 0) throw new ArgumentOutOfRangeException(nameof(cleanupReleaseWaitMilliseconds)); + this.owner = owner; this.createCommand = createCommand; this.isOpen = isOpen; this.cancelMilliseconds = cancelMilliseconds; this.cleanupReleaseWaitMilliseconds = cleanupReleaseWaitMilliseconds; + } + catch { try { if (owner != null) owner.Dispose(); } catch { } throw; } + } + + public MssqlImportTarget ReadTarget(CancellationToken cancellation) + { + object[] identity = ReadValues(@"SELECT CONVERT(nvarchar(128),SERVERPROPERTY('ServerName')),DB_NAME(),DB_ID(), +(SELECT create_date FROM sys.databases WHERE database_id=DB_ID()), +CONVERT(nvarchar(128),SERVERPROPERTY('ProductVersion')), +CONVERT(nvarchar(60),DATABASEPROPERTYEX(DB_NAME(),'Updateability')),USER_ID(), +HAS_PERMS_BY_NAME(DB_NAME(),'DATABASE','VIEW DEFINITION'), +HAS_PERMS_BY_NAME(DB_NAME(),'DATABASE','CREATE TABLE'),HAS_PERMS_BY_NAME(DB_NAME(),'DATABASE','CREATE VIEW');", 5, cancellation); + if (identity == null || identity.Length != 10 || Missing(identity[0]) || Missing(identity[1]) || Missing(identity[2]) || Missing(identity[3]) || Missing(identity[4])) + throw new InvalidOperationException("TargetIdentityUnavailable"); + string server = Convert.ToString(identity[0], CultureInfo.InvariantCulture); + string database = Convert.ToString(identity[1], CultureInfo.InvariantCulture); + string version = Convert.ToString(identity[4], CultureInfo.InvariantCulture); + int databaseId = Convert.ToInt32(identity[2], CultureInfo.InvariantCulture), major; + if (string.IsNullOrWhiteSpace(server) || string.IsNullOrWhiteSpace(database) || databaseId <= 0 || + !int.TryParse(version.Split('.')[0], NumberStyles.None, CultureInfo.InvariantCulture, out major) || major <= 0 || !(identity[3] is DateTime)) + throw new InvalidOperationException("TargetIdentityUnavailable"); + bool complete = !Missing(identity[6]) && Convert.ToInt32(identity[6], CultureInfo.InvariantCulture) == 1; + long? count = null; + try + { + if (!complete && Permission(identity[7]) == true) + { + bool securityVisible = major < 16 || Permission(Scalar("SELECT HAS_PERMS_BY_NAME(DB_NAME(),'DATABASE','VIEW SECURITY DEFINITION');", 5, cancellation)) == true; + if (securityVisible) + { + // Scan raw permission rows for the whole effective token, including roles/public/Windows groups. + // Joining objects here could hide the very DENY that makes an apparent empty catalog unsafe. + object denied = Scalar(@"SELECT COUNT_BIG(*) FROM sys.database_permissions AS p +WHERE p.state='D' AND p.grantee_principal_id IN (SELECT principal_id FROM sys.user_token);", 5, cancellation); + complete = !Missing(denied) && Convert.ToInt64(denied, CultureInfo.InvariantCulture) == 0; + } + } + if (complete) + { + object objects = Scalar("SELECT COUNT_BIG(*) FROM sys.objects WHERE is_ms_shipped=0;", 5, cancellation); + if (!Missing(objects) && Convert.ToInt64(objects, CultureInfo.InvariantCulture) >= 0) count = Convert.ToInt64(objects, CultureInfo.InvariantCulture); + else complete = false; + } + } + catch (MssqlImportBatchException error) + { + // Lack of metadata is acceptable for Generic/DataOnly, but a broken/cancelled session is not. + if (error.OutcomeUncertain || cancellation.IsCancellationRequested || !ConnectionUsable()) throw; + complete = false; count = null; + } + return new MssqlImportTarget(server, database, databaseId, (DateTime)identity[3], version, + string.Equals(Convert.ToString(identity[5], CultureInfo.InvariantCulture), "READ_WRITE", StringComparison.OrdinalIgnoreCase), + complete, complete ? count : null, Permission(identity[8]), Permission(identity[9])); + } + + public MssqlImportSessionState ReadState(CancellationToken cancellation) + { + cancellation.ThrowIfCancellationRequested(); + EnsureNoActiveCommand(); + if (!ConnectionUsable()) return new MssqlImportSessionState(false, 0, 0, 0); + // Observed on SQL17.0.4025.3: DB_ID metadata work exposed XACT_STATE=1 in the same SELECT at @@TRANCOUNT=0. + // Complete that statement first, then observe the actual caller transaction without normalizing its state. + object[] state = ReadValues("DECLARE @databaseId int=DB_ID(); SELECT @databaseId,@@TRANCOUNT,XACT_STATE();", 5, cancellation); + if (state == null || state.Length != 3 || Missing(state[0]) || Missing(state[1]) || Missing(state[2]) || Convert.ToInt32(state[0], CultureInfo.InvariantCulture) <= 0) + throw new InvalidOperationException("SessionStateUnavailable"); + return new MssqlImportSessionState(true, Convert.ToInt32(state[0], CultureInfo.InvariantCulture), + Convert.ToInt32(state[1], CultureInfo.InvariantCulture), Convert.ToInt32(state[2], CultureInfo.InvariantCulture)); + } + + public MssqlImportLobState ReadLobState(Guid jobId, CancellationToken cancellation) + { + if (jobId == Guid.Empty) throw new ArgumentException("A native LOB job identity is required.", nameof(jobId)); + string name = "#mp_" + jobId.ToString("N") + "_s"; + object[] state = ReadValues("IF OBJECT_ID(N'tempdb.." + name + "') IS NULL " + + "SELECT CAST(0 AS bit),CAST(0 AS bigint),CAST(0 AS bigint),CAST(0 AS bigint),CAST(0 AS bit),CAST(0 AS bit); " + + "ELSE SELECT CAST(1 AS bit),COUNT_BIG(*),COALESCE(MAX([Sequence]),0),COALESCE(MAX([ConsumedThrough]),0)," + + "CAST(COALESCE(MIN(CAST([Consumed] AS int)),0) AS bit),CAST(COALESCE(MAX(CAST([Failed] AS int)),0) AS bit) FROM " + Quote(name) + ";", 5, cancellation); + if (state == null || state.Length != 6) throw new InvalidOperationException("LobStateUnavailable"); + for (int i = 0; i < state.Length; i++) if (Missing(state[i])) throw new InvalidOperationException("LobStateUnavailable"); + return new MssqlImportLobState(Convert.ToBoolean(state[0], CultureInfo.InvariantCulture), Convert.ToInt64(state[1], CultureInfo.InvariantCulture), + Convert.ToInt64(state[2], CultureInfo.InvariantCulture), Convert.ToInt64(state[3], CultureInfo.InvariantCulture), + Convert.ToBoolean(state[4], CultureInfo.InvariantCulture), Convert.ToBoolean(state[5], CultureInfo.InvariantCulture)); + } + + public void ExecuteBatch(MssqlImportBatch batch, CancellationToken cancellation) + { + if (batch == null) throw new ArgumentNullException(nameof(batch)); + WithReader(batch.Text, 0, cancellation, reader => { Drain(reader, cancellation); return true; }); + } + + public MssqlImportCounts CountTargetRows(Action progress, CancellationToken cancellation) + { + var target = ReadTarget(cancellation); + if (!target.MetadataVisible) throw new InvalidOperationException("TargetMetadataIncomplete"); + int after = 0; long tables = 0, rows = 0; + while (true) + { + cancellation.ThrowIfCancellationRequested(); + // Keyset lookup closes before COUNT_BIG, so MARS is unnecessary and no catalog list grows in memory. + object[] table = ReadValues("SELECT TOP(1) t.object_id,s.name,t.name FROM sys.tables AS t JOIN sys.schemas AS s ON s.schema_id=t.schema_id " + + "WHERE t.is_ms_shipped=0 AND t.object_id>" + after.ToString(CultureInfo.InvariantCulture) + " ORDER BY t.object_id;", 5, cancellation); + if (table == null) break; + if (table.Length != 3 || Missing(table[0]) || Missing(table[1]) || Missing(table[2])) throw new InvalidOperationException("TargetTableUnavailable"); + int next = Convert.ToInt32(table[0], CultureInfo.InvariantCulture); + if (next <= after) throw new InvalidOperationException("TargetTableOrderInvalid"); + object actual = Scalar("SELECT COUNT_BIG(*) FROM " + Quote(Convert.ToString(table[1], CultureInfo.InvariantCulture)) + "." + + Quote(Convert.ToString(table[2], CultureInfo.InvariantCulture)) + ";", 0, cancellation); + if (Missing(actual)) throw new InvalidOperationException("TargetCountUnavailable"); + long count = Convert.ToInt64(actual, CultureInfo.InvariantCulture); + if (count < 0) throw new InvalidOperationException("TargetCountUnavailable"); + rows = checked(rows + count); tables = checked(tables + 1); after = next; + if (progress != null) progress(tables, rows); + } + return new MssqlImportCounts(tables, rows); + } + + public void CancelActiveCommand() + { + ActiveCommand command; + lock (gate) command = active; + if (command != null) RequestCancellation(command); + } + + private void RequestCancellation(ActiveCommand command) + { + lock (gate) + { + if (active != command || command.CancellationRequested) return; + command.CancellationRequested = true; + command.Watchdog = new Timer(_ => ReleaseOwner(command), null, cancelMilliseconds, Timeout.Infinite); + } + try { command.Command.Cancel(); } catch { /* Completion/disposal can win this exact command's race. */ } + } + + public MssqlImportCleanup Cleanup() + { + lock (gate) + { + if (cleanup != null) return cleanup; + if (active != null) return new MssqlImportCleanup(false, sessionReleased, false, "CommandStillActive"); + if (cleanupRunning) return new MssqlImportCleanup(false, sessionReleased, false, "CleanupStillActive"); + cleanupRunning = true; + } + bool transactionConfirmed = false; + IReadOnlyList cleanupErrors = null; + string issue = null; + try + { + if (!ConnectionUsable()) issue = "SessionUnavailable"; + else + { + object[] state = ReadValues("IF @@TRANCOUNT>0 ROLLBACK TRANSACTION; SELECT @@TRANCOUNT,XACT_STATE();", 10, CancellationToken.None, true); + transactionConfirmed = state != null && state.Length == 2 && !Missing(state[0]) && !Missing(state[1]) && + Convert.ToInt32(state[0], CultureInfo.InvariantCulture) == 0 && Convert.ToInt32(state[1], CultureInfo.InvariantCulture) == 0; + if (!transactionConfirmed) issue = "TransactionCleanupUnconfirmed"; + } + } + catch (MssqlImportBatchException error) { issue = "TransactionCleanupFailed"; cleanupErrors = error.Errors; } + catch { issue = "TransactionCleanupFailed"; } + finally { ReleaseOwner(null); } + lock (gate) + { + // A watchdog may still be inside Dispose after the provider has unwound. + // Take the final diagnostic snapshot only after publication or this monotonic bound. + var releaseWait = Stopwatch.StartNew(); + while (releaseStarted && !releaseCompleted) + { + long remaining = cleanupReleaseWaitMilliseconds - releaseWait.ElapsedMilliseconds; + if (remaining <= 0) break; + Monitor.Wait(gate, (int)remaining); + } + if (!sessionReleased) + { + string releaseIssue = releaseCompleted ? "SessionReleaseFailed" : "SessionReleasePending"; + issue = issue == null ? releaseIssue : issue + ";" + releaseIssue; + } + var errors = new List(); + if (cleanupErrors != null) errors.AddRange(cleanupErrors); + if (releaseErrors != null) errors.AddRange(releaseErrors); + cleanup = new MssqlImportCleanup(active == null, sessionReleased, transactionConfirmed, issue, errors); + cleanupRunning = false; + return cleanup; + } + } + + public void Dispose() { ReleaseOwner(null); } + + private void ReleaseOwner(ActiveCommand requiredActive) + { + IDisposable value; + lock (gate) + { + if (requiredActive != null && active != requiredActive || releaseStarted) return; + // Latch release under the same gate as the generation check, before any later command can start. + releaseStarted = true; value = owner; owner = null; + } + bool released = false; + IReadOnlyList details = null; + try { value.Dispose(); released = true; } + catch (SqlException error) + { + var errors = new List(); + foreach (SqlError detail in error.Errors) + errors.Add(new MssqlImportSqlError(detail.Number, detail.State, detail.Class, detail.LineNumber, detail.Message)); + details = errors.AsReadOnly(); + } + catch { /* Non-SQL release failures retain the fixed receipt code, never raw exception text. */ } + finally { lock (gate) { sessionReleased = released; releaseErrors = details; releaseCompleted = true; Monitor.PulseAll(gate); } } + } + + private bool ConnectionUsable() + { + lock (gate) { if (releaseStarted) return false; } + try { return isOpen(); } catch { return false; } + } + + private void EnsureNoActiveCommand() + { + lock (gate) { if (active != null || cleanupRunning) throw new InvalidOperationException("ImportCommandAlreadyActive"); } + } + + private T WithReader(string sql, int timeout, CancellationToken cancellation, Func action, bool cleaning = false) + { + cancellation.ThrowIfCancellationRequested(); + ActiveCommand owned; + lock (gate) + { + if (active != null || cleanupRunning && !cleaning) throw new InvalidOperationException("ImportCommandAlreadyActive"); + if (releaseStarted) throw new ObjectDisposedException(nameof(MssqlImportSqlSession)); + DbCommand command = createCommand(); + try { command.CommandText = sql; command.CommandTimeout = timeout; owned = new ActiveCommand(command); active = owned; } + catch { command.Dispose(); throw; } + } + DbDataReader reader = null; + var errors = new List(); + Exception failure = null; bool uncertain = false; + T result = default(T); + try + { + owned.Registration = cancellation.Register(() => RequestCancellation(owned)); + reader = owned.Command.ExecuteReaderAsync(CommandBehavior.SequentialAccess, cancellation).GetAwaiter().GetResult(); + result = action(reader); + cancellation.ThrowIfCancellationRequested(); + } + catch (Exception error) { failure = error; AddErrors(error, errors, ref uncertain); } + finally + { + try { if (reader != null) reader.Dispose(); } + catch (Exception error) { if (failure == null) failure = error; AddErrors(error, errors, ref uncertain); } + try { owned.Command.Dispose(); } + catch (Exception error) { if (failure == null) failure = error; AddErrors(error, errors, ref uncertain); } + try { owned.Registration.Dispose(); } + finally + { + lock (gate) + { + if (owned.Watchdog != null) owned.Watchdog.Dispose(); + if (active == owned) active = null; + } + } + } + if (failure != null) + { + if (errors.Count != 0 || !(failure is OperationCanceledException) || !cancellation.IsCancellationRequested) + throw new MssqlImportBatchException(errors, uncertain || failure is OperationCanceledException || cancellation.IsCancellationRequested || !ConnectionUsable()); + ExceptionDispatchInfo.Capture(failure).Throw(); + } + cancellation.ThrowIfCancellationRequested(); + return result; + } + + private static void AddErrors(Exception error, List errors, ref bool uncertain) + { + var sql = error as SqlException; + if (sql == null) { if (!(error is OperationCanceledException)) uncertain = true; return; } + foreach (SqlError detail in sql.Errors) + { + errors.Add(new MssqlImportSqlError(detail.Number, detail.State, detail.Class, detail.LineNumber, detail.Message)); + if (detail.Number == -2 || detail.Number == 0 || detail.Number == 64 || detail.Number == 233 || + detail.Number == 10053 || detail.Number == 10054 || detail.Number == 10060 || detail.Class >= 20) uncertain = true; + } + } + + private object Scalar(string sql, int timeout, CancellationToken cancellation) + { + object[] values = ReadValues(sql, timeout, cancellation); + if (values == null || values.Length != 1) throw new InvalidOperationException("ProbeResultUnavailable"); + return values[0]; + } + + private object[] ReadValues(string sql, int timeout, CancellationToken cancellation, bool cleaning = false) + { + return WithReader(sql, timeout, cancellation, reader => + { + object[] values = null; + if (reader.ReadAsync(cancellation).GetAwaiter().GetResult()) + { + if (reader.FieldCount > 16) throw new InvalidOperationException("ProbeShapeInvalid"); + values = new object[reader.FieldCount]; + for (int i = 0; i < values.Length; i++) values[i] = reader.GetValue(i); + if (reader.ReadAsync(cancellation).GetAwaiter().GetResult()) throw new InvalidOperationException("ProbeRowCountInvalid"); + } + if (reader.NextResultAsync(cancellation).GetAwaiter().GetResult()) throw new InvalidOperationException("ProbeResultCountInvalid"); + return values; + }, cleaning); + } + + private static void Drain(DbDataReader reader, CancellationToken cancellation) + { + do { while (reader.ReadAsync(cancellation).GetAwaiter().GetResult()) { cancellation.ThrowIfCancellationRequested(); } } + while (reader.NextResultAsync(cancellation).GetAwaiter().GetResult()); + } + + private static bool Missing(object value) { return value == null || value == DBNull.Value; } + private static bool? Permission(object value) + { + if (Missing(value)) return null; + int permission = Convert.ToInt32(value, CultureInfo.InvariantCulture); + return permission == 1 ? true : permission == 0 ? (bool?)false : null; + } + private static string Quote(string identifier) { return "[" + identifier.Replace("]", "]]") + "]"; } + } +} diff --git a/mySQLPunk/mySQLPunk.csproj b/mySQLPunk/mySQLPunk.csproj index 1c539b1..ec62418 100644 --- a/mySQLPunk/mySQLPunk.csproj +++ b/mySQLPunk/mySQLPunk.csproj @@ -300,7 +300,9 @@ + + @@ -364,6 +366,14 @@ + + + + + + + + diff --git a/tests/MssqlImportFixtures/expected-cases.json b/tests/MssqlImportFixtures/expected-cases.json new file mode 100644 index 0000000..f69dae0 --- /dev/null +++ b/tests/MssqlImportFixtures/expected-cases.json @@ -0,0 +1,78 @@ +{ + "schemaVersion": 1, + "Core": [ + "core.native-schema-data", + "core.native-schema-only", + "core.native-data-only", + "core.generic-session-temp-set", + "core.generic-identity-insert", + "core.generic-cross-go-transaction", + "core.generic-multiple-results", + "core.catalog-identities-indexes", + "core.catalog-fk-check-trust", + "core.catalog-views-properties", + "core.parser-failure-no-write", + "core.target-identity-recheck" + ], + "Lob": [ + "lob.binary-64m-roundtrip", + "lob.unicode-18m-roundtrip", + "lob.null-empty-xml-roundtrip", + "lob.native-data-only", + "lob.chunk-missing", + "lob.chunk-duplicate", + "lob.chunk-reordered", + "lob.final-replayed", + "lob.final-constraint-failed0tran", + "lob.xml-conversion-failed0tran", + "lob.state-missing-after-init", + "lob.state-duplicate", + "lob.pending-eof", + "lob.sequence-eof" + ], + "Geometry": [ + "geometry.bytes-metadata-roundtrip", + "geometry.large-serialize-roundtrip", + "geometry.native-data-only", + "geometry.null-empty-invalid-curves", + "geometry.chunk-missing", + "geometry.chunk-duplicate", + "geometry.chunk-reordered", + "geometry.final-replayed", + "geometry.deserialize-failed0tran", + "geometry.final-constraint-failed0tran", + "geometry.pending-eof" + ], + "Faults": [ + "faults.partial-batch-stop", + "faults.partial-batch-skip", + "faults.sqlerror-all-details", + "faults.transaction-error", + "faults.xact-abort", + "faults.unclosed-transaction-eof", + "faults.finite-timeout", + "faults.owned-transport-loss", + "faults.log-create", + "faults.log-start", + "faults.log-error-append", + "faults.log-flush", + "faults.log-end", + "faults.cancel-result-read", + "faults.cancel-next-result", + "faults.cancel-state-probe", + "faults.cancel-lob-probe", + "faults.cancel-count-big", + "faults.cancel-close-race", + "faults.cancel-success-race", + "faults.metadata-low-privilege-boundary", + "faults.target-unavailable-boundary" + ], + "Memory": [ + "memory.drain-million-rows", + "memory.drain-64m-returned-lob", + "memory.warmup-x64", + "memory.stream-256m", + "memory.stream-2g5", + "memory.compare-peaks" + ] +} \ No newline at end of file diff --git a/tests/MssqlImportIntegration.Guards.ps1 b/tests/MssqlImportIntegration.Guards.ps1 new file mode 100644 index 0000000..0c72d09 --- /dev/null +++ b/tests/MssqlImportIntegration.Guards.ps1 @@ -0,0 +1,95 @@ +# Pure runner authorization/receipt/ownership guards; no SQL or filesystem mutations. +# Both runner and guard tests use these exact fail-closed decisions. +Set-StrictMode -Version 2.0 + +$script:ImportExpectedInstance = 'mySQLPunkExportTest_20261012' +$script:ImportExpectedServer = '(localdb)\mySQLPunkExportTest_20261012' +$script:ImportExpectedRoot = 'C:\Users\johnho\AppData\Local\Temp\mysqlpunk-export-20261012-7cadb673332543c3b17c61924490a702' +$script:ImportExpectedApproval = '2026-10-12 dedicated LocalDB creation and startup approved' + +function Get-ImportGuardPathInfo { + param([string]$Path) + if (!(Test-Path -LiteralPath $Path)) { return @{ Exists=$false; IsDirectory=$false; IsReparse=$false; Entries=0 } } + $item = Get-Item -LiteralPath $Path -Force -ErrorAction Stop + return @{ Exists=$true; IsDirectory=[bool]$item.PSIsContainer; IsReparse=[bool]($item.Attributes -band [IO.FileAttributes]::ReparsePoint); Entries=$(if ($item.PSIsContainer) { @(Get-ChildItem -LiteralPath $Path -Force -ErrorAction Stop | Select-Object -First 1).Count } else { 0 }) } +} +function Assert-ImportAuthorizationEnvelope { + param( + [bool]$Run, [object]$Manifest, [string]$Server, [string]$OutputDirectory, + [long]$FreeCBytes, + [scriptblock]$PathProbe = ${function:Get-ImportGuardPathInfo} + ) + # All arguments are observations. This function cannot create files/open SQL/start instances. + if (!$Run) { return @{ Enabled=$false; Code='OptInRequired' } } + if ($null -eq $Manifest) { throw 'AuthorizationMissing' } + if ($Server -cne $script:ImportExpectedServer) { throw 'ServerMismatch' } + if ($Manifest.Instance -cne $script:ImportExpectedInstance -or $Manifest.CreatedByTask -isnot [bool] -or !$Manifest.CreatedByTask) { throw 'AuthorizationMismatch' } + if ($Manifest.UserAuthorization -cne $script:ImportExpectedApproval -or $Manifest.EngineVersion -cne '17.0') { throw 'AuthorizationMismatch' } + if ([string]::IsNullOrWhiteSpace($Manifest.FixtureRoot)) { throw 'FixtureRootMissing' } + if ($Manifest.FixtureRoot -notmatch '^[cC]:[\\/]') { throw 'FixtureRootNotC' } + $root = [IO.Path]::GetFullPath($Manifest.FixtureRoot).TrimEnd([char[]]@('\','/')) + if ($root -ine $script:ImportExpectedRoot) { throw 'FixtureRootMismatch' } + if ([string]::IsNullOrWhiteSpace($OutputDirectory) -or $OutputDirectory -notmatch '^[cC]:[\\/]') { throw 'OutputMustBeAbsoluteC' } + $output = [IO.Path]::GetFullPath($OutputDirectory).TrimEnd([char[]]@('\','/')) + if (!$output.StartsWith($root+'\',[StringComparison]::OrdinalIgnoreCase)) { throw 'OutputMustBeStrictDescendant' } + $observed = & $PathProbe $output + if ($observed.Exists -and (!$observed.IsDirectory -or $observed.Entries -ne 0)) { throw 'OutputNotEmptyDirectory' } + $ancestor = $output + while ($ancestor) { + $observed = & $PathProbe $ancestor + if ($observed.Exists -and $observed.IsReparse) { throw 'ReparseFixturePath' } + $parent = [IO.Directory]::GetParent($ancestor) + $ancestor = if ($null -eq $parent) { $null } else { $parent.FullName } + } + if ($FreeCBytes -lt 20GB) { throw 'InsufficientFixtureSpace' } + return @{ Enabled=$true; Server=$Server; Root=$root; Output=$output } +} +function Assert-ImportProfileReceipt { + param([object]$Receipt,[string]$AcceptanceProfile,[string]$RunId,[string[]]$ExpectedCases) + # Caller separately hashes case receipt files and matches executable SHA/base, output/runId. + if (!$Receipt -or $Receipt.schemaVersion -ne 1 -or $Receipt.profile -cne $AcceptanceProfile -or $Receipt.runId -cne $RunId -or $Receipt.complete -cne $true -or $Receipt.processBits -ne 64) { throw 'ProfileReceiptIdentityMismatch' } + $actual = @($Receipt.cases | ForEach-Object { [string]$_.caseId }) + if ($actual.Count -ne $ExpectedCases.Count -or @($actual | Select-Object -Unique).Count -ne $actual.Count) { throw 'ProfileReceiptCaseCountMismatch' } + if (@(Compare-Object ($ExpectedCases | Sort-Object) ($actual | Sort-Object)).Count -ne 0) { throw 'ProfileReceiptCaseSetMismatch' } + foreach ($case in $Receipt.cases) { + if ($case.status -cne 'Passed' -or $case.executed -cne $true -or $case.assertionsPassed -lt 1 -or $case.assertionsFailed -ne 0 -or $case.receiptSha256 -notmatch '^[a-f0-9]{64}$') { throw ('CaseNotAccepted:'+ $case.caseId) } + } + if ($Receipt.cleanupStatus -cne 'Confirmed') { throw 'ProfileCleanupNotConfirmed' } +} + +# File-reading caller (future runner): explicit AuthorizationPath must exist and be a file; +# parse JSON with -ErrorAction Stop, pass only the fixed instance/root/approval facts above. +# Only AFTER this helper passes: inspect exact LocalDB running state, create output/token, +# launch fixed filter, verify receipts and hash identities, restore all env names in finally. +# Never authorize from receipt contents, start SQL, sweep prefixes, or use saved settings. +function Assert-ImportCleanupObservation { + param( + [guid]$RunGuid, [string]$DatabaseName, [string[]]$CreatedDatabases, + [string]$RunOutput, [string]$InMemoryOwner, [string]$FileOwner, [object]$DatabaseOwner, + [string[]]$PhysicalFiles, [string[]]$ExpectedFiles, + [scriptblock]$PathProbe = ${function:Get-ImportGuardPathInfo} + ) + # Pure observation only. Actual fixture reads token/property/files immediately before + # calling this; only the successful caller may then issue exact generated DB DROP. + # Never emit owner values, even on failed checks. + if ($RunGuid -eq [guid]::Empty) { throw 'CleanupRunIdentityMissing' } + $prefix='mysqlpunk_import_'+$RunGuid.ToString('N') + if ($DatabaseName -cne $prefix+'_src' -and $DatabaseName -cne $prefix+'_dst') { throw 'CleanupDatabaseNotGenerated' } + if ($CreatedDatabases -cnotcontains $DatabaseName) { throw 'CleanupDatabaseNotRecorded' } + if ([string]::IsNullOrEmpty($InMemoryOwner) -or $FileOwner -cne $InMemoryOwner -or $null -eq $DatabaseOwner -or [string]$DatabaseOwner -cne $InMemoryOwner) { throw 'CleanupDualOwnershipMismatch' } + if (!$PhysicalFiles -or $PhysicalFiles.Count -ne $ExpectedFiles.Count -or @($PhysicalFiles | Select-Object -Unique).Count -ne $PhysicalFiles.Count) { throw 'CleanupFileSetMismatch' } + $root=[IO.Path]::GetFullPath($RunOutput).TrimEnd([char[]]@([char]92,[char]47)) + foreach ($path in $PhysicalFiles) { + if ($path -notmatch '^[cC]:[\\/]') { throw 'CleanupFileNotC' } + $file=[IO.Path]::GetFullPath($path) + if (!$file.StartsWith($root+[IO.Path]::DirectorySeparatorChar,[StringComparison]::OrdinalIgnoreCase) -or $ExpectedFiles -inotcontains $file) { throw 'CleanupFileNotOwned' } + $ancestor=$file + while ($ancestor) { + $observed = & $PathProbe $ancestor + if ($observed.Exists -and $observed.IsReparse) { throw 'CleanupReparsePath' } + $parent=[IO.Directory]::GetParent($ancestor) + $ancestor=if ($null -eq $parent) {$null} else {$parent.FullName} + } + } + return $true +} \ No newline at end of file diff --git a/tests/MssqlImportTransfer.Guards.ps1 b/tests/MssqlImportTransfer.Guards.ps1 new file mode 100644 index 0000000..3342ff7 --- /dev/null +++ b/tests/MssqlImportTransfer.Guards.ps1 @@ -0,0 +1,207 @@ +# Pure trial guards over caller-supplied snapshots; no SQL, filesystem probes or authorization acquisition. +# Shared by the one-shot runner and offline guard tests. +Set-StrictMode -Version 2.0 + +function New-TransferGuardDecision { + param([bool]$Allowed, [string]$Code) + [pscustomobject]@{ Allowed = $Allowed; Code = $Code } +} + +function Test-TransferTrue { + param($Value) + return ($Value -is [bool] -and $Value) +} + +function Test-TransferTrialInvocation { + param([bool]$Run, [hashtable]$Request, [hashtable]$File, [hashtable]$Output) + # The opt-in guard comes before any snapshot requirement. A future runner must likewise perform no source/DB/output I/O here. + if (!$Run) { return New-TransferGuardDecision $false 'SkippedNoRun' } + if (!$Request -or !$File -or !$Output) { return New-TransferGuardDecision $false 'MissingDescriptor' } + if (![string]::Equals($Request.Server, 'localhost', [StringComparison]::OrdinalIgnoreCase)) { return New-TransferGuardDecision $false 'WrongServer' } + if (![string]::Equals($Request.Database, 'Transfer', [StringComparison]::Ordinal)) { return New-TransferGuardDecision $false 'WrongDatabase' } + $expectedInput = 'C:\Users\johnho\Desktop\Transfer_dump_20261013_165730.sql' + if (![string]::Equals($Request.InputCanonicalPath, $expectedInput, [StringComparison]::OrdinalIgnoreCase) -or + ![string]::Equals($File.CanonicalPath, $expectedInput, [StringComparison]::OrdinalIgnoreCase)) { return New-TransferGuardDecision $false 'WrongInput' } + if (!(Test-TransferTrue $File.RegularFile) -or !(Test-TransferTrue $File.Readable) -or !(Test-TransferTrue $File.NoReparseChain)) { return New-TransferGuardDecision $false 'UnsafeInput' } + if ($File.Length -isnot [long] -or $File.Length -ne 4478414151L) { return New-TransferGuardDecision $false 'InputLengthMismatch' } + # Snapshot producer must resolve final volume and every existing ancestor; drive letter alone is insufficient. + if (!(Test-TransferTrue $Output.OnLocalCVolume) -or !(Test-TransferTrue $Output.NoReparseChain) -or + !(Test-TransferTrue $Output.EmptyOrNew) -or !(Test-TransferTrue $Output.Writable)) { return New-TransferGuardDecision $false 'UnsafeOutput' } + if ([string]::IsNullOrWhiteSpace($Request.OutputCanonicalPath) -or + ![string]::Equals($Request.OutputCanonicalPath, $Output.CanonicalPath, [StringComparison]::OrdinalIgnoreCase) -or + !($Output.CanonicalPath -match '^C:\\[^\\]') -or $Output.CanonicalPath -match '(^|[\\/])\.\.([\\/]|$)') { return New-TransferGuardDecision $false 'OutputMismatch' } + $outputBase = $Output.CanonicalPath.TrimEnd('\') + if ([string]::Equals($outputBase, 'C:\Users\johnho\Desktop', [StringComparison]::OrdinalIgnoreCase) -or + [string]::Equals($outputBase, $expectedInput, [StringComparison]::OrdinalIgnoreCase)) { return New-TransferGuardDecision $false 'OutputTouchesInput' } + if ($Output.FreeBytes -isnot [long] -or $Output.RequiredBytes -isnot [long] -or + $Output.RequiredBytes -ne 68719476736L -or $Output.FreeBytes -lt $Output.RequiredBytes) { return New-TransferGuardDecision $false 'InsufficientSpace' } + return New-TransferGuardDecision $true 'InvocationAccepted' +} + +function Test-TransferTrialTarget { + param([hashtable]$Target, [hashtable]$Baseline, [long]$ExpectedTables, $ContainsViews) + if (!$Target) { return New-TransferGuardDecision $false 'TargetUnavailable' } + if (![string]::Equals($Target.ServerName, 'JOHN\MSSQL2019', [StringComparison]::OrdinalIgnoreCase) -or + ![string]::Equals($Target.DatabaseName, 'Transfer', [StringComparison]::Ordinal)) { return New-TransferGuardDecision $false 'WrongIdentity' } + if ($Target.DatabaseId -isnot [int] -or $Target.DatabaseId -le 0 -or $Target.DatabaseCreateDate -isnot [datetime]) { return New-TransferGuardDecision $false 'TargetIdentityUnavailable' } + if (!($Target.Version -is [string] -and $Target.Version -match '^15\.\d+\.\d+\.\d+$') -or $Target.Compatibility -ne 150 -or + $Target.State -cne 'ONLINE' -or !(Test-TransferTrue $Target.Writable)) { return New-TransferGuardDecision $false 'WrongTargetState' } + if (!(Test-TransferTrue $Target.WindowsIntegratedJohnho) -or $Target.WindowsSid -cne $Target.OriginalSqlSid -or $Target.WindowsSid -cne $Target.CurrentSqlSid -or $Target.WindowsName -ine $Target.OriginalSqlName -or $Target.WindowsName -ine $Target.CurrentSqlName) { return New-TransferGuardDecision $false 'WrongLogin' } + # MetadataVisible must come from the accepted Task 3 conservative proof, never a COUNT result alone. + if (!(Test-TransferTrue $Target.MetadataVisible) -or $Target.UserObjectCount -isnot [long]) { return New-TransferGuardDecision $false 'MetadataUnknown' } + if ($Target.UserObjectCount -ne 0L) { return New-TransferGuardDecision $false 'TargetNonempty' } + if ($ExpectedTables -gt 0L -and !(Test-TransferTrue $Target.CanCreateTable)) { return New-TransferGuardDecision $false 'CreateTableDenied' } + # No blanket CREATE VIEW requirement: gate only when scan has independently known view presence. + if ((Test-TransferTrue $ContainsViews) -and !(Test-TransferTrue $Target.CanCreateView)) { return New-TransferGuardDecision $false 'CreateViewDenied' } + if ($Baseline -and ($Target.DatabaseId -ne $Baseline.DatabaseId -or $Target.DatabaseCreateDate.Ticks -ne $Baseline.DatabaseCreateDate.Ticks -or + ![string]::Equals($Target.ServerName, $Baseline.ServerName, [StringComparison]::OrdinalIgnoreCase) -or + ![string]::Equals($Target.DatabaseName, $Baseline.DatabaseName, [StringComparison]::Ordinal))) { return New-TransferGuardDecision $false 'TargetIdentityChanged' } + if (!(Test-TransferTrue $Target.SettingsUnchanged)) { return New-TransferGuardDecision $false 'SettingsChanged' } + return New-TransferGuardDecision $true 'TargetAccepted' +} + +function Test-TransferTrialAttemptClaim { + param([hashtable]$Proposed, $ExistingMarker) + # Existing marker of ANY terminal/intermediate/corrupt state consumes the original one-attempt authority. + # Future caller atomically CreateNew+flushes a FIXED C-path marker, independent of OutputDirectory, before SUT.Run. + # This pure test neither creates nor removes that marker. A null value is insufficient without verified absence. + if (!$Proposed -or !(Test-TransferTrue $Proposed.StableMarkerAbsenceVerified)) { return New-TransferGuardDecision $false 'AttemptAbsenceUnknown' } + if ($null -ne $ExistingMarker) { return New-TransferGuardDecision $false 'AttemptAlreadyClaimed' } + if ($Proposed.Nonce -isnot [string] -or $Proposed.Nonce -cnotmatch '^[0-9a-f]{32}$' -or + $Proposed.JobId -isnot [guid] -or $Proposed.JobId -eq [guid]::Empty) { return New-TransferGuardDecision $false 'AttemptIdentityInvalid' } + if (!(Test-TransferTrue $Proposed.GuardsPassed) -or !(Test-TransferTrue $Proposed.PreflightPassed)) { return New-TransferGuardDecision $false 'AttemptNotReady' } + return New-TransferGuardDecision $true 'AtomicClaimRequired' +} + +function ConvertFrom-TransferBoundedJson { + param([byte[]]$Bytes) + if(!$Bytes -or $Bytes.Length -gt 1048576){throw 'BoundedJsonSize'} + $text=[Text.UTF8Encoding]::new($false,$true).GetString($Bytes) + $scan=[Newtonsoft.Json.JsonTextReader]::new([IO.StringReader]::new($text));$scan.MaxDepth=4;$scan.DateParseHandling=[Newtonsoft.Json.DateParseHandling]::None + $names=[Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase) + try{ + if(!$scan.Read() -or $scan.TokenType -ne [Newtonsoft.Json.JsonToken]::StartObject){throw 'FlatStrictJsonRequired'} + while($scan.Read()){ + if($scan.TokenType -eq [Newtonsoft.Json.JsonToken]::Comment -or $scan.TokenType -eq [Newtonsoft.Json.JsonToken]::StartObject -or $scan.TokenType -eq [Newtonsoft.Json.JsonToken]::StartArray){throw 'FlatStrictJsonRequired'} + if($scan.TokenType -eq [Newtonsoft.Json.JsonToken]::PropertyName -and !$names.Add([string]$scan.Value)){throw 'FlatStrictJsonDuplicate'} + } + }finally{$scan.Dispose()} + $settings=[Newtonsoft.Json.Linq.JsonLoadSettings]::new() + $settings.DuplicatePropertyNameHandling=[Newtonsoft.Json.Linq.DuplicatePropertyNameHandling]::Error + $reader=[Newtonsoft.Json.JsonTextReader]::new([IO.StringReader]::new($text)) + $reader.MaxDepth=4;$reader.DateParseHandling=[Newtonsoft.Json.DateParseHandling]::None + try{ + $object=[Newtonsoft.Json.Linq.JObject]::Load($reader,$settings) + if($reader.Read()){throw 'BoundedJsonTrailingData'} + $result=@{} + foreach($property in $object.Properties()){ + if($result.ContainsKey($property.Name) -or $property.Value -isnot [Newtonsoft.Json.Linq.JValue]){throw 'FlatStrictJsonRequired'} + $result[$property.Name]=$property.Value.Value + } + return $result + }finally{$reader.Dispose()} +} +function Test-TransferTrialRequest { + param([hashtable]$Request) + $keys=@('SchemaVersion','Kind','Nonce','JobId','Server','DatabaseName','InputCanonicalPath','OutputCanonicalPath','RequiredFreeBytes','CancelBelowBytes','SourceCommit','SourceSnapshotSha256','ExecutableSha256','ProductExecutableSha256','ProductExecutablePath','RegistrationSha256') + if(!$Request -or $Request.Count -ne $keys.Count){return New-TransferGuardDecision $false 'RequestShape'} + foreach($key in $Request.Keys){if($key -cnotin $keys){return New-TransferGuardDecision $false 'RequestShape'}} + foreach($key in $keys){if(!$Request.ContainsKey($key)){return New-TransferGuardDecision $false 'RequestShape'}} + foreach($number in @('SchemaVersion','RequiredFreeBytes','CancelBelowBytes')){if($Request[$number] -isnot [long]){return New-TransferGuardDecision $false 'RequestType'}} + if($Request.SchemaVersion -ne 2 -or $Request.Kind -cne 'Request' -or $Request.Server -cne 'localhost' -or $Request.DatabaseName -cne 'Transfer' -or + $Request.InputCanonicalPath -cne 'C:\Users\johnho\Desktop\Transfer_dump_20261013_165730.sql' -or $Request.RequiredFreeBytes -ne 68719476736L -or $Request.CancelBelowBytes -ne 21474836480L -or $Request.ProductExecutablePath -cne 'C:\Users\johnho\Documents\mySQLPunk-worktrees\mssql-import\mySQLPunk\bin\CodexVerify\mySQLPunk.exe'){return New-TransferGuardDecision $false 'RequestPolicy'} + if($Request.Nonce -isnot [string] -or $Request.Nonce -cnotmatch '^[a-f0-9]{32}$' -or $Request.JobId -isnot [string] -or $Request.JobId -cnotmatch '^[a-f0-9]{8}(-[a-f0-9]{4}){3}-[a-f0-9]{12}$' -or [guid]$Request.JobId -eq [guid]::Empty -or [guid]::ParseExact($Request.Nonce,'N') -eq [guid]::Empty){return New-TransferGuardDecision $false 'RequestIdentity'} + if($Request.SourceCommit -isnot [string] -or $Request.SourceCommit -cnotmatch '^[a-f0-9]{40}$'){return New-TransferGuardDecision $false 'BuildPinInvalid'} + foreach($key in @('SourceSnapshotSha256','ExecutableSha256','ProductExecutableSha256','RegistrationSha256')){if($Request[$key] -isnot [string] -or $Request[$key] -cnotmatch '^[a-f0-9]{64}$'){return New-TransferGuardDecision $false 'BuildPinInvalid'}} + if($Request.OutputCanonicalPath -isnot [string] -or $Request.OutputCanonicalPath -notmatch '^C:\\[^\\]' -or $Request.OutputCanonicalPath -match '(^|[\\/])\.\.([\\/]|$)'){return New-TransferGuardDecision $false 'RequestPath'} + return New-TransferGuardDecision $true 'RequestAccepted' +} +function Test-TransferTrialCompletionReceipt { + param([hashtable]$Expected,[hashtable]$Marker,[hashtable]$Receipt,[string]$ActualMarkerSha256) + if(!$Expected -or !$Receipt -or !$Marker){return New-TransferGuardDecision $false 'ReceiptMissing'} + foreach($number in @('SchemaVersion','ProcessId','DatabaseId','DatabaseCreateDateTicks')){if($Receipt[$number] -isnot [long] -or $Marker[$number] -isnot [long]){return New-TransferGuardDecision $false 'ReceiptType'}} + foreach($number in @('FailedBatches','SkippedBytes','TotalErrors','CleanupErrors','ActualTables','ActualRows')){if($Receipt[$number] -isnot [long]){return New-TransferGuardDecision $false 'ReceiptType'}} + if(!(Test-TransferTrue $Receipt.Executed) -or !(Test-TransferTrue $Receipt.AttemptMarkerDurable) -or $Receipt.SchemaVersion -ne 2 -or $Receipt.Kind -cne 'Final' -or $Marker.Kind -cne 'Attempt' -or $Marker.SchemaVersion -ne 2){return New-TransferGuardDecision $false 'ReceiptNotExecuted'} + $allowedMarker=@('SchemaVersion','Kind','Nonce','JobId','Server','DatabaseName','InputCanonicalPath','OutputCanonicalPath','RequiredFreeBytes','CancelBelowBytes','SourceCommit','SourceSnapshotSha256','ExecutableSha256','ProductExecutableSha256','ProductExecutablePath','RegistrationSha256','ProcessId','RequestSha256','ServerName','DatabaseId','DatabaseCreateDateTicks','SourceSha256','WindowsSid','WindowsName','StartedUtc') + $allowedReceipt=$allowedMarker+@('MarkerSha256','Executed','AttemptMarkerDurable','Status','FailedBatches','SkippedBytes','TotalErrors','CleanupErrors','UncertainBatch','ActualTables','ActualRows','FailureCode','SecondaryFailureCode','ObjectVerificationBoundary','FieldHashEqualityVerified','ServiceCompletionValidated','CommandEnded','SessionReleased','TransactionCleanupConfirmed','LogComplete','IndependentVerificationCompleted','IndependentCatalogCaptured','MonitorComplete','LogPath','LastAcknowledgedBatch','SuccessfulBatches','ProcessedBytes','ScanBatchCount','MaxBatchCodeUnits','NativeJobId','EncodingName','SourceReadMode','FinishedUtc','PreRunLogUsedBytes','PreRunLogUsedAvailability','PreRunLogUsedUtc','TerminalLogUsedBytes','TerminalLogUsedAvailability','TerminalLogUsedUtc') + foreach($key in $Marker.Keys){if($key -cnotin $allowedMarker){return New-TransferGuardDecision $false 'ReceiptShape'}} + foreach($key in $Receipt.Keys){if($key -cnotin $allowedReceipt){return New-TransferGuardDecision $false 'ReceiptShape'}} + foreach($key in @('Nonce','JobId','Server','RequiredFreeBytes','CancelBelowBytes','SourceCommit','SourceSnapshotSha256','ExecutableSha256','ProductExecutableSha256','ProductExecutablePath','RegistrationSha256','InputCanonicalPath','OutputCanonicalPath','RequestSha256','ProcessId','WindowsSid','WindowsName')){ + if(!$Receipt.ContainsKey($key) -or !$Marker.ContainsKey($key) -or $Receipt[$key] -cne $Expected[$key] -or $Marker[$key] -cne $Expected[$key]){return New-TransferGuardDecision $false 'ReceiptIdentityMismatch'} + } + foreach($key in @('ServerName','DatabaseName','DatabaseId','DatabaseCreateDateTicks','SourceSha256','WindowsSid','WindowsName')){ + if(!$Receipt.ContainsKey($key) -or !$Marker.ContainsKey($key) -or $Receipt[$key] -cne $Marker[$key]){return New-TransferGuardDecision $false 'ReceiptIdentityMismatch'} + } + if($Receipt.ServerName -cne 'JOHN\MSSQL2019' -or $Receipt.DatabaseName -cne 'Transfer' -or $Receipt.DatabaseId -isnot [long] -or $Receipt.DatabaseId -le 0L -or $Receipt.DatabaseCreateDateTicks -isnot [long] -or $Receipt.DatabaseCreateDateTicks -le 0L){return New-TransferGuardDecision $false 'ReceiptIdentityMismatch'} + foreach($key in @('SourceSha256','RequestSha256','MarkerSha256','ExecutableSha256','ProductExecutableSha256','SourceSnapshotSha256','RegistrationSha256')){if($Receipt[$key] -isnot [string] -or $Receipt[$key] -cnotmatch '^[a-f0-9]{64}$'){return New-TransferGuardDecision $false 'ReceiptIdentityMismatch'}} + if($Receipt.MarkerSha256 -cne $ActualMarkerSha256){return New-TransferGuardDecision $false 'MarkerHashMismatch'} + if($Receipt.Status -cne 'Completed' -or $Receipt.FailedBatches -ne 0L -or $Receipt.SkippedBytes -ne 0L -or $Receipt.TotalErrors -ne 0L -or $Receipt.CleanupErrors -ne 0L -or $null -ne $Receipt.UncertainBatch -or $Receipt.FailureCode -cne '' -or $Receipt.SecondaryFailureCode -cne ''){return New-TransferGuardDecision $false 'TrialNotSuccessful'} + foreach($proof in @('ServiceCompletionValidated','CommandEnded','SessionReleased','TransactionCleanupConfirmed','LogComplete','IndependentVerificationCompleted','IndependentCatalogCaptured','MonitorComplete')){ + if(!(Test-TransferTrue $Receipt[$proof])){return New-TransferGuardDecision $false ('ReceiptProofMissing:'+$proof)} + } + if($Receipt.ActualTables -isnot [long] -or $Receipt.ActualRows -isnot [long] -or $Receipt.ActualTables -ne 126L -or $Receipt.ActualRows -ne 3247785L){return New-TransferGuardDecision $false 'IndependentCountMismatch'} + if($Receipt.ObjectVerificationBoundary -cne 'InventoryOnly;SourceDefinitionEqualityUnverified' -or $Receipt.FieldHashEqualityVerified -isnot [bool] -or $Receipt.FieldHashEqualityVerified){return New-TransferGuardDecision $false 'VerificationBoundaryMismatch'} + foreach($phase in @('PreRun','Terminal')){ + $value=$Receipt[($phase+'LogUsedBytes')];$availability=$Receipt[($phase+'LogUsedAvailability')] + if(($availability -ceq 'LogUsedAvailable' -and ($value -isnot [long] -or $value -lt 0L)) -or + ($availability -ceq 'LogUsedUnavailable' -and $null -ne $value) -or $availability -cnotin @('LogUsedAvailable','LogUsedUnavailable') -or + $Receipt[($phase+'LogUsedUtc')] -isnot [string] -or [string]::IsNullOrWhiteSpace($Receipt[($phase+'LogUsedUtc')])){return New-TransferGuardDecision $false 'UsedLogEvidenceInvalid'} + } + return New-TransferGuardDecision $true 'TrialVerifiedWithinReceiptBoundary' +} +function Get-TransferTrialEnvironment { + param([hashtable]$Inherited,[hashtable]$Trial) + $copy=@{};foreach($key in $Inherited.Keys){$copy[$key]=$Inherited[$key]} + foreach($key in @('MYSQLPUNK_MSSQL_IMPORT_INTEGRATION','MYSQLPUNK_MSSQL_IMPORT_AUTH','MYSQLPUNK_MSSQL_IMPORT_PROFILE','MYSQLPUNK_MSSQL_IMPORT_FIXTURES','MYSQLPUNK_MSSQL_IMPORT_MEMORY_SCALE','MYSQLPUNK_MSSQL_IMPORT_RUN_ID','MYSQLPUNK_MSSQL_IMPORT_WIZARD_PREVIEW','MYSQLPUNK_MSSQL_WIZARD_PREVIEW')){$copy[$key]=$null} + foreach($key in $Trial.Keys){$copy[$key]=$Trial[$key]} + return $copy +} +function Test-TransferTrialMonitorHealth { + param([long]$FreeCBytes,[bool]$IoSucceeded) + if(!$IoSucceeded){return New-TransferGuardDecision $false 'CancelMonitorIo'} + if($FreeCBytes -lt 21474836480L){return New-TransferGuardDecision $false 'CancelLowSpace'} + return New-TransferGuardDecision $true 'MonitorHealthy' +} +function New-TransferTrialMonitorRecord { + param([guid]$Nonce, [datetime]$Utc, [int]$ProcessId, [long]$Sequence, [long]$ElapsedMilliseconds, + [Nullable[long]]$ProcessCpuMilliseconds, [Nullable[long]]$PeakPrivateBytes, [Nullable[long]]$PeakWorkingSetBytes, + [Nullable[long]]$DataAllocatedBytes, [Nullable[long]]$LogAllocatedBytes, [Nullable[long]]$LogUsedBytes, + [long]$LastAcknowledgedBatch, [string]$Stage, [string]$MetricAvailability) + # Caller serializes ONE JSONL record directly to a held C-only stream; no SQL text, row payload or connection string accepted. + if ($Nonce -eq [guid]::Empty -or $ProcessId -le 0 -or $Sequence -lt 0L -or $ElapsedMilliseconds -lt 0L) { throw 'InvalidMonitorIdentity' } + if ($Stage -cnotin @('Scanning','Executing','Verifying','Stopping','Finished')) { throw 'InvalidMonitorStage' } + if ($MetricAvailability -cnotin @('Available','DatabaseMetricsUnavailable','ProcessMetricsUnavailable','MetricsUnavailable')) { throw 'InvalidMonitorAvailability' } + foreach ($metric in @($ProcessCpuMilliseconds,$PeakPrivateBytes,$PeakWorkingSetBytes,$DataAllocatedBytes,$LogAllocatedBytes,$LogUsedBytes)) { + if ($null -ne $metric -and $metric -lt 0L) { throw 'InvalidNegativeMonitorMetric' } + } + $processMissing=($null -eq $ProcessCpuMilliseconds -or $null -eq $PeakPrivateBytes -or $null -eq $PeakWorkingSetBytes) + $databaseMissing=($null -eq $DataAllocatedBytes -or $null -eq $LogAllocatedBytes -or $null -eq $LogUsedBytes) + if (($MetricAvailability -ceq 'Available' -and ($processMissing -or $databaseMissing)) -or + ($MetricAvailability -ceq 'DatabaseMetricsUnavailable' -and $processMissing) -or + ($MetricAvailability -ceq 'ProcessMetricsUnavailable' -and $databaseMissing)) { throw 'MonitorAvailabilityMismatch' } + [pscustomobject][ordered]@{ schemaVersion=1; nonce=$Nonce.ToString('N'); utc=$Utc.ToUniversalTime().ToString('o'); processId=$ProcessId; + sequence=$Sequence; elapsedMilliseconds=$ElapsedMilliseconds; processCpuMilliseconds=$ProcessCpuMilliseconds; + peakPrivateBytes=$PeakPrivateBytes; peakWorkingSetBytes=$PeakWorkingSetBytes; + dataAllocatedBytes=$DataAllocatedBytes; logAllocatedBytes=$LogAllocatedBytes; logUsedBytes=$LogUsedBytes; + lastAcknowledgedBatch=$LastAcknowledgedBatch; stage=$Stage; metricAvailability=$MetricAvailability } +} +function Test-TransferTrialBuildSnapshot { + param([hashtable]$Pin,[string]$ActualExecutableSha256,[string]$ActualProductExecutableSha256,[string]$ActualSourceSnapshotSha256,[string]$ActualRegistrationSha256,[bool]$RegistryContainsTrial) + if(!$Pin -or !(Test-TransferTrue $Pin.FinalPinAccepted) -or !(Test-TransferTrue $Pin.TransferTrialRegistered) -or !$RegistryContainsTrial){return New-TransferGuardDecision $false 'BuildPinNotReleased'} + foreach($entry in @(@('ExecutableSha256',$ActualExecutableSha256),@('ProductExecutableSha256',$ActualProductExecutableSha256),@('SourceSnapshotSha256',$ActualSourceSnapshotSha256),@('RegistrationSha256',$ActualRegistrationSha256))){ + if($Pin[$entry[0]] -isnot [string] -or $Pin[$entry[0]] -cnotmatch '^[a-f0-9]{64}$' -or $Pin[$entry[0]] -cne $entry[1]){return New-TransferGuardDecision $false 'BuildSnapshotChanged'} + } + return New-TransferGuardDecision $true 'BuildSnapshotAccepted' +} + +function Test-TransferTrialMarkerPresence { + param([hashtable]$Receipt,[bool]$MarkerExists,[int]$ChildExitCode) + if($MarkerExists){return New-TransferGuardDecision $true 'TrialMarkerPresent'} + if($Receipt -and $Receipt.Executed -is [bool] -and !$Receipt.Executed -and + $Receipt.AttemptMarkerDurable -is [bool] -and !$Receipt.AttemptMarkerDurable -and + $Receipt.Status -ceq 'NotStarted' -and $ChildExitCode -ne 0 -and $Receipt.FailureCode -is [string] -and + $Receipt.FailureCode -cmatch '^Trial[A-Za-z0-9:]{1,148}$'){ + return New-TransferGuardDecision $false ('TrialNotStarted:'+$Receipt.FailureCode) + } + return New-TransferGuardDecision $false 'TrialAttemptMarkerMissing' +} diff --git a/tests/Run-MssqlImportIntegration.ps1 b/tests/Run-MssqlImportIntegration.ps1 new file mode 100644 index 0000000..1962606 --- /dev/null +++ b/tests/Run-MssqlImportIntegration.ps1 @@ -0,0 +1,58 @@ +param( + [switch]$Run, + [string]$AuthorizationPath, + [Alias('Profile')][ValidateSet('Core','Lob','Geometry','Faults','Memory','All')][string]$AcceptanceProfile='Core', + [string]$OutputDirectory, + [switch]$SkipBuild +) +$ErrorActionPreference='Stop' +. (Join-Path $PSScriptRoot 'MssqlImportIntegration.Guards.ps1') +if(!$Run) { + Write-Host 'Opt-in isolated native MSSQL importer acceptance. Supply -Run -AuthorizationPath -Profile Core|Lob|Geometry|Faults|Memory|All -OutputDirectory .' + Write-Host 'Requires the exact already-running authorized LocalDB instance. No auto-start, saved connections or sqlcmd restore.' + return +} +if([string]::IsNullOrWhiteSpace($AuthorizationPath) -or !(Test-Path -LiteralPath $AuthorizationPath -PathType Leaf)) {throw 'Explicit authorization manifest required.'} +$authorization=[IO.Path]::GetFullPath($AuthorizationPath) +$manifest=Get-Content -LiteralPath $authorization -Raw -ErrorAction Stop | ConvertFrom-Json +$approval=Assert-ImportAuthorizationEnvelope -Run $true -Manifest $manifest -Server $script:ImportExpectedServer -OutputDirectory $OutputDirectory -FreeCBytes (Get-PSDrive C).Free +$instanceOutput=& sqllocaldb.exe info 'mySQLPunkExportTest_20261012' +if($LASTEXITCODE -ne 0 -or !($instanceOutput -match 'LOCALDB#')) {throw 'Authorized LocalDB must already be running; runner never starts it.'} +[IO.Directory]::CreateDirectory($approval.Output) | Out-Null +$names=@('MYSQLPUNK_MSSQL_IMPORT_INTEGRATION','MYSQLPUNK_MSSQL_IMPORT_SERVER','MYSQLPUNK_MSSQL_IMPORT_OUTPUT','MYSQLPUNK_MSSQL_IMPORT_AUTH','MYSQLPUNK_MSSQL_IMPORT_PROFILE','MYSQLPUNK_MSSQL_IMPORT_FIXTURES','MYSQLPUNK_MSSQL_IMPORT_MEMORY_SCALE','MYSQLPUNK_MSSQL_IMPORT_RUN_ID','MYSQLPUNK_SMOKE_FILTER') +$saved=@{};foreach($name in $names){$saved[$name]=[Environment]::GetEnvironmentVariable($name)} +try { + $env:MYSQLPUNK_MSSQL_IMPORT_INTEGRATION='1' + $env:MYSQLPUNK_MSSQL_IMPORT_MEMORY_SCALE=$null + $env:MYSQLPUNK_MSSQL_IMPORT_SERVER=$approval.Server + $env:MYSQLPUNK_MSSQL_IMPORT_OUTPUT=$approval.Output + $env:MYSQLPUNK_MSSQL_IMPORT_AUTH=$authorization + $env:MYSQLPUNK_MSSQL_IMPORT_FIXTURES=Join-Path $PSScriptRoot 'MssqlExportFixtures' + $env:MYSQLPUNK_SMOKE_FILTER='MSSQL import integration' + $expectations=Get-Content -LiteralPath (Join-Path $PSScriptRoot 'MssqlImportFixtures/expected-cases.json') -Raw | ConvertFrom-Json + $profiles=if($AcceptanceProfile -eq 'All'){@('Core','Lob','Geometry','Faults','Memory')}else{@($AcceptanceProfile)} + $built=[bool]$SkipBuild + foreach($one in $profiles) { + $env:MYSQLPUNK_MSSQL_IMPORT_PROFILE=$one + $env:MYSQLPUNK_MSSQL_IMPORT_RUN_ID=[Guid]::NewGuid().ToString('N') + $log=Join-Path $approval.Output ('run-'+$one+'.log') + $ErrorActionPreference='Continue' + try { + if(!$built){& (Join-Path $PSScriptRoot 'Run-SmokeTests.ps1') -Configuration Release *> $log;$built=$true} + else{& (Join-Path $PSScriptRoot '../mySQLPunk/bin/CodexVerify/mySQLPunk.SmokeTests.exe') *> $log} + $code=$LASTEXITCODE + }finally{$ErrorActionPreference='Stop'} + Get-Content -LiteralPath $log -Tail 10 + if($code -ne 0){throw "Native import profile $one failed (exit $code); inspect $log."} + $receiptPath=Join-Path $approval.Output ($one+'/profile-receipt.json') + if(!(Test-Path -LiteralPath $receiptPath -PathType Leaf)){throw 'No actual profile receipt; SKIP or unmatched tests cannot count as acceptance.'} + $receipt=Get-Content -LiteralPath $receiptPath -Raw | ConvertFrom-Json + Assert-ImportProfileReceipt -Receipt $receipt -AcceptanceProfile $one -RunId $env:MYSQLPUNK_MSSQL_IMPORT_RUN_ID -ExpectedCases @($expectations.$one) + $exe=Join-Path $PSScriptRoot '../mySQLPunk/bin/CodexVerify/mySQLPunk.SmokeTests.exe' + if($receipt.executableSha256 -ine (Get-FileHash -LiteralPath $exe -Algorithm SHA256).Hash){throw 'Profile executable receipt mismatch.'} + foreach($case in $receipt.cases) { + $casePath=Join-Path $approval.Output ($one+'/'+$case.caseId+'.json') + if((Get-FileHash -LiteralPath $casePath -Algorithm SHA256).Hash -ine $case.receiptSha256){throw 'Case receipt hash mismatch.'} + } + } +}finally{foreach($name in $names){[Environment]::SetEnvironmentVariable($name,$saved[$name])}} \ No newline at end of file diff --git a/tests/Run-MssqlImportTransferTrial.ps1 b/tests/Run-MssqlImportTransferTrial.ps1 new file mode 100644 index 0000000..d1bf513 --- /dev/null +++ b/tests/Run-MssqlImportTransferTrial.ps1 @@ -0,0 +1,107 @@ +# One authorized SQL2019 Transfer trial; no automatic build, cleanup or retry. +param([switch]$Run,[string]$Server,[string]$Database,[string]$InputFile,[string]$OutputDirectory) +$ErrorActionPreference='Stop' +if(!$Run){Write-Host 'SKIPPED: explicit -Run -Server localhost -Database Transfer -InputFile C:\Users\johnho\Desktop\Transfer_dump_20261013_165730.sql -OutputDirectory required.';exit 0} +$repoRoot='C:\Users\johnho\Documents\mySQLPunk-worktrees\mssql-import' +$testExe=Join-Path $repoRoot 'mySQLPunk\bin\CodexVerify\mySQLPunk.SmokeTests.exe' +$productExe=Join-Path $repoRoot 'mySQLPunk\bin\CodexVerify\mySQLPunk.exe' +if(!('Newtonsoft.Json.Linq.JObject' -as [type])){[Reflection.Assembly]::LoadFrom((Join-Path (Split-Path $testExe) 'Newtonsoft.Json.dll'))|Out-Null} +. (Join-Path $PSScriptRoot 'MssqlImportTransfer.Guards.ps1') +function Read-TrialBoundedBytes([string]$Path){ + $stream=[IO.File]::OpenRead($Path) + try{ + if($stream.Length -le 0 -or $stream.Length -gt 1048576L){throw 'Bounded artifact size rejected.'} + $bytes=[byte[]]::new([int]$stream.Length);$read=0 + while($read -lt $bytes.Length){$n=$stream.Read($bytes,$read,$bytes.Length-$read);if($n -eq 0){throw 'Artifact truncated.'};$read+=$n} + if($stream.ReadByte() -ne -1){throw 'Artifact changed.'} + return ,$bytes + }finally{$stream.Dispose()} +} +function Hash-TrialBytes([byte[]]$Bytes){$hash=[Security.Cryptography.SHA256]::Create();try{return [BitConverter]::ToString($hash.ComputeHash($Bytes)).Replace('-','').ToLowerInvariant()}finally{$hash.Dispose()}} +function Assert-TrialPath([string]$Path){ + $full=[IO.Path]::GetFullPath($Path) + if(!$full.StartsWith('C:\',[StringComparison]::OrdinalIgnoreCase) -or ([IO.DriveInfo]::new('C')).DriveType -ne [IO.DriveType]::Fixed){throw 'Local fixed C-only output required.'} + $current=$full + while($current){if((Test-Path -LiteralPath $current) -and ((Get-Item -LiteralPath $current -Force).Attributes -band [IO.FileAttributes]::ReparsePoint)){throw 'Reparse chain rejected.'};$current=[IO.Path]::GetDirectoryName($current)} + return $full +} +$expectedInput='C:\Users\johnho\Desktop\Transfer_dump_20261013_165730.sql' +$fixedAttempt='C:\Users\johnho\Documents\mySQLPunk-worktrees\mssql-import\.superpowers\sdd\implementation-plan\ImportTrials\JOHN_MSSQL2019-Transfer-Transfer_dump_20261013_165730.attempt.json' +if($Server -cne 'localhost' -or $Database -cne 'Transfer' -or [string]::IsNullOrWhiteSpace($InputFile) -or [string]::IsNullOrWhiteSpace($OutputDirectory)){throw 'Exact explicit trial arguments required.'} +# This pin cannot be created by the trial. Formal release supplies accepted source/registration manifests after build. +$pinPath=Join-Path $repoRoot '.superpowers/sdd/implementation-plan/task-7-build-pin.json' +$pin=ConvertFrom-TransferBoundedJson (Read-TrialBoundedBytes (Assert-TrialPath $pinPath)) +if(!(Test-TransferTrue $pin.FinalPinAccepted) -or !(Test-TransferTrue $pin.TransferTrialRegistered)){throw 'Final source/build/registration pin not released.'} +if(!(Test-Path -LiteralPath $testExe -PathType Leaf) -or (Get-FileHash -LiteralPath $testExe -Algorithm SHA256).Hash.ToLowerInvariant() -cne $pin.ExecutableSha256){throw 'Accepted executable missing or changed.'} +if($pin.ProductExecutablePath -cne $productExe -or !(Test-Path -LiteralPath $productExe -PathType Leaf) -or (Get-FileHash -LiteralPath (Assert-TrialPath $productExe) -Algorithm SHA256).Hash.ToLowerInvariant() -cne $pin.ProductExecutableSha256){throw 'Accepted actual product executable missing or changed.'} +foreach($manifestName in @('SourceSnapshot','Registration')){ + $manifestPath=Assert-TrialPath $pin[($manifestName+'Path')] + $bytes=Read-TrialBoundedBytes $manifestPath + if((Hash-TrialBytes $bytes) -cne $pin[($manifestName+'Sha256')]){throw 'Pinned source/registration manifest changed.'} + $manifest=ConvertFrom-TransferBoundedJson $bytes + foreach($name in $manifest.Keys){$null=Assert-TrialPath $name;if((Get-FileHash -LiteralPath $name -Algorithm SHA256).Hash.ToLowerInvariant() -cne $manifest[$name]){throw 'Pinned source/registration file changed.'}} +} +$registryText=[IO.File]::ReadAllText((Join-Path $repoRoot 'tests/SmokeTests.cs')) +$compileText=[IO.File]::ReadAllText((Join-Path $repoRoot 'tests/Run-SmokeTests.ps1')) +$registryContainsTrial=([regex]::Matches($registryText,'Run\("MSSQL import Transfer trial",\s*MssqlImportTransferTrial,\s*ref passed\);').Count -eq 1 -and [regex]::Matches($compileText,'SmokeTests\.MssqlImport\.TransferTrial\.cs').Count -eq 1) +$decision=Test-TransferTrialBuildSnapshot $pin ((Get-FileHash -LiteralPath $testExe -Algorithm SHA256).Hash.ToLowerInvariant()) ((Get-FileHash -LiteralPath $productExe -Algorithm SHA256).Hash.ToLowerInvariant()) $pin.SourceSnapshotSha256 $pin.RegistrationSha256 $registryContainsTrial +if(!$decision.Allowed){throw $decision.Code} +$inputFull=Assert-TrialPath $InputFile +if(![string]::Equals($inputFull,$expectedInput,[StringComparison]::OrdinalIgnoreCase)){throw 'Unauthorized source path.'} +if(!(Test-Path -LiteralPath $inputFull -PathType Leaf) -or (Get-Item -LiteralPath $inputFull).Length -ne 4478414151L){throw 'Source metadata mismatch.'} +$outputFull=Assert-TrialPath $OutputDirectory +if([string]::Equals($outputFull,$inputFull,[StringComparison]::OrdinalIgnoreCase) -or [string]::Equals($outputFull.TrimEnd('\'),'C:\Users\johnho\Desktop',[StringComparison]::OrdinalIgnoreCase)){throw 'Output must not touch input.'} +if(Test-Path -LiteralPath $outputFull){if(!(Test-Path -LiteralPath $outputFull -PathType Container) -or @(Get-ChildItem -LiteralPath $outputFull -Force).Count -ne 0){throw 'Fresh empty output required.'}} +$null=Assert-TrialPath $fixedAttempt +if(Test-Path -LiteralPath $fixedAttempt){throw 'This authorization attempt was already claimed. Preserve existing evidence; no automatic retry.'} +$requiredFreeBytes=68719476736L +if(([IO.DriveInfo]::new('C')).AvailableFreeSpace -lt $requiredFreeBytes){throw 'Insufficient C capacity for trial.'} +$foreign=@('MYSQLPUNK_MSSQL_IMPORT_INTEGRATION','MYSQLPUNK_MSSQL_IMPORT_AUTH','MYSQLPUNK_MSSQL_IMPORT_PROFILE','MYSQLPUNK_MSSQL_IMPORT_FIXTURES','MYSQLPUNK_MSSQL_IMPORT_MEMORY_SCALE','MYSQLPUNK_MSSQL_IMPORT_RUN_ID','MYSQLPUNK_MSSQL_IMPORT_WIZARD_PREVIEW','MYSQLPUNK_MSSQL_WIZARD_PREVIEW') +$variables=@('MYSQLPUNK_MSSQL_IMPORT_TRIAL','MYSQLPUNK_MSSQL_IMPORT_SERVER','MYSQLPUNK_MSSQL_IMPORT_DATABASE','MYSQLPUNK_MSSQL_IMPORT_INPUT','MYSQLPUNK_MSSQL_IMPORT_OUTPUT','MYSQLPUNK_SMOKE_FILTER')+$foreign +$previous=@{};foreach($name in $variables){$previous[$name]=[Environment]::GetEnvironmentVariable($name,'Process')} +$nonce=[guid]::NewGuid().ToString('N');$job=[guid]::NewGuid().ToString('D');$child=$null +try{ + [IO.Directory]::CreateDirectory($outputFull)|Out-Null + $request=@{SchemaVersion=2L;Kind='Request';Nonce=$nonce;JobId=$job;Server='localhost';DatabaseName='Transfer';InputCanonicalPath=$inputFull;OutputCanonicalPath=$outputFull;RequiredFreeBytes=68719476736L;CancelBelowBytes=21474836480L;SourceCommit=$pin.SourceCommit;SourceSnapshotSha256=$pin.SourceSnapshotSha256;ExecutableSha256=$pin.ExecutableSha256;ProductExecutableSha256=$pin.ProductExecutableSha256;ProductExecutablePath=$productExe;RegistrationSha256=$pin.RegistrationSha256} + $decision=Test-TransferTrialRequest $request;if(!$decision.Allowed){throw $decision.Code} + $requestPath=Join-Path $outputFull 'request.json' + $requestStream=[IO.FileStream]::new($requestPath,[IO.FileMode]::CreateNew,[IO.FileAccess]::Write,[IO.FileShare]::Read) + try{$requestBytes=[Text.UTF8Encoding]::new($false).GetBytes(($request|ConvertTo-Json -Depth 5));$requestStream.Write($requestBytes,0,$requestBytes.Length);$requestStream.Flush($true)}finally{$requestStream.Dispose()} + foreach($name in $foreign){[Environment]::SetEnvironmentVariable($name,$null,'Process')} + [Environment]::SetEnvironmentVariable('MYSQLPUNK_MSSQL_IMPORT_TRIAL','1','Process') + [Environment]::SetEnvironmentVariable('MYSQLPUNK_MSSQL_IMPORT_SERVER',$Server,'Process') + [Environment]::SetEnvironmentVariable('MYSQLPUNK_MSSQL_IMPORT_DATABASE',$Database,'Process') + [Environment]::SetEnvironmentVariable('MYSQLPUNK_MSSQL_IMPORT_INPUT',$inputFull,'Process') + [Environment]::SetEnvironmentVariable('MYSQLPUNK_MSSQL_IMPORT_OUTPUT',$outputFull,'Process') + [Environment]::SetEnvironmentVariable('MYSQLPUNK_SMOKE_FILTER','MSSQL import Transfer trial','Process') + # No Run-SmokeTests build/retry inside this one-shot runner. Existing registered SUT test exe is the sole child. + $productReadLock=$null;$exeReadLock=$null + try{ + $productReadLock=[IO.File]::Open($productExe,[IO.FileMode]::Open,[IO.FileAccess]::Read,[IO.FileShare]::Read) + $exeReadLock=[IO.File]::Open($testExe,[IO.FileMode]::Open,[IO.FileAccess]::Read,[IO.FileShare]::Read) + if((Get-FileHash -LiteralPath $testExe -Algorithm SHA256).Hash.ToLowerInvariant() -cne $pin.ExecutableSha256){throw 'Executable changed before launch.'} + if((Get-FileHash -LiteralPath $productExe -Algorithm SHA256).Hash.ToLowerInvariant() -cne $pin.ProductExecutableSha256){throw 'Product executable changed before launch.'} + $child=Start-Process -FilePath $testExe -WorkingDirectory $repoRoot -WindowStyle Hidden -PassThru -Wait -RedirectStandardOutput (Join-Path $outputFull 'stdout.log') -RedirectStandardError (Join-Path $outputFull 'stderr.log') + }finally{if($exeReadLock){$exeReadLock.Dispose()};if($productReadLock){$productReadLock.Dispose()}} + $child.Refresh() + $finalPath=Join-Path $outputFull 'final.json' + if(!(Test-Path -LiteralPath $finalPath -PathType Leaf)){throw 'Missing execution receipt; skipped or missing registration is not acceptance. Preserve marker/output.'} + $receipt=ConvertFrom-TransferBoundedJson (Read-TrialBoundedBytes (Assert-TrialPath $finalPath)) + $claimState=Test-TransferTrialMarkerPresence $receipt (Test-Path -LiteralPath $fixedAttempt -PathType Leaf) $child.ExitCode + if(!$claimState.Allowed){throw $claimState.Code} + $markerBytes=Read-TrialBoundedBytes (Assert-TrialPath $fixedAttempt) + $marker=ConvertFrom-TransferBoundedJson $markerBytes + $expected=$request.Clone();$expected.ProcessId=[long]$child.Id;$expected.RequestSha256=Hash-TrialBytes $requestBytes + $identity=[Security.Principal.WindowsIdentity]::GetCurrent() + try{ + $sid=[byte[]]::new($identity.User.BinaryLength);$identity.User.GetBinaryForm($sid,0) + $expected.WindowsSid=[BitConverter]::ToString($sid).Replace('-','').ToLowerInvariant();$expected.WindowsName=$identity.Name + }finally{$identity.Dispose()} + $decision=Test-TransferTrialCompletionReceipt $expected $marker $receipt (Hash-TrialBytes $markerBytes) + if($child.ExitCode -ne 0 -or !$decision.Allowed){throw ('Trial did not pass this invocation: '+$decision.Code+'; preserve marker, DB and output; no retry.')} + Write-Host ('Trial completed with independent footer counts. Local receipt: '+$finalPath) + Write-Host 'Object catalog inventory is separate. Source object-definition and field-hash equality are not proven; human UI/sound and production remain pending.' +}finally{ + foreach($name in $variables){[Environment]::SetEnvironmentVariable($name,$previous[$name],'Process')} + # No marker removal, user DB cleanup, setting changes, retry, process-wide SQL KILL or source deletion. +} diff --git a/tests/Run-SmokeTests.ps1 b/tests/Run-SmokeTests.ps1 index 5eea199..e7cbf59 100644 --- a/tests/Run-SmokeTests.ps1 +++ b/tests/Run-SmokeTests.ps1 @@ -143,7 +143,7 @@ if (!(Test-Path -LiteralPath $mysqlConnector)) { /r:System.Core.dll ` /r:System.IO.Compression.dll ` /r:System.IO.Compression.FileSystem.dll ` - $source $preferencesSource $schemaSyncSource $erModelSource (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Snapshot.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Schema.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.ViewReferences.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Properties.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Artifact.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Values.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Lob.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Worker.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Wizard.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Routing.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Integration.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.SnapshotIntegration.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.CompatibilityIntegration.cs") + (Join-Path $PSScriptRoot "SmokeTests.MssqlImport.TransferTrial.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlImport.Live.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlImport.Wizard.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlImport.Integration.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlImport.Worker.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlImport.Parser.cs") $source $preferencesSource $schemaSyncSource $erModelSource (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Snapshot.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Schema.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.ViewReferences.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Properties.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Artifact.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Values.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Lob.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Worker.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Wizard.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Routing.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.Integration.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.SnapshotIntegration.cs") (Join-Path $PSScriptRoot "SmokeTests.MssqlExport.CompatibilityIntegration.cs") if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE } diff --git a/tests/SmokeTests.MssqlExport.Values.cs b/tests/SmokeTests.MssqlExport.Values.cs index 5cd3c20..74c3f5b 100644 --- a/tests/SmokeTests.MssqlExport.Values.cs +++ b/tests/SmokeTests.MssqlExport.Values.cs @@ -42,6 +42,13 @@ private static void Values_Scalars() AssertEquals("'03:04:05.1234567'", MssqlValueCodec.FormatScalar(new TimeSpan(0,3,4,5).Add(TimeSpan.FromTicks(1234567)), ValueColumn("time")), "Time ticks"); AssertEquals("'2026-10-12'", MssqlValueCodec.FormatScalar(new DateTime(2026,10,12), ValueColumn("date")), "ISO date"); AssertEquals("'2026-10-12T03:04:05.1234567'", MssqlValueCodec.FormatScalar(new DateTime(2026,10,12,3,4,5).AddTicks(1234567), ValueColumn("datetime2")), "Date ticks"); + AssertEquals("'2026-10-12T03:04:05.123'", MssqlValueCodec.FormatScalar(new DateTime(2026,10,12,3,4,5).AddMilliseconds(123), ValueColumn("datetime")), "Legacy datetime literal uses SQL-compatible 3 digits"); + AssertEquals("'2026-10-12T03:04:00.000'", MssqlValueCodec.FormatScalar(new DateTime(2026,10,12,3,4,0), ValueColumn("smalldatetime")), "Legacy smalldatetime literal uses SQL-compatible 3 digits"); + foreach (var legacy in new[]{new DateTime(1753,1,1),new DateTime(2026,10,12,3,4,5).AddMilliseconds(3),new DateTime(2026,10,12,3,4,5).AddMilliseconds(7),new DateTime(9999,12,31,23,59,59).AddMilliseconds(997)}) { + var stored=new SqlDateTime(legacy); + string literal=MssqlValueCodec.FormatScalar(stored.Value,ValueColumn("datetime")).Trim('\''); + Assert(stored.Equals(new SqlDateTime(DateTime.ParseExact(literal,"yyyy-MM-dd'T'HH:mm:ss.fff",CultureInfo.InvariantCulture))),"Legacy datetime stored ticks survive 3-digit serialization"); + } AssertEquals("'00112233-4455-6677-8899-aabbccddeeff'", MssqlValueCodec.FormatScalar(new Guid("00112233-4455-6677-8899-aabbccddeeff"), ValueColumn("uniqueidentifier")), "GUID"); double number = 1.2345678901234567; Assert(double.Parse(MssqlValueCodec.FormatScalar(number, ValueColumn("float")), CultureInfo.InvariantCulture) == number, "Float roundtrip"); diff --git a/tests/SmokeTests.MssqlImport.Integration.cs b/tests/SmokeTests.MssqlImport.Integration.cs new file mode 100644 index 0000000..1cd6074 --- /dev/null +++ b/tests/SmokeTests.MssqlImport.Integration.cs @@ -0,0 +1,319 @@ +using System; +using System.Collections.Generic; +using System.Collections; +using System.Data; +using System.Data.Common; +using System.Data.SqlClient; +using System.IO; +using System.Linq; +using System.Reflection; +using System.Threading; +using System.Threading.Tasks; +using mySQLPunk.lib; +using mySQLPunk.lib.MssqlImport; +using mySQLPunk.lib.MssqlExport; + +public static partial class SmokeTests +{ + private static void MssqlImportConnectionOptions() + { + var settings = new Dictionary { + { "db_kind", "mssql" }, { "host", "fixture-server" }, { "port", "1433" }, + { "initial_database", "fixture-target" }, { "trusted_connection", "T" }, { "tls_mode", "Required" } + }; + var builder = new SqlConnectionStringBuilder(ConnectionConfigurationService.BuildMssqlImportConnectionString(settings)); + Assert(!builder.MultipleActiveResultSets && builder.ConnectRetryCount == 0, + "Import must disable MARS and reconnect before opening its owned connection."); + Assert(builder.InitialCatalog == "fixture-target" && builder.IntegratedSecurity && builder.Encrypt, + "Import keeps explicit target and existing authentication/TLS options."); + var old = new SqlConnectionStringBuilder(ConnectionConfigurationService.BuildSqlServerConnectionString(settings)); + Assert(old.MultipleActiveResultSets && old.ConnectRetryCount == 1, "Existing callers keep their original connection defaults."); + Assert((string)settings["initial_database"] == "fixture-target" && settings.Count == 6, "Builder does not mutate supplied settings."); + settings["initial_database"] = " "; + AssertThrows(() => ConnectionConfigurationService.BuildMssqlImportConnectionString(settings), "Missing target is refused before Open."); + AssertThrows(() => ConnectionOpenService.OpenMssqlImportSession(settings), "Invalid target does not open a provider or SSH tunnel."); + settings["db_kind"] = "mysql"; + AssertThrows(() => ConnectionOpenService.OpenMssqlImportSession(settings), "Importer refuses other engines."); + } + + private static readonly MssqlImportBatch ImportSessionBatch = new MssqlImportBatch(1, 0, 20, 1, 2, "SELECT 'original; GO';\r\n-- preserve\r\n"); + + private static void MssqlImportSessionDrainsResults() + { + var owner = new ImportSessionOwner(); + var reader = new ImportSessionReader(new[] { 200000, 0, 3 }, rejectValues: true); + var command = new ImportSessionCommand(reader); + using (var session = new MssqlImportSqlSession(owner, () => command, () => true)) { + session.ExecuteBatch(ImportSessionBatch, CancellationToken.None); + Assert(command.CommandText == ImportSessionBatch.Text && command.CommandTimeout == 0, "Original batch reaches SQL unchanged with timeout0."); + Assert(command.Behavior == CommandBehavior.SequentialAccess && reader.RowsRead == 200003 && reader.NextCalls == 3, + "Every row and result is drained sequentially without reading/materializing field values."); + Assert(reader.Disposes == 1 && command.Disposes == 1 && command.LastToken == CancellationToken.None, + "Completion includes reader and command disposal."); + } + Assert(owner.Disposes == 1, "Only the owned session is released."); + + foreach (string point in new[] { "start", "read", "next", "dispose" }) { + var failedReader = new ImportSessionReader(new[] { 1 }, rejectValues: true); + var failedCommand = new ImportSessionCommand(failedReader); + var primary = DiagnosticSql(547, 2627); + if (point == "start") failedCommand.StartError = primary; + if (point == "read") failedReader.ReadError = primary; + if (point == "next") failedReader.NextError = primary; + failedReader.DisposeError = point == "dispose" ? primary : DiagnosticSql(50001); + var failedOwner = new ImportSessionOwner(); + using (var session = new MssqlImportSqlSession(failedOwner, () => failedCommand, () => true)) { + try { session.ExecuteBatch(ImportSessionBatch, CancellationToken.None); throw new Exception("Expected SQL failure " + point); } + catch (MssqlImportBatchException e) { + Assert(e.Errors.Count == (point == "start" || point == "dispose" ? 2 : 3), "All SQL details survive " + point + " and reader disposal."); + Assert(e.Errors[0].Number == 547 && e.Errors[1].Number == 2627 && !e.OutcomeUncertain, + "Ordinary provider details preserve number/state/class/line without inventing uncertainty: " + point); + } + Assert(failedCommand.Disposes == 1, "Failed command releases once: " + point); + } + Assert(failedOwner.Disposes == 1, "Failure owns only its session: " + point); + } + var transport = new ImportSessionCommand(null) { StartError = new IOException("fixture transport failure") }; + using (var session = new MssqlImportSqlSession(new ImportSessionOwner(), () => transport, () => true)) { + try { session.ExecuteBatch(ImportSessionBatch, CancellationToken.None); throw new Exception("Expected transport failure"); } + catch (MssqlImportBatchException e) { Assert(e.OutcomeUncertain && e.Errors.Count == 0, "Transport failure is uncertain and cannot look like a skippable SQL error."); } + } + foreach (int code in new[] { -2, 0, 64, 233, 10053, 10054, 10060 }) { + var network = new ImportSessionCommand(null) { StartError = DiagnosticSql(code) }; + using (var session = new MssqlImportSqlSession(new ImportSessionOwner(), () => network, () => true)) { + try { session.ExecuteBatch(ImportSessionBatch, CancellationToken.None); throw new Exception("Expected network/timeout failure"); } + catch (MssqlImportBatchException e) { Assert(e.OutcomeUncertain && e.Errors.Count == 1, "Known timeout/transport codes cannot become ordinary skippable SQL errors: " + code); } + } + } + var unexpectedCancel = new ImportSessionCommand(null) { StartError = new OperationCanceledException() }; + using (var session = new MssqlImportSqlSession(new ImportSessionOwner(), () => unexpectedCancel, () => true)) + AssertThrows(() => session.ExecuteBatch(ImportSessionBatch, CancellationToken.None), "Provider cancellation without a caller request is uncertain, not user cancellation."); + } + + private static void MssqlImportSessionCancellationOwnership() + { + var owner = new ImportSessionOwner(); var tree = new ImportSessionOwner(); + var entered = new ManualResetEventSlim(); var release = new ManualResetEventSlim(); + var command = new ImportSessionCommand(new ImportSessionReader(new[] { 0 }, true)); + command.Start = (behavior, token) => { entered.Set(); release.Wait(); return Task.FromResult(command.Reader); }; + using (var cancel = new CancellationTokenSource()) using (var session = new MssqlImportSqlSession(owner, () => command, () => true, 40)) { + var task = Task.Run(() => { try { session.ExecuteBatch(ImportSessionBatch, cancel.Token); } catch (OperationCanceledException) { } }); + Assert(entered.Wait(2000), "Command startup enters active ownership."); + AssertThrows(() => session.ReadState(CancellationToken.None), "No state query can overlap active reader/startup."); + cancel.Cancel(); + Assert(command.Cancels > 0 && command.LastToken == cancel.Token, "Cancellation reaches the exact startup command and token."); + Assert(owner.Released.Wait(2000), "Watchdog attempts owned close after cancellation deadline."); + var pending = ImportWaitForPublishedReleaseWhileActive(session); + Assert(!pending.CommandEnded && pending.SessionReleased && !pending.TransactionCleanupConfirmed, + "Owned close is not command acknowledgement or rollback proof."); + release.Set(); Assert(task.Wait(2000), "Worker unwinds only after provider completion."); + var final = session.Cleanup(); + Assert(final.CommandEnded && final.SessionReleased && !final.TransactionCleanupConfirmed, "Unwound closed session still cannot prove rollback."); + Parallel.For(0, 20, i => session.Dispose()); + Assert(owner.Disposes == 1 && tree.Disposes == 0, "Cancel/cleanup/Dispose race releases this owner once and never the independent tree owner."); + } + entered.Dispose(); release.Dispose(); + + foreach (string phase in new[] { "read", "next", "read-sql", "dispose" }) { + using (var cancel = new CancellationTokenSource()) { + var readOwner = new ImportSessionOwner(); var reader = new ImportSessionReader(new[] { 1 }, true); + var reading = new ImportSessionCommand(reader); + if (phase == "dispose") reader.BeforeDispose = () => cancel.Cancel(); + else if (phase == "next") reader.BeforeNext = () => cancel.Cancel(); + else reader.BeforeRead = () => cancel.Cancel(); + if (phase == "read-sql") reader.ReadError = DiagnosticSql(547, 2627); + using (var session = new MssqlImportSqlSession(readOwner, () => reading, () => true)) { + try { session.ExecuteBatch(ImportSessionBatch, cancel.Token); throw new Exception("Expected cancellation during " + phase); } + catch (OperationCanceledException) { Assert(phase != "read-sql", "SQL details must not disappear behind cancellation."); } + catch (MssqlImportBatchException error) { Assert(phase == "read-sql" && error.OutcomeUncertain && error.Errors.Count == 2, "Cancellation retains all delivered SQL details before stopping."); } + Assert(reading.Cancels == 1 && reader.Disposes == 1 && reading.Disposes == 1, "Read/NextResult cancellation retains ownership until disposal: " + phase); + } + Assert(readOwner.Disposes == 1, "Read cancellation releases exact owner: " + phase); + } + } + var disposingOwner = new ImportSessionOwner(); + using (var disposeEntered = new ManualResetEventSlim()) using (var disposeRelease = new ManualResetEventSlim()) { + var reader = new ImportSessionReader(new[] { 0 }, true) { BeforeDispose = () => { disposeEntered.Set(); disposeRelease.Wait(); } }; + var cmd = new ImportSessionCommand(reader); + using (var session = new MssqlImportSqlSession(disposingOwner, () => cmd, () => true)) { + var task = Task.Run(() => session.ExecuteBatch(ImportSessionBatch, CancellationToken.None)); + Assert(disposeEntered.Wait(2000) && !task.IsCompleted, "Reader Dispose remains inside batch acknowledgement."); + AssertThrows(() => session.ReadState(CancellationToken.None), "A blocked reader Dispose still owns the command slot."); + Assert(!session.Cleanup().CommandEnded, "Cleanup cannot acknowledge an unfinished reader Dispose."); + disposeRelease.Set(); Assert(task.Wait(2000), "Batch acknowledgement follows actual Dispose completion."); + } + } + + var lateOwner = new ImportSessionOwner(); var commands = new Queue(); + var first = new ImportSessionCommand(new ImportSessionReader(new[] { 0 }, true)); + first.Start = (behavior, token) => { token.ThrowIfCancellationRequested(); return Task.FromResult(first.Reader); }; + var second = new ImportSessionCommand(new ImportSessionReader(new[] { 0 }, true)); + var secondEntered = new ManualResetEventSlim(); var secondRelease = new ManualResetEventSlim(); + second.Start = (behavior, token) => { secondEntered.Set(); secondRelease.Wait(); return Task.FromResult(second.Reader); }; + commands.Enqueue(first); commands.Enqueue(second); + using (var session = new MssqlImportSqlSession(lateOwner, () => commands.Dequeue(), () => true, 50)) { + first.Reader.BeforeRead = () => session.CancelActiveCommand(); + session.ExecuteBatch(ImportSessionBatch, CancellationToken.None); + var later = Task.Run(() => session.ExecuteBatch(ImportSessionBatch, CancellationToken.None)); + Assert(secondEntered.Wait(2000) && !lateOwner.Released.Wait(100), "A completed command's cancellation timer cannot close an active later generation."); + secondRelease.Set(); Assert(later.Wait(2000), "Later generation completes normally."); + } + secondEntered.Dispose(); secondRelease.Dispose(); + Assert(lateOwner.Disposes == 1, "Normal disposal releases the surviving owner."); + var invalidOwner = new ImportSessionOwner(); + AssertThrows(() => new MssqlImportSqlSession(invalidOwner, null, () => true), "Initialization failure releases transferred lifetime ownership."); + Assert(invalidOwner.Disposes == 1, "Failed initialization releases exactly once."); + + var closedConnection = new SqlConnection(); + var provider = new FakeExecDatabase("mssql", "0"); + using (var real = new MssqlImportSqlSession(new MssqlOwnedSession(provider, closedConnection))) { + AssertThrows(() => real.ExecuteBatch(ImportSessionBatch, CancellationToken.None), "Real closed SQL connection startup fails, without opening a database."); + } + closedConnection.Dispose(); + } + + private static ImportSessionReader ImportValues(params object[] row) { return new ImportSessionReader(new[] { new[] { row } }); } + private static object[] ImportIdentity(int user = 5, object viewDefinition = null, string version = "15.0.2000.5") + { return new object[] { "fixture-server", "fixture", 7, new DateTime(2026, 10, 13), version, "READ_WRITE", user, viewDefinition ?? 1, 0, 0 }; } + + private static void MssqlImportSessionMetadataProof() + { + foreach (string mode in new[] { "dbo", "no-deny", "deny", "low", "null", "2022-no-security", "2022-security", "denied-scan", "count-failure" }) { + var owner = new ImportSessionOwner(); var issued = new List(); + var identity = ImportIdentity(mode == "dbo" ? 1 : 5, mode == "low" ? (object)0 : mode == "null" ? DBNull.Value : 1, mode.StartsWith("2022") ? "16.0.1" : "15.0.1"); + var queue = new Queue(); queue.Enqueue(ImportValues(identity)); + if (mode.StartsWith("2022")) queue.Enqueue(ImportValues(mode == "2022-security" ? 1 : 0)); + if (mode != "dbo" && mode != "low" && mode != "null" && mode != "2022-no-security") queue.Enqueue(ImportValues(mode == "deny" ? 1L : 0L)); + bool complete = mode == "dbo" || mode == "no-deny" || mode == "2022-security"; + if (complete || mode == "count-failure") queue.Enqueue(ImportValues(0L)); + using (var session = new MssqlImportSqlSession(owner, () => { + var cmd = new ImportSessionCommand(queue.Dequeue()); issued.Add(cmd); + if (mode == "denied-scan" && issued.Count == 2 || mode == "count-failure" && issued.Count == 3) cmd.StartError = DiagnosticSql(229); + return cmd; + }, () => true)) { + var target = session.ReadTarget(CancellationToken.None); + Assert(target.DatabaseId == 7 && target.DatabaseName == "fixture" && target.Writable && !target.CanCreateTable.Value, + "Actual identity/writeability/create capability are independent: " + mode); + Assert(target.MetadataVisible == complete && target.UserObjectCount == (complete ? (long?)0 : null), + "Only proven complete metadata yields empty0; unknown remains null: " + mode); + Assert(issued.All(c => c.CommandTimeout == 5 && c.Disposes == 1), "Read-only metadata probes use5sec and dispose before another: " + mode); + var denyQuery = issued.Select(c => c.CommandText).FirstOrDefault(sql => sql.Contains("sys.database_permissions")); + if (denyQuery != null) Assert(denyQuery.Contains("sys.user_token") && !denyQuery.Contains("sys.objects"), "Effective-token DENY scan cannot hide rows via object-catalog join."); + } + } + foreach (int column in new[] { 0, 1, 2, 3, 4 }) { + var identity = ImportIdentity(); identity[column] = DBNull.Value; + using (var session = new MssqlImportSqlSession(new ImportSessionOwner(), () => new ImportSessionCommand(ImportValues(identity)), () => true)) + AssertThrows(() => session.ReadTarget(CancellationToken.None), "Missing target identity is a failed probe, never synthesized."); + } + } + + private static void MssqlImportSessionProbesAndCleanup() + { + var owner = new ImportSessionOwner(); var commands = new List(); + var readers = new Queue(new[] { ImportValues(7, 0, 0), ImportValues(true, 1L, 9L, 9L, true, false), ImportValues(0, 0) }); + using (var session = new MssqlImportSqlSession(owner, () => { var cmd = new ImportSessionCommand(readers.Dequeue()); commands.Add(cmd); return cmd; }, () => true)) { + var state = session.ReadState(CancellationToken.None); Assert(state.IsOpen && state.DatabaseId == 7 && state.TransactionCount == 0 && state.XactState == 0, "State keeps DB and transaction facts."); + Assert(commands[0].CommandText.StartsWith("DECLARE @databaseId int=DB_ID(); SELECT @databaseId,@@TRANCOUNT,XACT_STATE();", StringComparison.Ordinal), + "Capture metadata DB_ID in its own statement so its SQL17 internal transaction cannot pollute XACT_STATE in the state SELECT."); + var job = new Guid("10000000-0000-0000-0000-000000000001"); var lob = session.ReadLobState(job, CancellationToken.None); + Assert(lob.Exists && lob.RowCount == 1 && lob.Sequence == 9 && lob.ConsumedThrough == 9 && lob.Consumed && !lob.Failed, "LOB aggregate preserves unique row and protocol counters."); + Assert(commands[1].CommandText.Contains("#mp_" + job.ToString("N") + "_s") && commands[1].CommandTimeout == 5, "LOB lookup uses only GUID-derived identifier and bounded timeout."); + var result = session.Cleanup(); + Assert(result.CommandEnded && result.SessionReleased && result.TransactionCleanupConfirmed && result.IssueCode == null, "Only returned zero transaction state and released owner confirm cleanup."); + Assert(commands[2].CommandTimeout == 10 && commands[2].LastToken == CancellationToken.None && commands[2].CommandText.Contains("ROLLBACK"), "Cleanup is independent10sec with uncancelled token."); + Assert(object.ReferenceEquals(result, session.Cleanup()) && commands.Count == 3 && owner.Disposes == 1, "Repeated cleanup returns actual cached receipt without another query or release."); + } + foreach (string failure in new[] { "closed", "rollback", "release" }) { + var failedOwner = new ImportSessionOwner { Error = failure == "release" ? new IOException("fixture release") : null }; + var cmd = new ImportSessionCommand(ImportValues(0, 0)) { StartError = failure == "rollback" ? DiagnosticSql(-2) : null }; + using (var session = new MssqlImportSqlSession(failedOwner, () => cmd, () => failure != "closed")) { + var result = session.Cleanup(); + Assert(result.CommandEnded && result.SessionReleased == (failure != "release") && result.TransactionCleanupConfirmed == (failure == "release") && result.IssueCode != null, + "Cleanup distinguishes actual rollback and resource-release evidence: " + failure); + } + } + + var countCommands = new List(); + var rows = new Queue(new[] { ImportValues(ImportIdentity(1)), ImportValues(2L), ImportValues(10, "schema]x", "table]x"), ImportValues(3000000000L), ImportValues(11, "dbo", "t2"), ImportValues(4L), new ImportSessionReader(new object[][][] { new object[0][] }) }); + using (var session = new MssqlImportSqlSession(new ImportSessionOwner(), () => { var cmd = new ImportSessionCommand(rows.Dequeue()); countCommands.Add(cmd); return cmd; }, () => true)) { + long progressTables = 0, progressRows = 0; + var counts = session.CountTargetRows((tables, total) => { progressTables = tables; progressRows = total; }, CancellationToken.None); + Assert(counts.Tables == 2 && counts.Rows == 3000000004L && progressTables == 2 && progressRows == counts.Rows, "COUNT_BIG totals and callbacks retain Int64 actual results."); + Assert(countCommands.Any(c => c.CommandText.Contains("[schema]]x].[table]]x]")), "Every catalog identifier is bracket quoted."); + Assert(countCommands.Where(c => c.CommandText.Contains("COUNT_BIG(*) FROM [")).All(c => c.CommandTimeout == 0), "Long row counts remain cancellable without a default timeout."); + } + } + + private sealed class ImportSessionOwner : IDisposable + { + internal int Disposes; internal Exception Error; internal Action BeforeReturn; internal ManualResetEventSlim Released = new ManualResetEventSlim(); + public void Dispose() { Interlocked.Increment(ref Disposes); Released.Set(); if (BeforeReturn != null) BeforeReturn(); if (Error != null) throw Error; } + } + private sealed class ImportSessionCommand : DbCommand + { + internal ImportSessionReader Reader; internal Exception StartError; internal Func> Start; + internal CommandBehavior Behavior; internal CancellationToken LastToken; internal int Cancels, Disposes; + internal ImportSessionCommand(ImportSessionReader reader) { Reader = reader; } + public override string CommandText { get; set; } + public override int CommandTimeout { get; set; } + public override CommandType CommandType { get; set; } + public override UpdateRowSource UpdatedRowSource { get; set; } + public override bool DesignTimeVisible { get; set; } + protected override DbConnection DbConnection { get; set; } + protected override DbTransaction DbTransaction { get; set; } + protected override DbParameterCollection DbParameterCollection { get { throw new NotSupportedException(); } } + protected override DbParameter CreateDbParameter() { throw new NotSupportedException(); } + public override void Cancel() { Interlocked.Increment(ref Cancels); } + public override void Prepare() { throw new NotSupportedException(); } + public override int ExecuteNonQuery() { throw new Exception("ExecuteNonQuery would skip result-drain acknowledgement"); } + public override object ExecuteScalar() { throw new NotSupportedException(); } + protected override DbDataReader ExecuteDbDataReader(CommandBehavior behavior) { throw new Exception("Startup must use async cancellation token"); } + protected override Task ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken token) + { Behavior = behavior; LastToken = token; if (StartError != null) throw StartError; return Start != null ? Start(behavior, token) : Task.FromResult(Reader); } + protected override void Dispose(bool disposing) { if (disposing) Disposes++; base.Dispose(disposing); } + } + private sealed class ImportSessionReader : DbDataReader + { + private readonly int[] rowCounts; private readonly object[][][] values; private readonly bool rejectValues; + private int result, row = -1; + internal long RowsRead; internal int NextCalls, Disposes; internal Exception ReadError, NextError, DisposeError; internal Action BeforeRead, BeforeNext, BeforeDispose; + internal ImportSessionReader(int[] rowCounts, bool rejectValues) { this.rowCounts = rowCounts; this.rejectValues = rejectValues; } + internal ImportSessionReader(object[][][] values) { this.values = values; rowCounts = values.Select(x => x.Length).ToArray(); } + public override bool Read() { if (BeforeRead != null) { var action = BeforeRead; BeforeRead = null; action(); } if (ReadError != null) throw ReadError; bool found = ++row < rowCounts[result]; if (found) RowsRead++; return found; } + public override bool NextResult() { NextCalls++; if (BeforeNext != null) { var action = BeforeNext; BeforeNext = null; action(); } if (NextError != null) throw NextError; if (result + 1 >= rowCounts.Length) return false; result++; row = -1; return true; } + public override Task ReadAsync(CancellationToken token) { token.ThrowIfCancellationRequested(); return Task.FromResult(Read()); } + public override Task NextResultAsync(CancellationToken token) { token.ThrowIfCancellationRequested(); return Task.FromResult(NextResult()); } + public override int FieldCount { get { return values == null || values[result].Length == 0 ? 0 : values[result][0].Length; } } + public override object GetValue(int ordinal) { if (rejectValues) throw new Exception("Drain must not materialize LOB or field values"); return values[result][row][ordinal]; } + public override bool IsDBNull(int ordinal) { return GetValue(ordinal) == DBNull.Value; } + public override string GetString(int ordinal) { return (string)GetValue(ordinal); } + public override int GetInt32(int ordinal) { return Convert.ToInt32(GetValue(ordinal)); } + public override long GetInt64(int ordinal) { return Convert.ToInt64(GetValue(ordinal)); } + public override bool GetBoolean(int ordinal) { return Convert.ToBoolean(GetValue(ordinal)); } + public override DateTime GetDateTime(int ordinal) { return (DateTime)GetValue(ordinal); } + public override object this[int ordinal] { get { return GetValue(ordinal); } } + public override object this[string name] { get { throw new NotSupportedException(); } } + public override int Depth { get { return 0; } } + public override bool HasRows { get { return rowCounts[result] > 0; } } + public override bool IsClosed { get { return Disposes > 0; } } + public override int RecordsAffected { get { throw new Exception("Affected rows are not imported rows"); } } + public override void Close() { } + protected override void Dispose(bool disposing) { if (disposing) { if (BeforeDispose != null) BeforeDispose(); Disposes++; if (DisposeError != null) throw DisposeError; } } + public override string GetName(int ordinal) { return ordinal.ToString(); } + public override int GetOrdinal(string name) { throw new NotSupportedException(); } + public override string GetDataTypeName(int ordinal) { throw new NotSupportedException(); } + public override Type GetFieldType(int ordinal) { throw new NotSupportedException(); } + public override int GetValues(object[] buffer) { throw new NotSupportedException(); } + public override byte GetByte(int ordinal) { throw new NotSupportedException(); } + public override long GetBytes(int ordinal, long offset, byte[] buffer, int bufferOffset, int length) { throw new Exception("LOB getter forbidden in drain"); } + public override char GetChar(int ordinal) { throw new NotSupportedException(); } + public override long GetChars(int ordinal, long offset, char[] buffer, int bufferOffset, int length) { throw new Exception("LOB getter forbidden in drain"); } + public override Guid GetGuid(int ordinal) { throw new NotSupportedException(); } + public override short GetInt16(int ordinal) { throw new NotSupportedException(); } + public override float GetFloat(int ordinal) { throw new NotSupportedException(); } + public override double GetDouble(int ordinal) { throw new NotSupportedException(); } + public override decimal GetDecimal(int ordinal) { throw new NotSupportedException(); } + public override IEnumerator GetEnumerator() { throw new NotSupportedException(); } + } +} diff --git a/tests/SmokeTests.MssqlImport.Live.cs b/tests/SmokeTests.MssqlImport.Live.cs new file mode 100644 index 0000000..6c40f3d --- /dev/null +++ b/tests/SmokeTests.MssqlImport.Live.cs @@ -0,0 +1,1624 @@ +using System; +using System.Collections.Generic; +using System.Data; +using System.Data.Common; +using System.Collections; +using System.Security.Cryptography; +using System.Threading.Tasks; +using Newtonsoft.Json; +using mySQLPunk.lib; +using mySQLPunk.lib.MssqlExport; +using System.Data.SqlClient; +using System.Diagnostics; +using System.IO; +using System.Linq; +using System.Text; +using System.Threading; +using Newtonsoft.Json.Linq; +using mySQLPunk.lib.MssqlImport; + +public static partial class SmokeTests +{ + private const string ImportFixtureServer = @"(localdb)\mySQLPunkExportTest_20261012"; + private const string ImportFixtureRoot = @"C:\Users\johnho\AppData\Local\Temp\mysqlpunk-export-20261012-7cadb673332543c3b17c61924490a702"; + private static void MssqlImportIntegrationGuards() + { + var manifest = new JObject { ["Instance"]="mySQLPunkExportTest_20261012", ["EngineVersion"]="17.0", ["CreatedByTask"]=true, + ["UserAuthorization"]="2026-10-12 dedicated LocalDB creation and startup approved", ["FixtureRoot"]=ImportFixtureRoot }; + string output=Path.Combine(ImportFixtureRoot,"guard-only"); + ImportValidateEnvelope(manifest,ImportFixtureServer,output,20L*1024*1024*1024,path => new ImportGuardPath()); + ImportCleanupGuardCases(); + foreach(string fault in new[]{"server","instance","approval","root","sibling","root-itself","relative","non-c","nonempty","reparse","space"}) { + var modified=(JObject)manifest.DeepClone();string server=ImportFixtureServer,path=output;long free=20L*1024*1024*1024; + Func probe=x=>new ImportGuardPath(); + switch(fault) { + case "server":server="localhost";break;case "instance":modified["Instance"]="MSSQLLocalDB";break; + case "approval":modified["UserAuthorization"]="";break;case "root":modified["FixtureRoot"]=@"C:\other";break; + case "sibling":path=ImportFixtureRoot+"-sibling\\x";break;case "root-itself":path=ImportFixtureRoot;break; + case "relative":path="C:relative";break;case "non-c":path=@"D:\fixture";break;case "space":free--;break; + case "nonempty":probe=x=>new ImportGuardPath {Exists=true,IsDirectory=true,HasEntries=true};break; + case "reparse":probe=x=>new ImportGuardPath {Exists=true,IsDirectory=true,Reparse=true};break; + } + AssertThrows(()=>ImportValidateEnvelope(modified,server,path,free,probe),"Guard rejects before any SQL/file mutation: "+fault); + } + } + private sealed class ImportGuardPath { internal bool Exists,IsDirectory,HasEntries,Reparse; } + private static string ImportValidateEnvelope(JObject manifest,string server,string output,long free,Func probe) + { + if(manifest==null || server!=ImportFixtureServer || (string)manifest["Instance"]!="mySQLPunkExportTest_20261012" || + manifest["CreatedByTask"]==null || manifest["CreatedByTask"].Type!=JTokenType.Boolean || !(bool)manifest["CreatedByTask"] || + (string)manifest["EngineVersion"]!="17.0" || (string)manifest["UserAuthorization"]!="2026-10-12 dedicated LocalDB creation and startup approved") + throw new InvalidOperationException("FixtureAuthorizationMismatch"); + string declared=(string)manifest["FixtureRoot"]; + if(!ImportAbsoluteC(declared) || !string.Equals(Path.GetFullPath(declared).TrimEnd('\\'),ImportFixtureRoot,StringComparison.OrdinalIgnoreCase)) + throw new InvalidOperationException("FixtureRootMismatch"); + if(!ImportAbsoluteC(output)) throw new InvalidOperationException("FixtureOutputMustBeAbsoluteC"); + output=Path.GetFullPath(output).TrimEnd('\\'); + if(!output.StartsWith(ImportFixtureRoot+"\\",StringComparison.OrdinalIgnoreCase)) throw new InvalidOperationException("FixtureOutputNotDescendant"); + var leaf=probe(output); + if(leaf.Exists && (!leaf.IsDirectory || leaf.HasEntries)) throw new InvalidOperationException("FixtureOutputNotEmpty"); + ImportNoReparse(output,probe); + if(free<20L*1024*1024*1024) throw new InvalidOperationException("FixtureInsufficientSpace"); + return output; + } + private static bool ImportAbsoluteC(string path) + { return !string.IsNullOrEmpty(path) && path.Length>=3 && (path[0]=='C'||path[0]=='c') && path[1]==':' && (path[2]=='\\'||path[2]=='/'); } + private static void ImportNoReparse(string path,Func probe) + { + for(string ancestor=path;ancestor!=null;) { + var value=probe(ancestor); + if(value.Exists&&value.Reparse) throw new InvalidOperationException("FixtureReparsePath"); + var parent=Directory.GetParent(ancestor);ancestor=parent==null?null:parent.FullName; + } + } + private static ImportGuardPath ImportReadGuardPath(string path) + { + bool dir=Directory.Exists(path),file=File.Exists(path); + if(!dir&&!file)return new ImportGuardPath(); + return new ImportGuardPath {Exists=true,IsDirectory=dir,HasEntries=dir&&Directory.EnumerateFileSystemEntries(path).Any(),Reparse=(File.GetAttributes(path)&FileAttributes.ReparsePoint)!=0}; + } + private static void ImportAssertCleanup(string database,string[] created,string[] generated,string root,string memoryOwner,string fileOwner,string dbOwner,string[] files,Func probe) + { + if(!generated.Contains(database)||!created.Contains(database)||string.IsNullOrEmpty(memoryOwner)||fileOwner!=memoryOwner||dbOwner!=memoryOwner) + throw new InvalidOperationException("FixtureCleanupOwnershipMismatch"); + var expected=new[]{Path.Combine(root,database+".mdf"),Path.Combine(root,database+".ldf")}; + if(files==null||files.Length!=2||files.Distinct(StringComparer.OrdinalIgnoreCase).Count()!=2) throw new InvalidOperationException("FixtureCleanupFileSetMismatch"); + foreach(string file in files) { + if(!ImportAbsoluteC(file))throw new InvalidOperationException("FixtureCleanupFileNotC"); + string full=Path.GetFullPath(file); + if(!full.StartsWith(root+"\\",StringComparison.OrdinalIgnoreCase)||!expected.Contains(full,StringComparer.OrdinalIgnoreCase))throw new InvalidOperationException("FixtureCleanupFileNotOwned"); + ImportNoReparse(full,probe); + } + } + private static void ImportCleanupGuardCases() + { + string root=Path.Combine(ImportFixtureRoot,"pure-cleanup"); + string name="mysqlpunk_import_"+Guid.NewGuid().ToString("N")+"_src";string[] generated={name,name.Substring(0,name.Length-4)+"_dst"}; + string[] files={Path.Combine(root,name+".mdf"),Path.Combine(root,name+".ldf")};Func probe=x=>new ImportGuardPath(); + ImportAssertCleanup(name,new[]{name},generated,root,"invented-owner","invented-owner","invented-owner",files,probe); + foreach(string fault in new[]{"db","unrecorded","file-token","db-token","missing-file","sibling-file","reparse"}) { + string db=name,fileToken="invented-owner",dbToken="invented-owner";string[] created={name},actual=files;var observed=probe; + switch(fault){case "db":db="Transfer";break;case "unrecorded":created=new string[0];break;case "file-token":fileToken="other";break;case "db-token":dbToken=null;break; + case "missing-file":actual=new[]{files[0]};break;case "sibling-file":actual=new[]{files[0],root+"-other\\x.ldf"};break;case "reparse":observed=x=>new ImportGuardPath{Exists=true,Reparse=true};break;} + AssertThrows(()=>ImportAssertCleanup(db,created,generated,root,"invented-owner",fileToken,dbToken,actual,observed),"Cleanup guard rejects: "+fault); + } + } + // Every instance is test-owned, a single target phase and one-shot checkpoint. + // No saved configuration, global provider hook, command text substitution or replay. + private sealed class ImportLiveCheckpoint : IDisposable + { + internal readonly string TargetPhase; + internal readonly ManualResetEventSlim Issued = new ManualResetEventSlim(); + internal readonly ManualResetEventSlim ReaderReady = new ManualResetEventSlim(); + internal readonly ManualResetEventSlim DisposeEntered = new ManualResetEventSlim(); + internal readonly ManualResetEventSlim DisposeRelease = new ManualResetEventSlim(true); + internal Action OnRead, OnNext; + internal int FiniteTimeoutSeconds, CancelCalls, ReadCalls, NextCalls, ValueGetterCalls; internal int ReadCheckpointAt = 1; + internal bool HoldReaderDispose; + private int readCheckpoint, nextCheckpoint; + internal ImportLiveCheckpoint(string phase) { TargetPhase = phase; } + internal bool Matches(string sql) { return Phase(sql) == TargetPhase; } + internal static string Phase(string sql) + { + if (sql == "DECLARE @databaseId int=DB_ID(); SELECT @databaseId,@@TRANCOUNT,XACT_STATE();") return "StateProbe"; + if (sql.StartsWith("IF OBJECT_ID(N'tempdb..#mp_", StringComparison.Ordinal)) return "LobProbe"; + if (sql.StartsWith("SELECT COUNT_BIG(*) FROM [", StringComparison.Ordinal)) return "Count"; + if (sql.Contains("/*IMPORT_INTEGRATION_DRAIN*/")) return "Drain"; + if (sql.Contains("/*IMPORT_INTEGRATION_TIMEOUT*/")) return "Timeout"; + return "Other"; + } + internal void BeforeRead() + { + int read = Interlocked.Increment(ref ReadCalls); + if (read >= ReadCheckpointAt && Interlocked.Exchange(ref readCheckpoint, 1) == 0 && OnRead != null) OnRead(); + } + internal void BeforeNext() + { + Interlocked.Increment(ref NextCalls); + if (Interlocked.Exchange(ref nextCheckpoint, 1) == 0 && OnNext != null) OnNext(); + } + public void Dispose() { Issued.Dispose(); ReaderReady.Dispose(); DisposeEntered.Dispose(); DisposeRelease.Dispose(); } + } + + private sealed class ImportLiveCommand : DbCommand + { + private readonly SqlCommand inner; + private readonly ImportLiveCheckpoint point; + internal ImportLiveCommand(SqlCommand command, ImportLiveCheckpoint checkpoint) { inner = command; point = checkpoint; } + public override string CommandText { get { return inner.CommandText; } set { inner.CommandText = value; } } + public override int CommandTimeout { get { return inner.CommandTimeout; } set { inner.CommandTimeout = value; } } + public override CommandType CommandType { get { return inner.CommandType; } set { inner.CommandType = value; } } + public override UpdateRowSource UpdatedRowSource { get { return inner.UpdatedRowSource; } set { inner.UpdatedRowSource = value; } } + public override bool DesignTimeVisible { get { return inner.DesignTimeVisible; } set { inner.DesignTimeVisible = value; } } + protected override DbConnection DbConnection { get { return inner.Connection; } set { inner.Connection = (SqlConnection)value; } } + protected override DbTransaction DbTransaction { get { return inner.Transaction; } set { inner.Transaction = (SqlTransaction)value; } } + protected override DbParameterCollection DbParameterCollection { get { return inner.Parameters; } } + protected override DbParameter CreateDbParameter() { return inner.CreateParameter(); } + public override void Cancel() { if (point != null && point.Matches(CommandText)) Interlocked.Increment(ref point.CancelCalls); inner.Cancel(); } + public override void Prepare() { inner.Prepare(); } + public override int ExecuteNonQuery() { throw new InvalidOperationException("Importer must use its actual streaming reader."); } + public override object ExecuteScalar() { throw new InvalidOperationException("Importer probes must use owned reader acknowledgement."); } + protected override DbDataReader ExecuteDbDataReader(CommandBehavior behavior) { throw new InvalidOperationException("Expected async adapter startup."); } + protected override async Task ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellation) + { + bool selected = point != null && point.Matches(CommandText); + // Only the timeout fixture's test-local actual SqlCommand receives finite timeout. + // Production adapter's timeout selection remains untouched. + if (selected && point.TargetPhase == "Timeout" && point.FiniteTimeoutSeconds > 0) inner.CommandTimeout = point.FiniteTimeoutSeconds; + Task pending = inner.ExecuteReaderAsync(behavior, cancellation); + if (selected) point.Issued.Set(); // Actual provider method invoked, not a synthetic command. + SqlDataReader actual = await pending.ConfigureAwait(false); + if (!selected) return actual; + point.ReaderReady.Set(); + return new ImportLiveReader(actual, point); + } + protected override void Dispose(bool disposing) { if (disposing) inner.Dispose(); base.Dispose(disposing); } + } + + private sealed class ImportLiveReader : DbDataReader + { + private readonly DbDataReader inner; private readonly ImportLiveCheckpoint point; + internal ImportLiveReader(DbDataReader reader, ImportLiveCheckpoint checkpoint) { inner = reader; point = checkpoint; } + public override async Task ReadAsync(CancellationToken cancellation) { point.BeforeRead(); return await inner.ReadAsync(cancellation).ConfigureAwait(false); } + public override async Task NextResultAsync(CancellationToken cancellation) { point.BeforeNext(); return await inner.NextResultAsync(cancellation).ConfigureAwait(false); } + public override bool Read() { return inner.Read(); } + public override bool NextResult() { return inner.NextResult(); } + public override int Depth { get { return inner.Depth; } } + public override int FieldCount { get { return inner.FieldCount; } } + public override bool HasRows { get { return inner.HasRows; } } + public override bool IsClosed { get { return inner.IsClosed; } } + public override int RecordsAffected { get { return inner.RecordsAffected; } } + public override object this[int ordinal] { get { return inner[ordinal]; } } + public override object this[string name] { get { return inner[name]; } } + public override bool GetBoolean(int ordinal) { return inner.GetBoolean(ordinal); } + public override byte GetByte(int ordinal) { return inner.GetByte(ordinal); } + public override long GetBytes(int ordinal, long offset, byte[] buffer, int bufferOffset, int length) { Interlocked.Increment(ref point.ValueGetterCalls); return inner.GetBytes(ordinal, offset, buffer, bufferOffset, length); } + public override char GetChar(int ordinal) { return inner.GetChar(ordinal); } + public override long GetChars(int ordinal, long offset, char[] buffer, int bufferOffset, int length) { Interlocked.Increment(ref point.ValueGetterCalls); return inner.GetChars(ordinal, offset, buffer, bufferOffset, length); } + public override string GetDataTypeName(int ordinal) { return inner.GetDataTypeName(ordinal); } + public override DateTime GetDateTime(int ordinal) { return inner.GetDateTime(ordinal); } + public override decimal GetDecimal(int ordinal) { return inner.GetDecimal(ordinal); } + public override double GetDouble(int ordinal) { return inner.GetDouble(ordinal); } + public override Type GetFieldType(int ordinal) { return inner.GetFieldType(ordinal); } + public override float GetFloat(int ordinal) { return inner.GetFloat(ordinal); } + public override Guid GetGuid(int ordinal) { return inner.GetGuid(ordinal); } + public override short GetInt16(int ordinal) { return inner.GetInt16(ordinal); } + public override int GetInt32(int ordinal) { return inner.GetInt32(ordinal); } + public override long GetInt64(int ordinal) { return inner.GetInt64(ordinal); } + public override string GetName(int ordinal) { return inner.GetName(ordinal); } + public override int GetOrdinal(string name) { return inner.GetOrdinal(name); } + public override string GetString(int ordinal) { return inner.GetString(ordinal); } + public override object GetValue(int ordinal) { Interlocked.Increment(ref point.ValueGetterCalls); return inner.GetValue(ordinal); } + public override int GetValues(object[] values) { Interlocked.Increment(ref point.ValueGetterCalls); return inner.GetValues(values); } + public override bool IsDBNull(int ordinal) { return inner.IsDBNull(ordinal); } + public override DataTable GetSchemaTable() { return inner.GetSchemaTable(); } + public override IEnumerator GetEnumerator() { return ((IEnumerable)inner).GetEnumerator(); } + public override void Close() { inner.Close(); } + protected override void Dispose(bool disposing) + { + if (disposing) { + try { + if (point.HoldReaderDispose) { + point.DisposeEntered.Set(); + if (!point.DisposeRelease.Wait(10000)) throw new TimeoutException("Test-owned reader Dispose checkpoint deadline"); + } + } finally { inner.Dispose(); } + } + base.Dispose(disposing); + } + } + + // Deferred Task 3 minor fix: owner.Released is emitted INSIDE fake Dispose before + // adapter ReleaseOwner.finally publishes SessionReleased. Reuse at the two offline + // pending-cleanup assertions only while their provider release gate remains held. + private static MssqlImportCleanup ImportWaitForPublishedReleaseWhileActive(MssqlImportSqlSession session) + { + var wait = Stopwatch.StartNew(); + while (true) { + MssqlImportCleanup receipt = session.Cleanup(); + Assert(!receipt.CommandEnded && !receipt.TransactionCleanupConfirmed, + "Provider still held: cleanup cannot fabricate command/rollback completion while release publication catches up."); + if (receipt.SessionReleased) return receipt; + if (wait.ElapsedMilliseconds >= 2000) throw new Exception("Adapter did not publish owner-release receipt before test deadline."); + Thread.Sleep(5); + } + } + + // Live checkpoint usage (the normal importer retains its production constructor): + // var owned = ConnectionOpenService.OpenMssqlImportSession(explicitFixtureSettings); + // var connection = owned.Connection; // Capture before transferring this exact lifetime. + // var adapter = new MssqlImportSqlSession(owned, + // () => new ImportLiveCommand(connection.CreateCommand(), point), + // () => connection.State == ConnectionState.Open); + // Service factory returns this actual adapter; normal runs use production constructor. + // All test checkpoint subscriptions/events/gates disposed in finally AFTER worker unwind. + // Count: independent fixture-owned transaction holds table X lock; wait Issued then + // cancel; release lock in finally. Actual SQL started and actual CancelCalls>0 required. + // State/Lob: OnRead cancellation after ReaderReady proves actual probe startup and + // adapter callback path, not prolonged server execution; label that limited boundary. + // Drain: cancel after the configured selected read count, + // next-result callback, and held Dispose; independent DB sentinel validates no next GO. + // Timeout: unchanged WAITFOR fixture SQL+finite SqlCommand timeout; require error -2. + // Transport: exact transferred owner.Dispose while command active; require uncertain + // failure and released-session evidence, never stop instance/KILL unrelated SPID. + // Closing race: hold actual reader Dispose, cancel/close in parallel, inspect pending + // cleanup; release gate then require worker unwind before final receipt. No broad sleep. + + private static void ImportDraftRequireFixture(string server, string database, string[] allowedDatabases) + { + Assert(server == @"(localdb)\mySQLPunkExportTest_20261012", "Exact authorized LocalDB endpoint only."); + Assert(allowedDatabases != null && allowedDatabases.Length == 2 && allowedDatabases.Contains(database), "Exact two generated owned DB allowlist."); + foreach (string name in allowedDatabases) { + Assert(name.StartsWith("mysqlpunk_import_", StringComparison.Ordinal) && (name.EndsWith("_src", StringComparison.Ordinal) || name.EndsWith("_dst", StringComparison.Ordinal)), "Generated fixture DB name only."); + Guid id; + Assert(name.Length == "mysqlpunk_import_".Length + 32 + 4 && Guid.TryParseExact(name.Substring("mysqlpunk_import_".Length, 32), "N", out id) && id != Guid.Empty, "GUID fixture identifier only."); + } + } + private static MssqlOwnedSession ImportDraftOpenOwned(string server, string database, string[] allowedDatabases) + { + ImportDraftRequireFixture(server, database, allowedDatabases); + ImportLiveFixture.ImportAssertRunning(); + var provider = new my_mssql(); + try { + provider.SetConn(new SqlConnectionStringBuilder { + DataSource = server, InitialCatalog = database, IntegratedSecurity = true, + Pooling = false, MultipleActiveResultSets = false, ConnectRetryCount = 0, + TrustServerCertificate = true, ConnectTimeout = 8 + }.ConnectionString); + provider.Open(); + return new MssqlOwnedSession(provider, provider.MCT); + } catch { provider.Dispose(); throw; } + } + private static MssqlImportResult ImportDraftRunActual(string server, string target, string[] allowedDatabases, + string script, string output, bool skip = false, CancellationToken cancellation = default(CancellationToken), + ImportLiveCheckpoint point = null, Func logFactory = null, + Action progress = null) + { + // No fake session, external sqlcmd, Exec loop or synthetic success result. + Func sessions = () => { + var owned = ImportDraftOpenOwned(server, target, allowedDatabases); + if (point == null) return new ImportObservedSession(new MssqlImportSqlSession(owned), Path.Combine(output,"state-"+Guid.NewGuid().ToString("N")+".csv")); + var connection = owned.Connection; + return new MssqlImportSqlSession(owned, () => new ImportLiveCommand(connection.CreateCommand(), point), + () => connection.State == ConnectionState.Open); + }; + var service = new MssqlImportService(sessions, logFactory ?? (job => new MssqlImportLog(job, Path.Combine(output, "logs")))); + var request = new MssqlImportRequest(Guid.NewGuid(), script, target, skip); + using (var preflight = service.Preflight(request, progress, cancellation)) { + Assert(preflight.Passed, "Actual SUT preflight: " + string.Join(",", preflight.Issues.Where(x => !x.IsWarning).Select(x => x.Code))); + return service.Run(request, preflight, progress, cancellation); + } + } + private static void ImportDraftRunGenericSessionCase(string server, string target, string[] allowed, string output) + { + const string text = @"SET DATEFIRST 3; +CREATE TABLE #carry(id int NOT NULL PRIMARY KEY); +INSERT #carry VALUES(7); +CREATE TABLE dbo.ImportSession(id int IDENTITY(10,2) NOT NULL PRIMARY KEY,v int NOT NULL); +GO -- supported delimiter comment +IF @@DATEFIRST<>3 OR (SELECT COUNT(*) FROM #carry)<>1 THROW 51000,'Session continuity',1; +SET IDENTITY_INSERT dbo.ImportSession ON; +GO +INSERT dbo.ImportSession(id,v) VALUES(37,7); +SET IDENTITY_INSERT dbo.ImportSession OFF; +BEGIN TRANSACTION; +INSERT dbo.ImportSession(v) VALUES(11); +GO +IF @@TRANCOUNT<>1 THROW 51000,'Transaction continuity',1; +INSERT dbo.ImportSession(v) VALUES(13); +SELECT id,v FROM dbo.ImportSession ORDER BY id; +SELECT COUNT_BIG(*) AS rows FROM dbo.ImportSession; +COMMIT TRANSACTION; +GO +"; + string path = Path.Combine(output, "generic-session.sql"); + File.WriteAllText(path, text.Replace("\r\n", "\n").Replace("\n", "\r\n"), new UTF8Encoding(false)); + var result = ImportDraftRunActual(server, target, allowed, path, output); + Assert(result.Status == MssqlImportStatus.Completed && result.FailedBatches == 0 && result.LastAcknowledgedBatch == 4, + "Actual generic same-session/transaction/identity/result batches complete."); + Assert(result.Cleanup.CommandEnded && result.Cleanup.SessionReleased && result.Cleanup.TransactionCleanupConfirmed, + "Actual command, owned connection and transaction cleanup each confirmed."); + using (var owner = ImportDraftOpenOwned(server, target, allowed)) { + using (var cmd = new SqlCommand("SELECT COUNT_BIG(*) FROM dbo.ImportSession WHERE (id=37 AND v=7) OR (id=39 AND v=11) OR (id=41 AND v=13)", owner.Connection)) + Assert(Convert.ToInt64(cmd.ExecuteScalar()) == 3, "Independent actual DB content matches identity and committed transaction."); + } + } + private static MssqlExportRequest ImportDraftExportNative(string server, string source, string[] allowed, + string output, string filename, MssqlExportMode mode, Func select = null, bool views = true) + { + var recovery = new MssqlSnapshotRecoveryStore(Path.Combine(output, "export-journal")); + var exporter = new MssqlSqlExportService(() => new MssqlExportSource(ImportDraftOpenOwned(server, source, allowed), recovery), + new MssqlArtifactFiles(), null, recovery); + var tables = exporter.LoadCatalog(source, CancellationToken.None).Tables.Where(t => select == null || select(t)).Select(t => t.Key).ToArray(); + string path = Path.Combine(output, filename); + var request = new MssqlExportRequest(Guid.NewGuid(), source, tables, mode, views, path, + MssqlExportArtifact.CaptureDestination(path, CancellationToken.None), null, false); + Assert(exporter.Preflight(request, CancellationToken.None).CanStart, "Actual exporter fixture preflight."); + var exported = exporter.Export(request, new WorkerProgress(p => { }), CancellationToken.None); + Assert(exported.Stage == MssqlExportStage.Completed && File.Exists(exported.PublishedPath), "Actual native exporter published artifact."); + return request; + } + private static void ImportDraftCompareSmallCatalogs(SqlConnection source, SqlConnection target) + { + // Existing tiny catalog helper allowed here; never applied to unbounded user DBs. + foreach (var query in IntegrationMetadataQueries()) { + if (query.Key == "comments") continue; + Assert(IntegrationCatalog(source, query.Value) == IntegrationCatalog(target, query.Value), "Actual native catalog equality: " + query.Key); + } + const string properties = "SELECT OBJECT_SCHEMA_NAME(major_id),OBJECT_NAME(major_id),minor_id,name,CONVERT(nvarchar(max),value),CONVERT(nvarchar(128),SQL_VARIANT_PROPERTY(value,'BaseType')) FROM sys.extended_properties WHERE class=1"; + Assert(IntegrationCatalog(source, properties) == IntegrationCatalog(target, properties), "All table/column properties match; database ownership marker excluded."); + } + private static string ImportDraftCurrentRowDigest(SqlDataReader reader, byte[] buffer) + { + using (var hash = SHA256.Create()) { + for (int i = 0; i < reader.FieldCount; i++) { + bool nil = reader.IsDBNull(i); + // Sequential column hashes bind NULL and byte length, not only concatenated bytes. + long length = nil ? -1 : reader.GetBytes(i, 0, null, 0, 0); + byte[] marker = BitConverter.GetBytes(length); + hash.TransformBlock(marker, 0, marker.Length, null, 0); + if (nil) continue; + long at = 0; int read; + while ((read = checked((int)reader.GetBytes(i, at, buffer, 0, buffer.Length))) != 0) { + hash.TransformBlock(buffer, 0, read, null, 0); at += read; + } + Assert(at == length, "Sequential raw-byte read matches actual length."); + } + hash.TransformFinalBlock(new byte[0], 0, 0); + return BitConverter.ToString(hash.Hash).Replace("-", "").ToLowerInvariant(); + } + } + private static long ImportDraftCompareOrderedBytes(SqlConnection source, SqlConnection target, string fixedFixtureSql) + { + var leftBuffer = new byte[65536]; var rightBuffer = new byte[65536]; long rows = 0; + using (var a = new SqlCommand(fixedFixtureSql, source) { CommandTimeout = 0 }) + using (var b = new SqlCommand(fixedFixtureSql, target) { CommandTimeout = 0 }) + using (var left = a.ExecuteReader(CommandBehavior.SequentialAccess)) + using (var right = b.ExecuteReader(CommandBehavior.SequentialAccess)) { + Assert(left.FieldCount == right.FieldCount, "Same projected column count."); + while (left.Read()) { + Assert(right.Read(), "Target has corresponding ordered row."); + Assert(ImportDraftCurrentRowDigest(left, leftBuffer) == ImportDraftCurrentRowDigest(right, rightBuffer), "NULL/length/raw-byte row digest matches."); + rows++; + } + Assert(!right.Read(), "No extra target rows."); + } + return rows; + } + + // Fixed fixture SQL supplies a stable unique ORDER BY; query projects varbinary(max), + // text/ntext/XML via CONVERT, geometry via Serialize(). Never hash rowversion as copied + // content; verify regenerated length/metadata separately. Do not reuse List + // IntegrationRowHashes for millions of rows, and do not GetValue on any LOB. + + private const string ImportDraftLobSetup = @" +CREATE TABLE dbo.ImportLob(id int NOT NULL PRIMARY KEY,b varbinary(max),n nvarchar(max),x xml); +INSERT dbo.ImportLob VALUES(1,NULL,NULL,NULL),(2,0x,N'',CONVERT(xml,N'')), +(3,CONVERT(varbinary(max),REPLICATE(CONVERT(varchar(max),'B'),67108864)), + REPLICATE(CONVERT(nvarchar(max),N'W'),9437182)+NCHAR(0xD83D)+NCHAR(0xDE00),CONVERT(xml,N' a b ',1)); +CREATE TABLE dbo.ImportLobGuard(id int NOT NULL PRIMARY KEY,b varbinary(max),x xml); +INSERT dbo.ImportLobGuard VALUES(1,CONVERT(varbinary(max),REPLICATE(CONVERT(varchar(max),'x'),300000)),CONVERT(xml,N'')), +(2,0x42,CONVERT(xml,N'')); +"; + private const string ImportDraftGenericPartialFailure = @" +CREATE TABLE dbo.ImportFault(id int NOT NULL PRIMARY KEY); +GO +INSERT dbo.ImportFault VALUES(1); +RAISERROR(N'first runtime error',16,1); +RAISERROR(N'second runtime error',16,2); +GO +INSERT dbo.ImportFault VALUES(2); +GO +"; + private const string ImportDraftTimeout = @" +CREATE TABLE dbo.ImportTimeout(id int); +GO +/*IMPORT_INTEGRATION_TIMEOUT*/ WAITFOR DELAY '00:00:08'; +INSERT dbo.ImportTimeout VALUES(1); +GO +INSERT dbo.ImportTimeout VALUES(2); +GO +"; + private const string ImportDraftDrainingResults = @" +/*IMPORT_INTEGRATION_DRAIN*/ +WITH n AS(SELECT TOP(2500000) ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) AS n FROM sys.all_objects a CROSS JOIN sys.all_objects b) +SELECT n FROM n; +SELECT CONVERT(varbinary(max),REPLICATE(CONVERT(varchar(max),'z'),67108864)) AS big; +SELECT 1 AS final_result; +GO +"; + private const string ImportDraftTransactionFailure = @" +CREATE TABLE dbo.ImportTransaction(id int NOT NULL PRIMARY KEY); +GO +BEGIN TRANSACTION; +INSERT dbo.ImportTransaction VALUES(1); +GO +INSERT dbo.ImportTransaction VALUES(1); +GO +INSERT dbo.ImportTransaction VALUES(2); +COMMIT TRANSACTION; +GO +"; + + // Mutate only the first large-row protocol in a TINY guard artifact generated by + // actual exporter. Retains at most two batch strings. Footer/native GUID unchanged. + private static void ImportDraftMutateGuardArtifact(string source, string destination, string fault) + { + using (var input = MssqlImportInput.Open(source)) + using (var writer = new StreamWriter(new FileStream(destination, FileMode.CreateNew, FileAccess.Write, FileShare.None, 65536), new UTF8Encoding(false), 65536)) { + var scan=input.Scan(CancellationToken.None,null); + bool firstChunk = false, changed = false, firstCommit = false; + MssqlImportBatch held = null; + foreach (var batch in input.ReadBatches(CancellationToken.None)) { + string text = batch.Text; + bool chunk = text.Contains(".WRITE("); + if (fault == "reordered" && held != null) { + Assert(chunk, "Guard fixture first chunks are adjacent export batches."); + writer.Write(text); writer.Write("\r\nGO\r\n"); + writer.Write(held.Text); writer.Write("\r\nGO\r\n"); + held = null; changed = true; continue; + } + if (chunk && !firstChunk) { + firstChunk = true; + if (fault == "missing") { changed = true; continue; } + if (fault == "reordered") { held = batch; continue; } + if (fault == "deserialize") { + int hex = text.IndexOf(".WRITE(0x", StringComparison.Ordinal) + 9; + Assert(hex >= 9 && hex + 10 <= text.Length, "Serialized geometry first chunk found."); + // Serialized header: SRID 4 bytes then version byte; alter version, + // retain total byte count so error occurs in actual Deserialize. + text = text.Substring(0, hex + 8) + "FF" + text.Substring(hex + 10); changed = true; + } + } + writer.Write(text); writer.Write("\r\nGO\r\n"); + if (chunk && firstChunk && !changed && fault == "duplicate") { writer.Write(text); writer.Write("\r\nGO\r\n"); changed = true; } + // Artifact selected only ImportLobGuard or GeometryGuard; first commit + // identified by target INSERT, not arbitrary comments/messages. + bool commit = text.Contains("INSERT INTO [dbo].[ImportLobGuard]") || text.Contains("INSERT INTO [dbo].[GeometryGuard]"); + if (fault == "replayed" && commit && !firstCommit) { firstCommit = true; writer.Write(text); writer.Write("\r\nGO\r\n"); changed = true; } + } + Assert(held == null && changed, "Requested bounded protocol mutation applied once."); + writer.Write("-- mysqlpunk-export-complete job="+scan.Script.NativeJobId.Value.ToString("N")+" tables="+scan.Script.ExpectedTables+" rows="+scan.Script.ExpectedRows+"\r\n"); + } + } + private static void ImportDraftGenerateGeometry(SqlConnection source, string wkbPath, string fixedOwnedTable, int points) + { + Assert(fixedOwnedTable == "dbo.GeometryLarge" || fixedOwnedTable == "dbo.GeometryGuard", "Exact fixed fixture table only."); + using (var stream = new FileStream(wkbPath, FileMode.CreateNew, FileAccess.Write, FileShare.None, 65536)) + using (var writer = new BinaryWriter(stream)) { + writer.Write((byte)1); writer.Write((uint)2); writer.Write((uint)points); + for (int i=0; i1) forces final INSERT failure. XML mutation + // changes raw UTF16 '' to '' same length before final CONVERT(xml). + // Missing state / duplicate state / pending EOF mutations must preserve native + // header/footer and inspector role; SQL-corruption preflight failure is separate. + + // Faults workflow: new case-specific tables/targets, run actual SUT skip=false + // then skip=true; independent SELECT proves partial id1 remains, id2 only on safe + // skip. Full actual log must contain both RAISERROR details before allowing skip. + // Transaction error runs both policies: rollback cleanup should leave no committed + // rows and no next batch, while earlier DDL may remain. Unclosed successful BEGIN + // reaching EOF is failure; receipt records rollback confirmed only from actual query. + // Timeout checkpoint wraps actual WAITFOR SqlCommand with 1 sec timeout, requires + // provider -2 and stopped sentinel. No timeout on ordinary production execution. + // Log factory fail-create is before write; wrapper of actual log can fail Start, + // append, Flush or End independently, preserving any already written evidence. + + private static long ImportDraftGenerateMemorySql(string destination, int scale) + { + Assert(scale == 1 || scale == 10, "Only planned 1x/10x memory scales."); + // Identical ~1MiB command batches, 256 vs2560 batches. Small retained comment + // block and fixed writer buffers; no whole-file bytes/text/batch collections. + string block = new string('m', 65536); + using (var stream = new FileStream(destination, FileMode.CreateNew, FileAccess.Write, FileShare.None, 65536)) + using (var writer = new StreamWriter(stream, new System.Text.UTF8Encoding(false), 65536)) { + for (int batch = 0; batch < checked(256 * scale); batch++) { + writer.Write("SELECT CAST(1 AS int); /*"); + for (int part=0; part<16; part++) writer.Write(block); + writer.Write("*/\r\nGO\r\n"); + } + } + return new FileInfo(destination).Length; + } + private sealed class ImportDraftMemory : IDisposable + { + internal volatile string Stage = "Baseline"; + private readonly object gate = new object(); + private readonly Dictionary stages = new Dictionary(); + private readonly Timer timer; + internal ImportDraftMemory() { timer = new Timer(_ => Sample(), null, 0, 25); } + private void Sample() + { + using (var process = Process.GetCurrentProcess()) { + process.Refresh(); string stage = Stage; + lock (gate) { + ImportDraftMemoryStage value; + if (!stages.TryGetValue(stage, out value)) stages[stage] = value = new ImportDraftMemoryStage { + FirstPrivateBytes=process.PrivateMemorySize64, FirstWorkingSet=process.WorkingSet64 + }; + value.PeakPrivateBytes=Math.Max(value.PeakPrivateBytes,process.PrivateMemorySize64); + value.PeakWorkingSet=Math.Max(value.PeakWorkingSet,process.WorkingSet64); + value.LastPrivateBytes=process.PrivateMemorySize64; value.LastWorkingSet=process.WorkingSet64; + value.Samples++; + } + } + } + internal Dictionary Snapshot() + { + Sample(); var result = new Dictionary(); + lock(gate) foreach(var pair in stages) result[pair.Key] = pair.Value.Copy(); + return result; + } + public void Dispose() { using(var done = new ManualResetEvent(false)) { timer.Dispose(done); Assert(done.WaitOne(5000), "Memory sampler disposed within deadline."); } } + } + private sealed class ImportDraftMemoryStage + { + public long FirstPrivateBytes,FirstWorkingSet,PeakPrivateBytes,PeakWorkingSet,LastPrivateBytes,LastWorkingSet,Samples; + internal ImportDraftMemoryStage Copy() { return (ImportDraftMemoryStage)MemberwiseClone(); } + } + // Memory child contract: + // - Parent saves/restores MYSQLPUNK_MSSQL_IMPORT_MEMORY_SCALE as well as five required + // import flags+fixed smoke filter. Scale accepts only1/10, no arbitrary child command. + // - Child repeats exact manifest/endpoint/root/ownership guards, requires IntPtr.Size=8. + // - Generate input before measurement; own same tiny DB setup; warm actual importer + // with identical bounded batch shape before baseline. Parent builds once before any + // measured child. Never ReportExportTests or parallel builds/profiles. + // - New sampler after warmup; Stage from actual import progress: Scanning/Executing/ + // Verifying/Finished, plus Begin measurement. Actual Run consumes same preflight + // handle, completes all256/2560 batches. Record script streamed SHA + exact bytes. + // - Child saves actual result/cleanup/log, baseline and per-stage aggregate receipts. + // Parent independently requires both child receipts and6 expected Memory case IDs. + // - Compare peak-private-minus-warm-baseline deltas and raw peaks; >256MiB difference + // requires investigation/fix; ratio is evidence, never sole pass when no work matched. + // - Include million-row and64MiB returned-LOB drain cases with separate aggregates; + // no result field getters or DataTable and no per-row memory/progress history. + + private sealed class ImportDraftLogFailure : IMssqlImportLog + { + private readonly MssqlImportLog inner; private readonly ImportDraftFailingStream stream; + private readonly string fault; + internal ImportDraftLogFailure(Guid job,string directory,string mode) + { + fault=mode; + if(mode=="create") throw new IOException("Fixture log create failure"); + ImportDraftFailingStream created=null; + inner=new MssqlImportLog(job,directory,path => created=new ImportDraftFailingStream(new FileStream(path,FileMode.CreateNew,FileAccess.Write,FileShare.Read))); + stream=created; + } + public string Path {get{return inner.Path;}} + public MssqlImportLogTail Tail {get{return inner.Tail;}} + public void WriteStart(MssqlImportRequest request,MssqlImportScan scan,MssqlImportTarget target) + { if(fault=="start") stream.FailWrite=true; inner.WriteStart(request,scan,target); } + public void WriteError(MssqlImportBatch batch,MssqlImportBatchException error,string action) + { + if(fault=="append") stream.FailWrite=true; + inner.WriteError(batch,error,action); + // append fixture uses >=3 actual SQL errors of1800 chars each so actual + // StreamWriter writes >4096 buffer while WriteError is still running. + } + public void Flush() { if(fault=="flush") stream.FailFlush=true; inner.Flush(); } + public void WriteEnd(MssqlImportResult result) { if(fault=="end") stream.FailWrite=true; inner.WriteEnd(result); } + public void Dispose() { inner.Dispose(); } + } + private sealed class ImportDraftFailingStream : Stream + { + private readonly FileStream inner; + internal bool FailWrite,FailFlush; internal int RejectedWrites,RejectedFlushes; + internal ImportDraftFailingStream(FileStream file){inner=file;} + public override bool CanRead{get{return false;}} public override bool CanSeek{get{return inner.CanSeek;}} public override bool CanWrite{get{return true;}} + public override long Length{get{return inner.Length;}} public override long Position{get{return inner.Position;}set{inner.Position=value;}} + public override int Read(byte[] buffer,int offset,int count){throw new NotSupportedException();} + public override long Seek(long offset,SeekOrigin origin){return inner.Seek(offset,origin);} + public override void SetLength(long value){inner.SetLength(value);} + public override void Write(byte[] buffer,int offset,int count){if(FailWrite){RejectedWrites++;throw new IOException("Fixture actual log stream write failure");}inner.Write(buffer,offset,count);} + public override void Flush(){if(FailFlush){RejectedFlushes++;throw new IOException("Fixture actual log stream flush failure");}inner.Flush(true);} + protected override void Dispose(bool disposing){if(disposing)inner.Dispose();base.Dispose(disposing);} + } + // Actual service's factory: job=>new ImportDraftLogFailure(job,ownedRunLogs,mode). + // Preflight failure for create/start, worker stop for append/flush/end; verify actual + // DB next-GO sentinel absence and status, not merely seam exception count. Available + // log bytes/hash and rejected stream operation counts survive into failure receipt. + // Provider unwind precedes watchdog owner publication; the actual worker must wait + // only within its bounded diagnostic window and never retry the owned Dispose. + private static void ImportInverseReleasePublication(bool beyondWindow) + { + using(var f=new ImportWorkerFixture("SELECT 'inverse-release';\r\nGO\r\n")) + using(var issued=new ManualResetEventSlim())using(var providerReturn=new ManualResetEventSlim()) + using(var ownerEntered=new ManualResetEventSlim())using(var ownerReturn=new ManualResetEventSlim()) + using(var cleanupEntered=new ManualResetEventSlim())using(var cancel=new CancellationTokenSource()) { + var owner=new ImportSessionOwner{Error=DiagnosticSql(2627,50001),BeforeReturn=()=>{ownerEntered.Set();Assert(ownerReturn.Wait(5000),"Inverse owner gate bounded.");}}; + int opens=0;MssqlImportSqlSession execution=null; + f.Service=new MssqlImportService(()=>{ + bool worker=++opens==2; + var adapter=new MssqlImportSqlSession(worker?owner:new ImportSessionOwner(),()=>{ + var command=new ImportSessionCommand(null); + command.Start=(behavior,token)=>{ + if(command.CommandText.StartsWith("SELECT CONVERT(nvarchar(128),SERVERPROPERTY",StringComparison.Ordinal))return Task.FromResult(ImportValues(ImportIdentity(1))); + if(command.CommandText=="SELECT COUNT_BIG(*) FROM sys.objects WHERE is_ms_shipped=0;")return Task.FromResult(ImportValues(0L)); + if(command.CommandText=="DECLARE @databaseId int=DB_ID(); SELECT @databaseId,@@TRANCOUNT,XACT_STATE();")return Task.FromResult(ImportValues(7,0,0)); + Assert(worker&&command.CommandText.Contains("inverse-release"),"Only unchanged source batch reaches provider gate."); + issued.Set();Assert(providerReturn.Wait(5000),"Inverse provider gate bounded."); + return Task.FromResult(new ImportSessionReader(new[]{0},true)); + };return command; + },()=>true,40,beyondWindow?150:2000); + if(!worker)return (IMssqlImportSession)adapter; + execution=adapter;return new ImportCleanupEnteredSession(adapter,cleanupEntered); + },job=>new MssqlImportLog(job,f.DirectoryPath)); + Task run=null; + try { + using(var pre=f.Service.Preflight(f.Request,null,CancellationToken.None)) { + Assert(pre.Passed,"Inverse release actual adapter preflight passes."); + run=Task.Run(()=>f.Service.Run(f.Request,pre,null,cancel.Token)); + Assert(issued.Wait(2000),"Inverse worker command entered.");cancel.Cancel(); + Assert(ownerEntered.Wait(2000),"Watchdog enters owned Dispose before provider unwind."); + providerReturn.Set();Assert(cleanupEntered.Wait(2000),"Provider unwinds first and actual worker enters Cleanup."); + if(!beyondWindow) { + Assert(!run.Wait(75),"Final worker log cannot precede an in-window owner publication."); + ownerReturn.Set();Assert(run.Wait(3000),"Worker completes after delivered owner details."); + }else Assert(run.Wait(2000),"Pending owner cannot hold final worker beyond bounded cleanup wait."); + var result=run.Result; + Assert(result.Status==MssqlImportStatus.CleanupUncertain&&result.Cleanup.CommandEnded&&!result.Cleanup.SessionReleased&&!result.Cleanup.TransactionCleanupConfirmed, + "Inverse cancellation preserves command acknowledgement and uncertain release/rollback truth."); + string log=File.ReadAllText(result.LogPath); + if(!beyondWindow) { + Assert(result.Cleanup.Errors.Select(x=>x.Number).SequenceEqual(new[]{2627,50001}),"In-window inverse publication retains both typed SqlError details."); + foreach(int code in new[]{2627,50001})Assert(log.Split(new[]{"Number="+code+" "},StringSplitOptions.None).Length==2,"Actual worker log records delivered owner error exactly once."); + Assert(log.IndexOf("Origin=Cleanup",StringComparison.Ordinal){var publication=typeof(MssqlImportSqlSession).GetField("gate",System.Reflection.BindingFlags.Instance|System.Reflection.BindingFlags.NonPublic).GetValue(execution);lock(publication)return (bool)completed.GetValue(execution);},2000),"Late publication is observed boundedly after terminal timeout."); + Assert(log==File.ReadAllText(result.LogPath),"Late publication cannot append duplicate or post-terminal worker log entries."); + } + Assert(object.ReferenceEquals(result.Cleanup,execution.Cleanup())&&owner.Disposes==1,"Final snapshot remains one-shot; late owner completion cannot redispose or relog."); + execution.Dispose();Assert(owner.Disposes==1,"Dispose after cached cleanup never retries owner."); + } + }finally { + ownerReturn.Set();providerReturn.Set(); + if(run!=null)Assert(run.Wait(3000),"Inverse worker always unwinds before fixture release."); + } + } + } + private sealed class ImportCleanupEnteredSession : IMssqlImportSession + { + private readonly MssqlImportSqlSession inner;private readonly ManualResetEventSlim entered; + internal ImportCleanupEnteredSession(MssqlImportSqlSession inner,ManualResetEventSlim entered){this.inner=inner;this.entered=entered;} + public MssqlImportTarget ReadTarget(CancellationToken token){return inner.ReadTarget(token);} + public MssqlImportSessionState ReadState(CancellationToken token){return inner.ReadState(token);} + public MssqlImportLobState ReadLobState(Guid job,CancellationToken token){return inner.ReadLobState(job,token);} + public void ExecuteBatch(MssqlImportBatch batch,CancellationToken token){inner.ExecuteBatch(batch,token);} + public MssqlImportCounts CountTargetRows(Action progress,CancellationToken token){return inner.CountTargetRows(progress,token);} + public void CancelActiveCommand(){inner.CancelActiveCommand();} + public MssqlImportCleanup Cleanup(){entered.Set();return inner.Cleanup();} + public void Dispose(){inner.Dispose();} + } + private static void MssqlImportCleanupErrorDiagnostics() + { + ImportInverseReleasePublication(false); + ImportInverseReleasePublication(true); + var cleanupCommand=new ImportSessionCommand(ImportValues(0,0)){StartError=DiagnosticSql(547,50000)}; + using(var session=new MssqlImportSqlSession(new ImportSessionOwner(),()=>cleanupCommand,()=>true)) { + var receipt=session.Cleanup(); + Assert(receipt.Errors.Count==2 && receipt.Errors[0].Number==547 && receipt.Errors[1].Number==50000, + "Cleanup must preserve every delivered provider SqlError, rather than only a fixed failure code."); + Assert(receipt.CommandEnded&&receipt.SessionReleased&&!receipt.TransactionCleanupConfirmed&&receipt.IssueCode=="TransactionCleanupFailed", "Preserving diagnostics cannot fabricate rollback success."); + } + var releaseOwner=new ImportSessionOwner{Error=DiagnosticSql(2627,50001)}; + using(var session=new MssqlImportSqlSession(releaseOwner,()=>new ImportSessionCommand(ImportValues(0,0)),()=>true)){ + var receipt=session.Cleanup(); + Assert(receipt.Errors.Select(x=>x.Number).SequenceEqual(new[]{2627,50001}),"Every actual typed SqlError thrown by owned Dispose survives cleanup receipt."); + Assert(receipt.CommandEnded&&!receipt.SessionReleased&&receipt.TransactionCleanupConfirmed,"Release SQL details cannot rewrite release/rollback truth."); + Assert(object.ReferenceEquals(receipt,session.Cleanup())&&releaseOwner.Disposes==1,"Cached receipt neither repeats release nor duplicates release details."); + } + var mergedOwner=new ImportSessionOwner{Error=DiagnosticSql(2627,50001)}; + using(var session=new MssqlImportSqlSession(mergedOwner,()=>new ImportSessionCommand(ImportValues(0,0)){StartError=DiagnosticSql(547,50000)},()=>true)) + using(var f=new ImportWorkerFixture()){ + f.Session.Finish=()=>session.Cleanup();var result=f.Run(); + Assert(result.Cleanup.Errors.Select(x=>x.Number).SequenceEqual(new[]{547,50000,2627,50001}),"Rollback and current-release SQL details merge once in actual order."); + string log=File.ReadAllText(result.LogPath); + Assert(log.Contains("Origin=Cleanup")&&new[]{547,50000,2627,50001}.All(number=>log.Contains("Number="+number+" ")),"Actual worker log preserves merged cleanup and release diagnostics."); + Assert(result.Status==MssqlImportStatus.CleanupUncertain&&!result.Cleanup.SessionReleased&&!result.Cleanup.TransactionCleanupConfirmed,"Merged error log retains uncertain cleanup truth."); + } foreach(bool primaryFailure in new[]{false,true})using(var f=new ImportWorkerFixture()) { + if(primaryFailure)f.Session.Execute=(b,t)=>{throw WorkerSqlError();}; + var errors=new[]{new MssqlImportSqlError(547,2,16,9,"cleanup detail one"),new MssqlImportSqlError(50000,3,16,10,"cleanup detail two")}; + f.Session.Finish=()=>new MssqlImportCleanup(true,true,false,"TransactionCleanupFailed",errors); + var result=f.Run();string log=File.ReadAllText(result.LogPath); + Assert(log.Contains("Origin=Cleanup")&&log.Contains("RelativeLine=9 SourceLine=unknown Message=cleanup detail one")&&log.Contains("cleanup detail two"), + "Actual worker log streams all cleanup SQL diagnostics with unknown source coordinates."); + Assert(log.IndexOf("Origin=Cleanup",StringComparison.Ordinal) created = new List(); + private bool disposed; + internal string CleanupStatus = "Pending"; + internal readonly Dictionary Receipts = new Dictionary(); + internal ImportLiveFixture(string acceptanceProfile) + { + if(Environment.GetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_INTEGRATION")!="1" || !ImportProfiles.Contains(acceptanceProfile))throw new InvalidOperationException("ExplicitImportRunnerRequired"); + AcceptanceProfile=acceptanceProfile;Server=Environment.GetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_SERVER"); + string authorization=Environment.GetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_AUTH"); + if(string.IsNullOrWhiteSpace(authorization)||!File.Exists(authorization))throw new InvalidOperationException("FixtureAuthorizationMissing"); + JObject manifest=JObject.Parse(File.ReadAllText(authorization)); + string output=Environment.GetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_OUTPUT"); + if(string.IsNullOrWhiteSpace(output))throw new InvalidOperationException("FixtureOutputMissing"); + Root=ImportValidateEnvelope(manifest,Server,Path.Combine(output,acceptanceProfile),new DriveInfo("C").AvailableFreeSpace,ImportReadGuardPath); + ImportAssertRunning(); // SqlConnection.Open can implicitly start LocalDB; reject BEFORE Open. + string identity=Guid.NewGuid().ToString("N");Source="mysqlpunk_import_"+identity+"_src";Target="mysqlpunk_import_"+identity+"_dst";Databases=new[]{Source,Target}; + Directory.CreateDirectory(Root); + using(var token=new FileStream(Path.Combine(Root,"owner.token"),FileMode.CreateNew,FileAccess.Write,FileShare.None)) + using(var writer=new StreamWriter(token,new UTF8Encoding(false))){writer.Write(owner);writer.Flush();token.Flush(true);} + try {CreateOwned(Source);CreateOwned(Target);} + catch {try{Dispose();}catch{}throw;} + } + internal static void ImportAssertRunning() + { + var start=new ProcessStartInfo("sqllocaldb.exe","info mySQLPunkExportTest_20261012") {UseShellExecute=false,CreateNoWindow=true,RedirectStandardOutput=true,RedirectStandardError=true}; + using(var process=Process.Start(start)) { + string output=process.StandardOutput.ReadToEnd();string error=process.StandardError.ReadToEnd(); + if(!process.WaitForExit(5000)||process.ExitCode!=0||output.IndexOf("LOCALDB#",StringComparison.OrdinalIgnoreCase)<0) + throw new InvalidOperationException("AuthorizedLocalDBMustAlreadyBeRunning"); + // No owner/account/raw instance output is printed or retained in evidence. + GC.KeepAlive(error); + } + } + private void RequireOwnerFile() + { + ImportNoReparse(Path.Combine(Root,"owner.token"),ImportReadGuardPath); + if(File.ReadAllText(Path.Combine(Root,"owner.token"))!=owner)throw new InvalidOperationException("FixtureFileOwnershipMismatch"); + } + private SqlConnection OpenAdmin(string database) + { + if(database!="master"&&!Databases.Contains(database))throw new InvalidOperationException("FixtureDatabaseNotAllowed"); + RequireOwnerFile();ImportAssertRunning(); + var connection=new SqlConnection(new SqlConnectionStringBuilder{DataSource=Server,InitialCatalog=database,IntegratedSecurity=true,Pooling=false,ConnectRetryCount=0,MultipleActiveResultSets=false,TrustServerCertificate=true,ConnectTimeout=8}.ConnectionString); + try{connection.Open();return connection;}catch{connection.Dispose();throw;} + } + internal SqlConnection Open(string database) + { if(!Databases.Contains(database))throw new InvalidOperationException("FixtureDatabaseNotAllowed");return OpenAdmin(database); } + private void CreateOwned(string database) + { + if(!Databases.Contains(database)||created.Contains(database))throw new InvalidOperationException("FixtureCreateNameNotOwned"); + RequireOwnerFile(); + if(new DriveInfo("C").AvailableFreeSpace<20L*1024*1024*1024)throw new InvalidOperationException("FixtureInsufficientSpace"); + string data=Path.Combine(Root,database+".mdf"),log=Path.Combine(Root,database+".ldf"); + if(File.Exists(data)||File.Exists(log))throw new InvalidOperationException("FixtureCreateFilesAlreadyExist"); + ImportNoReparse(data,ImportReadGuardPath);ImportNoReparse(log,ImportReadGuardPath); + using(var connection=OpenAdmin("master"))Exec(connection,"CREATE DATABASE "+Q(database)+" ON PRIMARY(NAME=N'data',FILENAME="+L(data)+",SIZE=16MB,MAXSIZE=8192MB,FILEGROWTH=64MB) LOG ON(NAME=N'log',FILENAME="+L(log)+",SIZE=16MB,MAXSIZE=2048MB,FILEGROWTH=64MB) COLLATE Latin1_General_100_BIN2;"); + created.Add(database); // CREATE succeeded: retain exact name even if marking fails. + using(var connection=OpenAdmin(database))Exec(connection,"EXEC sys.sp_addextendedproperty @name=N'mysqlpunk_import_owner',@value="+L(owner)+";"); + } + private void RequireOwnedDatabaseFiles(string database) + { + // Fresh ownership and exact physical-file evidence before any owned DB mutation. + RequireOwnerFile(); + if(!Databases.Contains(database)||!created.Contains(database))throw new InvalidOperationException("FixtureMutationNameNotOwned"); + string marker;var files=new List(); + using(var connection=OpenAdmin(database)) { + marker=Convert.ToString(Scalar(connection,"SELECT value FROM sys.extended_properties WHERE class=0 AND name=N'mysqlpunk_import_owner';")); + using(var command=new SqlCommand("SELECT physical_name FROM sys.database_files;",connection))using(var reader=command.ExecuteReader())while(reader.Read())files.Add(reader.GetString(0)); + } + ImportAssertCleanup(database,created.ToArray(),Databases,Root,owner,File.ReadAllText(Path.Combine(Root,"owner.token")),marker,files.ToArray(),ImportReadGuardPath); + } + internal void SetTargetReadOnly(bool readOnly) + { + RequireOwnedDatabaseFiles(Target); + // Fixed exact target/options; NO_WAIT fails rather than terminating other sessions. + using(var connection=OpenAdmin("master"))Exec(connection,"ALTER DATABASE "+Q(Target)+(readOnly?" SET READ_ONLY WITH NO_WAIT;":" SET READ_WRITE WITH NO_WAIT;")); + } + internal void RecordTargetProperties(string label) + { + using(var connection=OpenAdmin("master")) + using(var command=new SqlCommand("SELECT is_read_only,is_auto_close_on FROM sys.databases WHERE name="+L(Target)+";",connection)) + using(var reader=command.ExecuteReader()) { + Assert(reader.Read(),"Exact owned target remains visible for lifecycle observation."); + File.WriteAllText(Path.Combine(Root,"target-"+label+".json"),JsonConvert.SerializeObject(new{label,database=Target,observationDatabase=connection.Database,isReadOnly=reader.GetBoolean(0),autoClose=reader.GetBoolean(1)},Formatting.Indented)); + } + } + private void DropOwned(string database) + { + RequireOwnedDatabaseFiles(database); + // Exact generated DB and physical paths + dual ownership rechecked immediately before mutation. + using(var connection=OpenAdmin("master"))Exec(connection,"ALTER DATABASE "+Q(database)+" SET SINGLE_USER WITH ROLLBACK IMMEDIATE; DROP DATABASE "+Q(database)+";"); + created.Remove(database); + } + internal void ResetTarget(){DropOwned(Target);CreateOwned(Target);} + internal static string Q(string identifier){return "["+identifier.Replace("]","]]")+"]";} + internal static string L(string value){return "N'"+value.Replace("'","''")+"'";} + internal static void Exec(SqlConnection connection,string sql){using(var command=new SqlCommand(sql,connection){CommandTimeout=60})command.ExecuteNonQuery();} + internal static object Scalar(SqlConnection connection,string sql){using(var command=new SqlCommand(sql,connection){CommandTimeout=60})return command.ExecuteScalar();} + internal string WriteScript(string name,string text){string path=Path.Combine(Root,name);File.WriteAllText(path,text.Replace("\r\n","\n").Replace("\n","\r\n"),new UTF8Encoding(false));return path;} + internal void Setup(string smallFixture) + { + string fixture=Path.Combine(Environment.GetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_FIXTURES"),smallFixture); + Assert(new FileInfo(fixture).Length<1024*1024,"Only small tracked fixture source may be read wholly."); + string sql=File.ReadAllText(fixture);using(var connection=Open(Source))foreach(string batch in System.Text.RegularExpressions.Regex.Split(sql,@"(?im)^GO\s*\r?$"))if(!string.IsNullOrWhiteSpace(batch))Exec(connection,batch); + } + internal void Case(string caseId,Action assertion) + { + var clock=Stopwatch.StartNew();string status="Failed"; + try{assertion();status="Passed";} + finally { + string path=Path.Combine(Root,caseId+".json");File.WriteAllText(path,JsonConvert.SerializeObject(new{schemaVersion=1,caseId,status,executed=true,durationMs=clock.ElapsedMilliseconds},Formatting.Indented)); + string hash;using(var sha=SHA256.Create())using(var stream=File.OpenRead(path))hash=BitConverter.ToString(sha.ComputeHash(stream)).Replace("-","").ToLowerInvariant(); + Receipts[caseId]=new{caseId,status,executed=true,assertionsPassed=status=="Passed"?1:0,assertionsFailed=status=="Passed"?0:1,receiptSha256=hash}; + } + } + public void Dispose() + { + if(disposed)return;disposed=true; + try{foreach(string db in created.ToArray())DropOwned(db);CleanupStatus="Confirmed";} + finally{File.WriteAllText(Path.Combine(Root,"cleanup.json"),JsonConvert.SerializeObject(new{status=CleanupStatus,remaining=created.ToArray()},Formatting.Indented));} + } + } private static void MssqlImportPublishedReleaseRace() + { + using(var startup=new ManualResetEventSlim())using(var providerRelease=new ManualResetEventSlim()) + using(var ownerInDispose=new ManualResetEventSlim())using(var ownerReturn=new ManualResetEventSlim())using(var cancel=new CancellationTokenSource()) { + var owner=new ImportSessionOwner { BeforeReturn=()=>{ownerInDispose.Set();Assert(ownerReturn.Wait(3000),"Controlled owner-return gate releases.");} }; + var command=new ImportSessionCommand(new ImportSessionReader(new[]{0},true)); + command.Start=(behavior,token)=>{startup.Set();Assert(providerRelease.Wait(3000),"Controlled provider gate releases.");return Task.FromResult(command.Reader);}; + using(var session=new MssqlImportSqlSession(owner,()=>command,()=>true,40)) { + var worker=Task.Run(()=>{try{session.ExecuteBatch(ImportSessionBatch,cancel.Token);}catch(OperationCanceledException){}}); + try { + Assert(startup.Wait(1000),"Provider actual seam entered before cancellation.");cancel.Cancel(); + Assert(ownerInDispose.Wait(2000),"Fake owner signals while still inside Dispose, before adapter publication."); + var pending=session.Cleanup();Assert(!pending.CommandEnded&&!pending.SessionReleased&&!pending.TransactionCleanupConfirmed,"Pre-return event is not published adapter release receipt."); + ownerReturn.Set();var published=ImportWaitForPublishedReleaseWhileActive(session); + Assert(published.SessionReleased&&!published.CommandEnded,"Bounded published-receipt wait removes the fake pre-return event race."); + }finally{ownerReturn.Set();providerRelease.Set();} + Assert(worker.Wait(2000),"Provider unwinds after gates release."); + } + } + } private static void MssqlImportIntegrationLive() + { + if(Environment.GetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_INTEGRATION")!="1") {Console.WriteLine("SKIP native import live acceptance (explicit runner required)");return;} + string acceptanceProfile=Environment.GetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_PROFILE"); + if(!ImportProfiles.Contains(acceptanceProfile))throw new InvalidOperationException("UnknownImportAcceptanceProfile"); + ImportLiveFixture fixture=null;bool complete=false; + try { + fixture=new ImportLiveFixture(acceptanceProfile); + switch(acceptanceProfile) { + case "Core":ImportLiveCore(fixture);break; + case "Lob":ImportLiveLob(fixture);break; + case "Geometry":ImportLiveGeometry(fixture);break; + case "Faults":ImportLiveFaults(fixture);break; + case "Memory": + string scale=Environment.GetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_MEMORY_SCALE"); + if(scale=="1"||scale=="10")ImportLiveMemoryChild(fixture,int.Parse(scale));else ImportLiveMemory(fixture); + break; + default:throw new NotImplementedException("Actual profile not yet wired: "+acceptanceProfile); + } + complete=true; + } finally { + if(fixture!=null) { + try{fixture.Dispose();} + finally { + string exe=typeof(SmokeTests).Assembly.Location;string hash;using(var sha=SHA256.Create())using(var file=File.OpenRead(exe))hash=BitConverter.ToString(sha.ComputeHash(file)).Replace("-","").ToLowerInvariant(); + File.WriteAllText(Path.Combine(fixture.Root,"profile-receipt.json"),JsonConvert.SerializeObject(new{schemaVersion=1,profile=acceptanceProfile, + runId=Environment.GetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_RUN_ID"),complete,processBits=IntPtr.Size*8,executableSha256=hash, + cases=fixture.Receipts.Values.ToArray(),cleanupStatus=fixture.CleanupStatus},Formatting.Indented)); + } + } + } + } + private static void ImportLiveCore(ImportLiveFixture f) + { + using(var c=f.Open(f.Target)){ + var observations=new Dictionary(); + foreach(var query in new[]{new[]{"alone","SELECT @@TRANCOUNT,XACT_STATE();"},new[]{"db-id-first","SELECT DB_ID(),@@TRANCOUNT,XACT_STATE();"},new[]{"db-id-captured","DECLARE @db int=DB_ID(); SELECT @db,@@TRANCOUNT,XACT_STATE();"},new[]{"xact-captured","DECLARE @x int=XACT_STATE(); SELECT DB_ID(),@@TRANCOUNT,@x;"},new[]{"options","SELECT @@OPTIONS;"}}) + using(var cmd=new SqlCommand(query[1],c))using(var reader=cmd.ExecuteReader()){Assert(reader.Read(),"State diagnostic row.");var values=new int[reader.FieldCount];for(int i=0;i{ + full=ImportDraftExportNative(f.Server,f.Source,f.Databases,f.Root,"native-full.sql",MssqlExportMode.SchemaAndData); + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,full.TargetPath,f.Root); + ImportRecordResult(f,"full",full.TargetPath,result); + Assert(result.Status==MssqlImportStatus.Completed&&result.ValidationSummary.Contains("NativeSummaryCountsMatched"),"Actual native importer EOF/table/row verification completed."); + using(var source=f.Open(f.Source))using(var target=f.Open(f.Target))ImportDraftCompareSmallCatalogs(source,target); + ImportCompareFixtureRows(f,full); + }); + f.Case("core.catalog-identities-indexes",()=>{using(var c=f.Open(f.Target)){ + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(c,"INSERT dbo.IdentityEmpty(v) VALUES(9);SELECT SCOPE_IDENTITY();"))==11,"Empty identity seed preserved."); + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(c,"INSERT dbo.IdentityNegative(v) VALUES(9);SELECT SCOPE_IDENTITY();"))==-9,"Descending identity next preserved."); + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(c,"SELECT COUNT(*) FROM sys.indexes WHERE name=N'IX_filtered' AND has_filter=1;"))==1,"Filtered included index exists."); + }}); + f.Case("core.catalog-fk-check-trust",()=>{using(var c=f.Open(f.Target)){ + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(c,"SELECT COUNT(*) FROM sys.foreign_keys WHERE name=N'FK_Untrusted' AND is_not_trusted=1;"))==1,"Untrusted FK stays untrusted."); + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(c,"SELECT COUNT(*) FROM sys.foreign_keys WHERE name=N'FK_same' AND is_not_trusted=0;"))==1,"Trusted FK stays trusted."); + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(c,"SELECT COUNT(*) FROM sys.foreign_key_columns WHERE constraint_object_id=OBJECT_ID(N'dbo.FK_Composite');"))==2,"Actual native composite FK preserves both column ordinals."); + }}); + f.Case("core.catalog-views-properties",()=>{using(var a=f.Open(f.Source))using(var b=f.Open(f.Target)){ + const string sql="SELECT OBJECT_SCHEMA_NAME(major_id),OBJECT_NAME(major_id),minor_id,name,CONVERT(nvarchar(max),value) FROM sys.extended_properties WHERE class=1"; + Assert(IntegrationCatalog(a,sql)==IntegrationCatalog(b,sql),"Table/column properties independently match."); + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(b,"SELECT COUNT(*) FROM [一].[view]]中文]"))==3,"Native view executes actual restored rows."); + }}); + f.ResetTarget(); + f.Case("core.native-schema-only",()=>{ + var request=ImportDraftExportNative(f.Server,f.Source,f.Databases,f.Root,"native-schema.sql",MssqlExportMode.SchemaOnly); + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,request.TargetPath,f.Root);ImportRecordResult(f,"schema",request.TargetPath,result); + Assert(result.Status==MssqlImportStatus.Completed,"Actual SchemaOnly native importer completes."); + using(var target=f.Open(f.Target))Assert(Convert.ToInt64(ImportLiveFixture.Scalar(target,"SELECT COUNT_BIG(*) FROM dbo.MultiLob"))==0,"SchemaOnly creates empty native table."); + }); + f.Case("core.native-data-only",()=>{ + var request=ImportDraftExportNative(f.Server,f.Source,f.Databases,f.Root,"native-data.sql",MssqlExportMode.DataOnly,views:false); + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,request.TargetPath,f.Root);ImportRecordResult(f,"data",request.TargetPath,result); + Assert(result.Status==MssqlImportStatus.Completed&&!result.ValidationSummary.Contains("NativeSummaryCountsMatched"),"DataOnly completes without pretending full schema/content verification."); + ImportCompareFixtureRows(f,request); + }); + f.Case("core.generic-session-temp-set",()=>ImportDraftRunGenericSessionCase(f.Server,f.Target,f.Databases,f.Root)); + f.Case("core.generic-identity-insert",()=>{using(var c=f.Open(f.Target))Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c,"SELECT COUNT_BIG(*) FROM dbo.ImportSession WHERE id IN(37,39,41)"))==3,"Actual generic IDENTITY_INSERT continuity result.");}); + f.Case("core.generic-cross-go-transaction",()=>{using(var c=f.Open(f.Target))Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c,"SELECT SUM(CONVERT(bigint,v)) FROM dbo.ImportSession"))==31,"Actual committed cross-GO transaction visible on independent session.");}); + f.Case("core.generic-multiple-results",()=>{ + string script=f.WriteScript("multi-results.sql","SELECT 1;SELECT 2 WHERE 1=0;SELECT 3;\r\nGO\r\nINSERT dbo.ImportSession(v) VALUES(17);\r\nGO\r\n"); + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,script,f.Root);ImportRecordResult(f,"multi",script,result); + Assert(result.Status==MssqlImportStatus.Completed&&result.SuccessfulBatches==2,"All result sets drained before next GO acknowledgement."); + }); + f.Case("core.parser-failure-no-write",()=>{ + string path=f.WriteScript("parser-invalid.sql","CREATE TABLE dbo.NeverParser(id int);\r\nGO 2\r\n");int opens=0; + var service=new MssqlImportService(()=>{opens++;return new MssqlImportSqlSession(ImportDraftOpenOwned(f.Server,f.Target,f.Databases));},job=>new MssqlImportLog(job,Path.Combine(f.Root,"logs"))); + using(var pre=service.Preflight(new MssqlImportRequest(Guid.NewGuid(),path,f.Target),null,CancellationToken.None))Assert(!pre.Passed&&opens==0,"Actual parser rejection precedes any metadata/execution connection."); + using(var c=f.Open(f.Target))Assert(ImportLiveFixture.Scalar(c,"SELECT OBJECT_ID(N'dbo.NeverParser')")==DBNull.Value,"Parser failure did not create target object."); + }); + f.Case("core.target-identity-recheck",()=>{ + string path=f.WriteScript("identity-recheck.sql","CREATE TABLE dbo.NeverRecreated(id int);\r\nGO\r\n"); + var service=new MssqlImportService(()=>new MssqlImportSqlSession(ImportDraftOpenOwned(f.Server,f.Target,f.Databases)),job=>new MssqlImportLog(job,Path.Combine(f.Root,"logs"))); + var request=new MssqlImportRequest(Guid.NewGuid(),path,f.Target); + using(var pre=service.Preflight(request,null,CancellationToken.None)){ + Assert(pre.Passed,"Actual identity preflight.");f.ResetTarget();var result=service.Run(request,pre,null,CancellationToken.None);ImportRecordResult(f,"recreated",path,result); + Assert(result.Status==MssqlImportStatus.Failed&&result.SuccessfulBatches==0,"Recreated owned target rejected before execution."); + } + }); + } + private static void ImportCompareFixtureRows(ImportLiveFixture f,MssqlExportRequest request) + { + using(var source=f.Open(f.Source))using(var target=f.Open(f.Target))foreach(var table in request.Tables){ + string qualified=ImportLiveFixture.Q(table.Schema)+"."+ImportLiveFixture.Q(table.Name);var expressions=new List(); + using(var command=new SqlCommand("SELECT c.name,ty.name FROM sys.columns c JOIN sys.types ty ON c.user_type_id=ty.user_type_id WHERE c.object_id=OBJECT_ID("+ImportLiveFixture.L(qualified)+") ORDER BY column_id",source))using(var reader=command.ExecuteReader())while(reader.Read()){ + string name=ImportLiveFixture.Q(reader.GetString(0)),type=reader.GetString(1);if(type=="timestamp")continue; + if(type=="geometry")name+=".Serialize()";if(type=="text")name="CONVERT(varchar(max),"+name+")";if(type=="ntext"||type=="xml")name="CONVERT(nvarchar(max),"+name+")";expressions.Add("CONVERT(varbinary(max),"+name+")"); + } + string query="SELECT "+string.Join(",",expressions)+" FROM "+qualified; + Assert(IntegrationRowHashes(source,query).SequenceEqual(IntegrationRowHashes(target,query)),"Actual fixture raw bytes/NULL/length match "+qualified); + } + } + private static void ImportRecordResult(ImportLiveFixture f,string key,string script,MssqlImportResult result) + { + string hash;using(var sha=SHA256.Create())using(var file=File.OpenRead(script))hash=BitConverter.ToString(sha.ComputeHash(file)).Replace("-","").ToLowerInvariant(); + File.WriteAllText(Path.Combine(f.Root,"result-"+key+".json"),JsonConvert.SerializeObject(new{script=Path.GetFileName(script),bytes=new FileInfo(script).Length,sha256=hash, + result.Status,result.FileBytes,result.ProcessedBytes,result.SuccessfulBatches,result.FailedBatches,result.LastAcknowledgedBatch,result.UncertainBatch,result.LogPath,result.ValidationSummary,result.Elapsed, + cleanup=result.Cleanup==null?null:new{result.Cleanup.CommandEnded,result.Cleanup.SessionReleased,result.Cleanup.TransactionCleanupConfirmed,result.Cleanup.IssueCode}},Formatting.Indented)); + } private sealed class ImportObservedSession : IMssqlImportSession + { + private readonly IMssqlImportSession inner;private readonly string path;private readonly Action targetObserved;private readonly Queue states=new Queue();private long batch;private Guid job; + internal ImportObservedSession(IMssqlImportSession session,string tracePath,Action targetObserved=null){inner=session;path=tracePath;this.targetObserved=targetObserved;} + public MssqlImportTarget ReadTarget(CancellationToken token){var target=inner.ReadTarget(token);if(targetObserved!=null)targetObserved(target);return target;} + public MssqlImportSessionState ReadState(CancellationToken token){var state=inner.ReadState(token);states.Enqueue(batch+","+state.DatabaseId+","+state.TransactionCount+","+state.XactState+","+state.IsOpen);while(states.Count>64)states.Dequeue();return state;} + public MssqlImportLobState ReadLobState(Guid identity,CancellationToken token){var state=inner.ReadLobState(identity,token);File.WriteAllText(path+".lob.json",JsonConvert.SerializeObject(new{batch,state.Exists,state.RowCount,state.Sequence,state.ConsumedThrough,state.Consumed,state.Failed}));return state;} + public void ExecuteBatch(MssqlImportBatch value,CancellationToken token){batch=value.Number;if(job==Guid.Empty){var match=System.Text.RegularExpressions.Regex.Match(value.Text,@"#mp_([a-fA-F0-9]{32})_s");if(match.Success)job=Guid.ParseExact(match.Groups[1].Value,"N");}inner.ExecuteBatch(value,token);} + public MssqlImportCounts CountTargetRows(Action progress,CancellationToken token){return inner.CountTargetRows(progress,token);} + public void CancelActiveCommand(){inner.CancelActiveCommand();} + public MssqlImportCleanup Cleanup(){if(job!=Guid.Empty)try{var current=ReadState(CancellationToken.None);if(current.IsOpen)ReadLobState(job,CancellationToken.None);}catch{}return inner.Cleanup();} + public void Dispose(){try{inner.Dispose();}finally{File.WriteAllLines(path,new[]{"batch,databaseId,transactionCount,xactState,isOpen"}.Concat(states),new UTF8Encoding(false));}} + } private static void ImportLiveLob(ImportLiveFixture f) + { + using(var c=f.Open(f.Source))ImportLiveFixture.Exec(c,ImportDraftLobSetup); + MssqlExportRequest full=null; + f.Case("lob.binary-64m-roundtrip",()=>{ + full=ImportDraftExportNative(f.Server,f.Source,f.Databases,f.Root,"lob-full.sql",MssqlExportMode.SchemaAndData,views:false); + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,full.TargetPath,f.Root);ImportRecordResult(f,"lob-full",full.TargetPath,result); + Assert(result.Status==MssqlImportStatus.Completed,"Actual 64MiB binary native import completed."); + using(var source=f.Open(f.Source))using(var target=f.Open(f.Target))Assert(ImportDraftCompareOrderedBytes(source,target,"SELECT CONVERT(varbinary(max),id),b FROM dbo.ImportLob ORDER BY id")==3,"64MiB binary bytes stream equal."); + }); + f.Case("lob.unicode-18m-roundtrip",()=>{using(var a=f.Open(f.Source))using(var b=f.Open(f.Target)){ + Assert(Convert.ToInt64(ImportLiveFixture.Scalar(b,"SELECT DATALENGTH(n) FROM dbo.ImportLob WHERE id=3"))==18L*1024*1024,"18MiB Unicode length including surrogate pair."); + Assert(ImportDraftCompareOrderedBytes(a,b,"SELECT CONVERT(varbinary(max),id),CONVERT(varbinary(max),n) FROM dbo.ImportLob ORDER BY id")==3,"18MiB UTF16 raw bytes match."); + }}); + f.Case("lob.null-empty-xml-roundtrip",()=>{using(var a=f.Open(f.Source))using(var b=f.Open(f.Target)){ + Assert(ImportDraftCompareOrderedBytes(a,b,"SELECT CONVERT(varbinary(max),id),b,CONVERT(varbinary(max),n),CONVERT(varbinary(max),CONVERT(nvarchar(max),x)) FROM dbo.ImportLob ORDER BY id")==3,"NULL/empty/XML raw-byte projections match."); + Assert(Convert.ToInt64(ImportLiveFixture.Scalar(b,"SELECT COUNT_BIG(*) FROM dbo.ImportLob WHERE (id=1 AND b IS NULL AND n IS NULL AND x IS NULL) OR(id=2 AND DATALENGTH(b)=0 AND DATALENGTH(n)=0)"))==2,"NULL remains distinct from empty."); + }}); + f.Case("lob.native-data-only",()=>{ + using(var c=f.Open(f.Target)){ImportLiveFixture.Exec(c,"TRUNCATE TABLE dbo.ImportLob;TRUNCATE TABLE dbo.ImportLobGuard;");} + var request=ImportDraftExportNative(f.Server,f.Source,f.Databases,f.Root,"lob-data.sql",MssqlExportMode.DataOnly,views:false); + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,request.TargetPath,f.Root);ImportRecordResult(f,"lob-data",request.TargetPath,result); + Assert(result.Status==MssqlImportStatus.Completed&&!result.ValidationSummary.Contains("NativeSummaryCountsMatched"),"Actual native DataOnly verifies protocol but does not claim full schema manifest validation."); + using(var a=f.Open(f.Source))using(var b=f.Open(f.Target))Assert(ImportDraftCompareOrderedBytes(a,b,"SELECT CONVERT(varbinary(max),id),b,CONVERT(varbinary(max),n) FROM dbo.ImportLob ORDER BY id")==3,"DataOnly large byte streams equal."); + }); + var guard=ImportDraftExportNative(f.Server,f.Source,f.Databases,f.Root,"lob-guard.sql",MssqlExportMode.DataOnly,t=>t.Key.Name=="ImportLobGuard",false); + foreach(var fault in new[]{"missing","duplicate","reordered","replayed","constraint","xml","missing-state","duplicate-state","pending-eof","sequence"}){ + string id= fault=="missing"?"lob.chunk-missing":fault=="duplicate"?"lob.chunk-duplicate":fault=="reordered"?"lob.chunk-reordered":fault=="replayed"?"lob.final-replayed":fault=="constraint"?"lob.final-constraint-failed0tran":fault=="xml"?"lob.xml-conversion-failed0tran":fault=="missing-state"?"lob.state-missing-after-init":fault=="duplicate-state"?"lob.state-duplicate":fault=="pending-eof"?"lob.pending-eof":"lob.sequence-eof"; + f.Case(id,()=>ImportGuardFault(f,guard.TargetPath,fault,"ImportLobGuard",false)); + } + } + private static void ImportGuardFault(ImportLiveFixture f,string source,string fault,string table,bool geometry) + { + string altered=Path.Combine(f.Root,table+"-"+fault+".sql"); + if(new[]{"missing","duplicate","reordered","replayed","deserialize"}.Contains(fault))ImportDraftMutateGuardArtifact(source,altered,fault); + else if(fault=="constraint")File.Copy(source,altered); + else ImportMutateProtocolState(source,altered,fault,table); + foreach(bool skip in new[]{false,true}){ + f.ResetTarget(); + using(var target=f.Open(f.Target))ImportLiveFixture.Exec(target,"CREATE TABLE dbo."+table+"(id int NOT NULL PRIMARY KEY,"+(geometry?"g geometry NULL":"b varbinary(max),x xml")+(fault=="constraint"?",CONSTRAINT CK_reject CHECK(id<>1)":"")+");"); + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,altered,f.Root,skip);ImportRecordResult(f,table+"-"+fault+"-"+skip,altered,result); + Assert(result.Status==MssqlImportStatus.Failed,"Actual native protocol failure cannot be skipped: "+fault+" skip="+skip); + using(var target=f.Open(f.Target)){ + long count=Convert.ToInt64(ImportLiveFixture.Scalar(target,"SELECT COUNT_BIG(*) FROM dbo."+table)); + long next=Convert.ToInt64(ImportLiveFixture.Scalar(target,"SELECT COUNT_BIG(*) FROM dbo."+table+" WHERE id=2")); + Assert(next==0&&count==(fault=="replayed"||fault=="sequence"?1:0),"Failed native row/next-row boundary preserved: "+fault); + } + } + } + private static void ImportMutateProtocolState(string source,string destination,string fault,string table) + { + using(var input=MssqlImportInput.Open(source)){ + var scan=input.Scan(CancellationToken.None,null);string state="#mp_"+scan.Script.NativeJobId.Value.ToString("N")+"_s";bool applied=false,stopBody=false; + using(var writer=new StreamWriter(new FileStream(destination,FileMode.CreateNew,FileAccess.Write,FileShare.None,65536),new UTF8Encoding(false),65536)){ + foreach(var batch in input.ReadBatches(CancellationToken.None)){ + string text=batch.Text; + if(stopBody)continue; + if(fault=="xml"&&text.Contains("3C006F006B002F003E00")){text=text.Replace("3C006F006B002F003E00","3C006200610064003E00");applied=true;} + writer.Write(text);writer.Write("\r\nGO\r\n"); + if(!applied&&text.Contains("LOB job already exists")&&(fault=="missing-state"||fault=="duplicate-state")){ + writer.Write(fault=="missing-state"?"DROP TABLE "+state+";":"INSERT INTO "+state+" SELECT * FROM "+state+";");writer.Write("\r\nGO\r\n");applied=true; + } + if(!applied&&fault=="pending-eof"&&text.Contains("[Consumed]=0")&&text.Contains("CREATE TABLE #mp_")){applied=true;stopBody=true;} + if(!applied&&fault=="sequence"&&text.Contains("INSERT INTO [dbo].["+table+"]")){ + writer.Write("UPDATE "+state+" SET [Sequence]=[Sequence]+1;\r\nGO\r\n");applied=true; + } + } + Assert(applied,"Actual tiny exporter protocol mutation applied: "+fault); + writer.Write("-- mysqlpunk-export-complete job="+scan.Script.NativeJobId.Value.ToString("N")+" tables="+scan.Script.ExpectedTables+" rows="+scan.Script.ExpectedRows+"\r\n"); + } + } + } private static void ImportLiveGeometry(ImportLiveFixture f) + { + f.Setup("geometry.sql"); + using(var c=f.Open(f.Source)){ + ImportDraftGenerateGeometry(c,Path.Combine(f.Root,"geometry-large.wkb"),"dbo.GeometryLarge",1100001); + ImportDraftGenerateGeometry(c,Path.Combine(f.Root,"geometry-guard.wkb"),"dbo.GeometryGuard",20001); + ImportLiveFixture.Exec(c,"INSERT dbo.GeometryGuard(id,g) VALUES(2,geometry::Point(3,4,4326));"); + } + MssqlExportRequest full=null; + f.Case("geometry.bytes-metadata-roundtrip",()=>{ + full=ImportDraftExportNative(f.Server,f.Source,f.Databases,f.Root,"geometry-full.sql",MssqlExportMode.SchemaAndData,views:false); + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,full.TargetPath,f.Root);ImportRecordResult(f,"geometry-full",full.TargetPath,result); + Assert(result.Status==MssqlImportStatus.Completed,"Actual native Geometry full importer completes.");ImportCompareFixtureRows(f,full); + using(var source=f.Open(f.Source))using(var target=f.Open(f.Target)){ + ImportDraftCompareSmallCatalogs(source,target); + const string sql="SELECT id,g.STSrid AS srid,g.STGeometryType() AS kind,g.STIsEmpty() AS empty,g.STIsValid() AS valid,g.STNumPoints() AS points,g.STPointN(1).Z AS z,g.STPointN(1).M AS m,h.STSrid AS secondSrid,h.STGeometryType() AS secondKind,h.STIsEmpty() AS secondEmpty FROM dbo.GeometryCases WHERE id<>8"; + Assert(IntegrationCatalog(source,sql)==IntegrationCatalog(target,sql),"Native SRID/type/Z/M/curves/empty metadata independently equal."); + } + }); + f.Case("geometry.large-serialize-roundtrip",()=>{using(var source=f.Open(f.Source))using(var target=f.Open(f.Target)){ + Assert(Convert.ToInt64(ImportLiveFixture.Scalar(target,"SELECT DATALENGTH(g.Serialize()) FROM dbo.GeometryLarge"))>16L*1024*1024,"Actual serialized Geometry exceeds16MiB."); + Assert(ImportDraftCompareOrderedBytes(source,target,"SELECT CONVERT(varbinary(max),id),g.Serialize() FROM dbo.GeometryLarge ORDER BY id")==1,"Large Geometry raw Serialize stream equal."); + }}); + f.Case("geometry.null-empty-invalid-curves",()=>{using(var target=f.Open(f.Target)){ + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(target,"SELECT COUNT(*) FROM dbo.GeometryCases WHERE id=1 AND g IS NULL AND h.STIsEmpty()=1"))==1,"NULL and valid empty remain distinct."); + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(target,"SELECT g.STIsValid() FROM dbo.GeometryCases WHERE id=8"))==0,"Invalid-but-storable geometry never normalized or repaired."); + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(target,"SELECT COUNT(*) FROM dbo.GeometryCases WHERE id=3 AND g.STSrid=3826 AND g.Z=3 AND g.M=4"))==1,"SRID/Z/M preserved."); + Assert(Convert.ToString(ImportLiveFixture.Scalar(target,"SELECT g.STGeometryType() FROM dbo.GeometryCases WHERE id=6"))=="CircularString","Curve remains native curve."); + }}); + f.Case("geometry.native-data-only",()=>{ + using(var target=f.Open(f.Target))ImportLiveFixture.Exec(target,"TRUNCATE TABLE dbo.GeometryCases;TRUNCATE TABLE dbo.GeometryLarge;TRUNCATE TABLE dbo.GeometryGuard;"); + var data=ImportDraftExportNative(f.Server,f.Source,f.Databases,f.Root,"geometry-data.sql",MssqlExportMode.DataOnly,views:false); + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,data.TargetPath,f.Root);ImportRecordResult(f,"geometry-data",data.TargetPath,result); + Assert(result.Status==MssqlImportStatus.Completed&&!result.ValidationSummary.Contains("NativeSummaryCountsMatched"),"Actual Geometry DataOnly protocol verified with limited summary claim.");ImportCompareFixtureRows(f,data); + }); + var guard=ImportDraftExportNative(f.Server,f.Source,f.Databases,f.Root,"geometry-guard.sql",MssqlExportMode.DataOnly,t=>t.Key.Name=="GeometryGuard",false); + foreach(string fault in new[]{"missing","duplicate","reordered","replayed","deserialize","constraint","pending-eof"}){ + string id=fault=="missing"?"geometry.chunk-missing":fault=="duplicate"?"geometry.chunk-duplicate":fault=="reordered"?"geometry.chunk-reordered":fault=="replayed"?"geometry.final-replayed":fault=="deserialize"?"geometry.deserialize-failed0tran":fault=="constraint"?"geometry.final-constraint-failed0tran":"geometry.pending-eof"; + f.Case(id,()=>ImportGuardFault(f,guard.TargetPath,fault,"GeometryGuard",true)); + } + } + private static void ImportLiveFaults(ImportLiveFixture f) + { + foreach (bool skip in new[] { false, true }) { + bool localSkip = skip; + f.Case("faults.partial-batch-" + (skip ? "skip" : "stop"), () => { + f.ResetTarget(); + string path = f.WriteScript("partial-" + localSkip + ".sql", ImportDraftGenericPartialFailure); + var result = ImportDraftRunActual(f.Server, f.Target, f.Databases, path, f.Root, localSkip); + ImportRecordResult(f, "partial-" + localSkip, path, result); + Assert(result.Status == (localSkip ? MssqlImportStatus.CompletedWithErrors : MssqlImportStatus.Failed), "Actual partial SQL error status follows requested stop/skip policy."); + Assert(result.FailedBatches == 1 && result.LastAcknowledgedBatch == (localSkip ? 3 : 1), "Only acknowledged successful/skipped GO boundaries advance the receipt."); + Assert(result.SkippedBytes > 0 == localSkip, "Skipped byte accounting follows safe skip decision."); + using (var c = f.Open(f.Target)) { + Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM dbo.ImportFault WHERE id=1")) == 1, "Autocommitted partial row survives the failed batch."); + Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM dbo.ImportFault WHERE id=2")) == (localSkip ? 1 : 0), "Independent connection proves next GO sentinel follows safe stop/skip."); + } + string log = File.ReadAllText(result.LogPath); + Assert(log.Contains("first runtime error") && log.Contains("second runtime error"), "Every real provider error from failed batch reaches the log."); + Assert(log.IndexOf("ErrorUtc=", StringComparison.Ordinal) < log.IndexOf("EndUtc=", StringComparison.Ordinal), "Error details precede terminal log record."); + ImportFaultClean(result, true); + }); + } + f.Case("faults.sqlerror-all-details", () => { + f.ResetTarget(); + string path = f.WriteScript("all-errors.sql", "CREATE TABLE dbo.ImportFault(id int PRIMARY KEY);\nGO\nRAISERROR(N'fixture detail alpha',16,17);\nRAISERROR(N'fixture detail beta',16,18);\nRAISERROR(N'fixture detail gamma',16,19);\nGO\nINSERT dbo.ImportFault VALUES(2);\nGO\n"); + var capture = new ImportFaultObservedLog(f); + var result = ImportDraftRunActual(f.Server, f.Target, f.Databases, path, f.Root, true, logFactory: job => capture.Open(job, f.Root)); + ImportRecordResult(f, "all-errors", path, result); + Assert(result.Status == MssqlImportStatus.CompletedWithErrors && capture.Details.Count == 3, "Three actual SqlErrors survive provider aggregation."); + string log = File.ReadAllText(result.LogPath); + foreach (var detail in capture.Details) { + Assert(detail.Number == 50000 && detail.Class == 16 && detail.RelativeLine > 0, "Real provider number/class/line retained."); + Assert(log.Contains("Number=" + detail.Number + " State=" + detail.State + " Class=" + detail.Class + " RelativeLine=" + detail.RelativeLine) && log.Contains(detail.Message), "Full log contains every delivered typed provider detail."); + } + Assert(capture.Details.Select(x => x.State).SequenceEqual(new[] { 17, 18, 19 }), "Distinct provider states preserve order."); + Assert(capture.ErrorDurableBeforeNext, "Error append and durable flush completed while independent next-GO row count was still zero."); + ImportFaultClean(result, true); + }); + foreach (bool abort in new[] { false, true }) { + bool localAbort = abort; + f.Case(abort ? "faults.xact-abort" : "faults.transaction-error", () => { + f.ResetTarget(); + string path = f.WriteScript("transaction-" + localAbort + ".sql", "CREATE TABLE dbo.ImportFault(id int PRIMARY KEY);\nGO\nSET XACT_ABORT " + (localAbort ? "ON" : "OFF") + "; BEGIN TRAN; INSERT dbo.ImportFault VALUES(1);\nGO\nBEGIN TRY\nINSERT dbo.ImportFault VALUES(1);\nEND TRY\nBEGIN CATCH\nDECLARE @actual int=XACT_STATE(); RAISERROR(N'actual caught XACT_STATE=%d',16,7,@actual);\nEND CATCH;\nGO\nINSERT dbo.ImportFault VALUES(2);COMMIT;\nGO\n"); + var result = ImportDraftRunActual(f.Server, f.Target, f.Databases, path, f.Root, true); + ImportRecordResult(f, "transaction-" + localAbort, path, result); + Assert(result.Status == MssqlImportStatus.Failed && result.LastAcknowledgedBatch == 2 && result.FailedBatches == 1 && result.SkippedBytes == 0, "Explicit transaction failure cannot be safely skipped, even with skip enabled."); + Assert(File.ReadAllText(result.LogPath).Contains("actual caught XACT_STATE=" + (localAbort ? "-1" : "1")), "Server TRY/CATCH records actual doomed/committable transaction state before batch unwind."); + using (var c = f.Open(f.Target)) Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM dbo.ImportFault")) == 0, "Owned cleanup rollback removes transaction row and prevents later GO."); + ImportFaultClean(result, true); + }); + } + f.Case("faults.unclosed-transaction-eof", () => { + f.ResetTarget(); + string path = f.WriteScript("transaction-eof.sql", "CREATE TABLE dbo.ImportFault(id int);\nGO\nBEGIN TRAN;INSERT dbo.ImportFault VALUES(1);\nGO\n"); + var result = ImportDraftRunActual(f.Server, f.Target, f.Databases, path, f.Root, true); + ImportRecordResult(f, "transaction-eof", path, result); + Assert(result.Status == MssqlImportStatus.Failed && result.ValidationSummary == "EofTransactionOpen", "Actual EOF gate refuses success with a live transaction."); + using (var c = f.Open(f.Target)) Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM dbo.ImportFault")) == 0, "EOF cleanup rolled back uncommitted target row."); + ImportFaultClean(result, true); + }); + f.Case("faults.finite-timeout", () => { + f.ResetTarget(); + string path = f.WriteScript("timeout.sql", ImportDraftTimeout); + using (var point = new ImportLiveCheckpoint("Timeout") { FiniteTimeoutSeconds = 1 }) { + var result = ImportDraftRunActual(f.Server, f.Target, f.Databases, path, f.Root, true, point: point); + ImportRecordResult(f, "timeout", path, result); + Assert(point.Issued.IsSet && result.Status == MssqlImportStatus.Failed && result.UncertainBatch == 2 && result.SkippedBytes == 0, "Finite actual SqlCommand timeout remains uncertain and cannot be skipped."); + Assert(File.ReadAllText(result.LogPath).Contains("Number=-2 "), "Actual provider timeout -2 is logged."); + using (var c = f.Open(f.Target)) Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM dbo.ImportTimeout")) == 0, "Timeout stops before source/next GO sentinel inserts."); + ImportFaultClean(result, true); + } + }); + f.Case("faults.owned-transport-loss", () => { + f.ResetTarget(); + string path = f.WriteScript("owned-transport.sql", ImportDraftTimeout); + using (var point = new ImportLiveCheckpoint("Timeout")) using (var cancellation = new CancellationTokenSource()) { + var context = new ImportFaultRunContext(f, point); + var service = context.Service(); + var request = new MssqlImportRequest(Guid.NewGuid(), path, f.Target, true); + using (var preflight = service.Preflight(request, null, CancellationToken.None)) { + Assert(preflight.Passed, "Transport case passes real preflight."); + var worker = Task.Run(() => service.Run(request, preflight, null, cancellation.Token)); + try { Assert(point.Issued.Wait(8000), "Actual WAITFOR SqlCommand starts before exact owner is closed."); context.WorkerSession.Dispose(); } + finally { if (!worker.Wait(15000)) { cancellation.Cancel(); Assert(worker.Wait(15000), "Owned transport worker must unwind after close/cancel."); } } + var result = worker.GetAwaiter().GetResult(); ImportRecordResult(f, "transport", path, result); + Assert(result.Status == MssqlImportStatus.Failed && result.UncertainBatch == 2 && result.LastAcknowledgedBatch == 1 && result.SkippedBytes == 0, "Actual owned connection loss never becomes safe skip/success."); + Assert(result.Cleanup.CommandEnded && result.Cleanup.SessionReleased && !result.Cleanup.TransactionCleanupConfirmed, "Closed session proves release, while rollback acknowledgement remains unavailable."); + using (var c = f.Open(f.Target)) Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM dbo.ImportTimeout")) == 0, "Independent connection sees no transport or later sentinel row."); + } + } + }); + foreach (string mode in new[] { "create", "start", "append", "flush", "end" }) { + string localMode = mode; + f.Case("faults.log-" + (mode == "append" ? "error-append" : mode), () => ImportFaultLogCase(f, localMode)); + } + f.Case("faults.cancel-result-read", () => { + f.ResetTarget(); + string path = f.WriteScript("cancel-read.sql", ImportDraftDrainingResults); + ImportFaultCancel(f, "result-read", path, "Drain", false, 4096); + f.ResetTarget(); + string lob = f.WriteScript("cancel-returned-lob.sql", "/*IMPORT_INTEGRATION_DRAIN*/ SELECT CONVERT(varbinary(max),REPLICATE(CONVERT(varchar(max),'z'),67108864));SELECT 2;\nGO\nCREATE TABLE dbo.NeverAfterCancel(id int);\nGO\n"); + ImportFaultCancel(f, "returned-lob", lob, "Drain", false, 2); + }); + f.Case("faults.cancel-next-result", () => { + f.ResetTarget(); + string path = f.WriteScript("cancel-next.sql", "/*IMPORT_INTEGRATION_DRAIN*/ SELECT 1; SELECT 2; SELECT 3;\nGO\nCREATE TABLE dbo.NeverAfterCancel(id int);\nGO\n"); + ImportFaultCancel(f, "next-result", path, "Drain", true, 1); + }); + f.Case("faults.cancel-state-probe", () => { + f.ResetTarget(); + string path = f.WriteScript("cancel-state.sql", "CREATE TABLE dbo.NeverAfterCancel(id int);\nGO\n"); + ImportFaultCancel(f, "state", path, "StateProbe", false, 1); + }); + using (var source = f.Open(f.Source)) ImportLiveFixture.Exec(source, "CREATE TABLE dbo.ImportFaultCount(id int PRIMARY KEY);INSERT dbo.ImportFaultCount VALUES(1);CREATE TABLE dbo.ImportFaultLob(id int PRIMARY KEY,b varbinary(max));INSERT dbo.ImportFaultLob VALUES(1,CONVERT(varbinary(max),REPLICATE(CONVERT(varchar(max),'x'),300000)));"); + var nativeCount = ImportDraftExportNative(f.Server, f.Source, f.Databases, f.Root, "fault-native-count.sql", MssqlExportMode.SchemaAndData, t => t.Key.Name == "ImportFaultCount", false); + var nativeLob = ImportDraftExportNative(f.Server, f.Source, f.Databases, f.Root, "fault-native-lob.sql", MssqlExportMode.SchemaAndData, t => t.Key.Name == "ImportFaultLob", false); + f.Case("faults.cancel-lob-probe", () => { f.ResetTarget(); ImportFaultCancel(f, "lob-probe", nativeLob.TargetPath, "LobProbe", false, 1); }); + f.Case("faults.cancel-count-big", () => { + f.ResetTarget(); + SqlConnection blocker = null; + using (var point = new ImportLiveCheckpoint("Count")) using (var cancellation = new CancellationTokenSource()) { + int blocked = 0; + var context = new ImportFaultRunContext(f, point); + var service = context.Service(); var request = new MssqlImportRequest(Guid.NewGuid(), nativeCount.TargetPath, f.Target); + using (var preflight = service.Preflight(request, null, CancellationToken.None)) { + Assert(preflight.Passed, "Count cancellation uses actual full native import preflight."); + Exception blockerFailure = null; + Action progress = p => { + context.Observe(p); + if (p.Stage == MssqlImportStage.Verifying && Interlocked.Exchange(ref blocked, 1) == 0) { + try { blocker = f.Open(f.Target); ImportLiveFixture.Exec(blocker, "BEGIN TRAN;SELECT TOP(1) id FROM dbo.ImportFaultCount WITH(TABLOCKX,HOLDLOCK);"); } + catch (Exception error) { blockerFailure = error; } + } + }; + var worker = Task.Run(() => service.Run(request, preflight, progress, cancellation.Token)); + try { + Assert(point.Issued.Wait(10000), "Actual COUNT_BIG provider method issued while independent owned TABLOCKX transaction holds target."); + Assert(blockerFailure == null && blocker != null && !point.ReaderReady.IsSet, "Count is genuinely blocked at SQL provider rather than a fake reader gate."); + cancellation.Cancel(); Assert(worker.Wait(15000), "Production 10s cancellation watchdog bounds blocked COUNT_BIG unwind."); + } finally { + cancellation.Cancel(); + if (blocker != null) { try { if (blocker.State == ConnectionState.Open) ImportLiveFixture.Exec(blocker, "IF @@TRANCOUNT>0 ROLLBACK;"); } finally { blocker.Dispose(); } } + Assert(worker.Wait(15000), "Count worker ended before fixture cleanup."); + } + var result = worker.GetAwaiter().GetResult(); ImportRecordResult(f, "count-cancel", nativeCount.TargetPath, result); + ImportFaultCancelled(result, context, point, true); + Assert(result.UncertainBatch == null, "Verification cancellation cannot invent an executing source batch."); + } + } + }); + f.Case("faults.cancel-close-race", () => { + f.ResetTarget(); + string path = f.WriteScript("cancel-close.sql", "/*IMPORT_INTEGRATION_DRAIN*/ SELECT 1;SELECT 2;\nGO\nCREATE TABLE dbo.NeverAfterCancel(id int);\nGO\n"); + using (var point = new ImportLiveCheckpoint("Drain") { HoldReaderDispose = true }) using (var cancellation = new CancellationTokenSource()) { + point.DisposeRelease.Reset(); point.OnRead = () => cancellation.Cancel(); + var context = new ImportFaultRunContext(f, point); var service = context.Service(); var request = new MssqlImportRequest(Guid.NewGuid(), path, f.Target); + using (var preflight = service.Preflight(request, null, CancellationToken.None)) { + Assert(preflight.Passed, "Close race passes actual preflight."); + var worker = Task.Run(() => service.Run(request, preflight, context.Observe, cancellation.Token)); + try { + Assert(point.DisposeEntered.Wait(8000), "Actual reader Dispose is held after real provider reader started and cancellation requested."); + Assert(context.Stopping != null, "Cancellation publishes Stopping before close."); + var pending = context.WorkerSession.Cleanup(); Assert(!pending.CommandEnded && !pending.TransactionCleanupConfirmed, "Held reader cannot claim command completion or rollback."); + context.WorkerSession.Dispose(); + pending = context.WorkerSession.Cleanup(); Assert(!pending.CommandEnded && pending.SessionReleased && !pending.TransactionCleanupConfirmed, "Exact owner close proves release only while actual reader remains held."); + } finally { point.DisposeRelease.Set(); cancellation.Cancel(); Assert(worker.Wait(15000), "Reader and worker unwind before final race receipt."); } + var result = worker.GetAwaiter().GetResult(); ImportRecordResult(f, "cancel-close", path, result); + ImportFaultCancelled(result, context, point, false); + Assert(result.Status == MssqlImportStatus.CleanupUncertain && !result.Cleanup.TransactionCleanupConfirmed, "Close race preserves missing rollback acknowledgement in final status."); + } + } + }); + f.Case("faults.cancel-success-race", () => { + f.ResetTarget(); + string path = f.WriteScript("success-race.sql", "CREATE TABLE dbo.ImportFault(id int);INSERT dbo.ImportFault VALUES(1);\nGO\n"); + using (var cancellation = new CancellationTokenSource()) { + int finishSeen = 0, stoppingSeen = 0; + var result = ImportDraftRunActual(f.Server, f.Target, f.Databases, path, f.Root, cancellation: cancellation.Token, progress: p => { + if (p.Stage == MssqlImportStage.Stopping) Interlocked.Increment(ref stoppingSeen); + if (p.Stage == MssqlImportStage.Finished) { Interlocked.Increment(ref finishSeen); cancellation.Cancel(); } + }); + ImportRecordResult(f, "success-race", path, result); + Assert(finishSeen == 1 && stoppingSeen == 0 && cancellation.IsCancellationRequested && result.Status == MssqlImportStatus.Completed, "Cancellation arriving after terminal publication cannot rewrite completed truth or publish later Stopping."); + ImportFaultClean(result, true); + using (var c = f.Open(f.Target)) Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM dbo.ImportFault")) == 1, "Independent committed success receipt."); + } + }); + f.Case("faults.metadata-low-privilege-boundary", () => ImportFaultPermissions(f, nativeCount.TargetPath)); + f.Case("faults.target-unavailable-boundary", () => { + f.ResetTarget(); + string path = f.WriteScript("unavailable.sql", "CREATE TABLE dbo.NeverAfterCancel(id int);\nGO\n"); + int targetOpens=0; + Action masterState=f.RecordTargetProperties; + var service=new MssqlImportService(()=>{ + int ordinal=++targetOpens;var owned=ImportDraftOpenOwned(f.Server,f.Target,f.Databases); + try { + if(ordinal<=3)masterState("factory-open-"+ordinal); + return new ImportObservedSession(new MssqlImportSqlSession(owned),Path.Combine(f.Root,"target-trace-"+ordinal+".csv"),target=>{ + if(ordinal<=3)File.WriteAllText(Path.Combine(f.Root,"target-sut-"+ordinal+".json"),JsonConvert.SerializeObject(new{ordinal,target.ServerName,target.DatabaseName,target.DatabaseId,target.DatabaseCreateDate,target.Writable},Formatting.Indented)); + }); + }catch{owned.Dispose();throw;} + },job=>new MssqlImportLog(job,Path.Combine(f.Root,"logs"))); + var request = new MssqlImportRequest(Guid.NewGuid(), path, f.Target); + try { + f.SetTargetReadOnly(true); + using(var denied=service.Preflight(request,null,CancellationToken.None)) { + Assert(!denied.Passed&&denied.Target!=null&&!denied.Target.Writable&&denied.Issues.Any(x=>x.Code=="TargetReadOnly"&&!x.IsWarning),"Actual readonly target rejects preflight without source SQL."); + using(var c=f.Open(f.Target))Assert(ImportLiveFixture.Scalar(c,"SELECT OBJECT_ID(N'dbo.NeverAfterCancel')")==DBNull.Value,"Readonly preflight makes zero source writes."); + File.WriteAllText(Path.Combine(f.Root,"target-readonly-preflight.json"),JsonConvert.SerializeObject(new{passed=denied.Passed,writable=denied.Target.Writable,issueCodes=denied.Issues.Select(x=>x.Code).ToArray(),sourceWrites=0},Formatting.Indented)); + } + }finally{f.SetTargetReadOnly(false);masterState("restored-"+targetOpens);} + using(var checkedTarget=service.Preflight(request,null,CancellationToken.None)) { + Assert(checkedTarget.Passed&&checkedTarget.Target.Writable,"Read/write target passes before readonly transition."); + try { + f.SetTargetReadOnly(true); + using(var readonlyProbe=f.Open(f.Target)) { + Assert(Convert.ToString(ImportLiveFixture.Scalar(readonlyProbe,"SELECT DATABASEPROPERTYEX(DB_NAME(),'Updateability')"))=="READ_ONLY","Fixture actual updateability transition completed."); + masterState("probe-open"); + masterState("before-run"); + var rejected=service.Run(request,checkedTarget,null,CancellationToken.None);ImportRecordResult(f,"target-readonly-after-preflight",path,rejected); + masterState("after-run"); + using(var c=f.Open(f.Target)) { + bool sourceExists=ImportLiveFixture.Scalar(c,"SELECT OBJECT_ID(N'dbo.NeverAfterCancel')")!=DBNull.Value; + bool directCreated=false;var directErrors=new List(); + try{ImportLiveFixture.Exec(c,"CREATE TABLE dbo.NeverDirectReadonly(id int);");directCreated=true;} + catch(SqlException error){foreach(SqlError detail in error.Errors)directErrors.Add(detail.Number);} + File.WriteAllText(Path.Combine(f.Root,"target-readonly-effects.json"),JsonConvert.SerializeObject(new{sourceExists,directCreated,directErrors},Formatting.Indented)); + Assert(!directCreated&&directErrors.Contains(3906),"Fresh independent SqlConnection rejects equivalent CREATE in actual readonly target."); + } + Assert(rejected.Status==MssqlImportStatus.Failed&&rejected.ValidationSummary=="TargetReadOnly"&&rejected.ProcessedBytes==0&&rejected.SuccessfulBatches==0&&rejected.FailedBatches==0,"Actual run rechecks readonly target before source execution."); + ImportFaultClean(rejected,true); + using(var c=f.Open(f.Target))Assert(ImportLiveFixture.Scalar(c,"SELECT OBJECT_ID(N'dbo.NeverAfterCancel')")==DBNull.Value,"Readonly execution gate makes zero source writes."); + } + }finally{try{masterState("probe-closed");}finally{f.SetTargetReadOnly(false);masterState("restored-"+targetOpens);}} + } + using(var c=f.Open(f.Target))Assert(Convert.ToString(ImportLiveFixture.Scalar(c,"SELECT DATABASEPROPERTYEX(DB_NAME(),'Updateability')"))=="READ_WRITE","Finally restores actual read/write before recreation/cleanup."); + using (var preflight = service.Preflight(request, null, CancellationToken.None)) { + Assert(preflight.Passed, "Unavailable case initially passes actual preflight."); + // Guarded drop/recreate changes the exact owned target lifetime; no user DB/instance is touched. + f.ResetTarget(); + var result = service.Run(request, preflight, null, CancellationToken.None); ImportRecordResult(f, "target-recreated", path, result); + Assert(result.Status == MssqlImportStatus.Failed && result.ValidationSummary == "TargetChanged" && result.SuccessfulBatches == 0, "Recreated target is rejected before any source SQL executes."); + ImportFaultClean(result, true); + using (var c = f.Open(f.Target)) Assert(ImportLiveFixture.Scalar(c, "SELECT OBJECT_ID(N'dbo.NeverAfterCancel')") == DBNull.Value, "New target has no source write."); + } + }); + } + + private static void ImportFaultClean(MssqlImportResult result, bool transactionConfirmed) + { + Assert(result.Cleanup != null && result.Cleanup.CommandEnded && result.Cleanup.SessionReleased && result.Cleanup.TransactionCleanupConfirmed == transactionConfirmed, "Separate actual command/session/transaction receipts match asserted boundary."); + } + private static void ImportFaultCancelled(MssqlImportResult result, ImportFaultRunContext context, ImportLiveCheckpoint point, bool transactionConfirmed) + { + Assert(point.Issued.IsSet && point.CancelCalls > 0, "Cancellation reached the exact actual provider command."); + Assert(context.Stopping != null && context.Finished != null && context.Finished.Stage == MssqlImportStage.Finished, "Stopping and terminal progress are observed independently of command cancellation."); + Assert(result.Status == (transactionConfirmed ? MssqlImportStatus.Cancelled : MssqlImportStatus.CleanupUncertain), "Final cancellation status follows actual cleanup acknowledgement."); + ImportFaultClean(result, transactionConfirmed); + } + private static void ImportFaultCancel(ImportLiveFixture f, string key, string path, string phase, bool next, int readAt) + { + using (var point = new ImportLiveCheckpoint(phase) { ReadCheckpointAt = readAt }) using (var cancellation = new CancellationTokenSource()) { + var context = new ImportFaultRunContext(f, point); + if (next) point.OnNext = () => cancellation.Cancel(); else point.OnRead = () => cancellation.Cancel(); + var service = context.Service(); var request = new MssqlImportRequest(Guid.NewGuid(), path, f.Target); + using (var preflight = service.Preflight(request, null, CancellationToken.None)) { + Assert(preflight.Passed, "Cancellation fixture passes real preflight: " + key); + var elapsed = Stopwatch.StartNew(); + var worker = Task.Run(() => service.Run(request, preflight, context.Observe, cancellation.Token)); + try { Assert(worker.Wait(20000), "Real reader/probe cancellation finishes within provider+production watchdog budget: " + key); } + finally { cancellation.Cancel(); Assert(worker.Wait(15000), "Cancelled worker is joined before fixture advances: " + key); } + var result = worker.GetAwaiter().GetResult(); ImportRecordResult(f, "cancel-" + key, path, result); + ImportFaultCancelled(result, context, point, true); + Assert(point.ReaderReady.IsSet && (next ? point.NextCalls > 0 : point.ReadCalls >= readAt), "Cancellation occurs at actual selected reader callback."); + Assert(elapsed.ElapsedMilliseconds < 20000, "Cancellation callback and cleanup remain bounded."); + using (var c = f.Open(f.Target)) Assert(ImportLiveFixture.Scalar(c, "SELECT OBJECT_ID(N'dbo.NeverAfterCancel')") == DBNull.Value, "Later GO sentinel absent after selected cancellation boundary."); + File.WriteAllText(Path.Combine(f.Root, "cancel-" + key + "-boundary.json"), JsonConvert.SerializeObject(new { phase, point.CancelCalls, point.ReadCalls, point.NextCalls, elapsedMs = elapsed.ElapsedMilliseconds, result.LastAcknowledgedBatch, result.UncertainBatch, result.Status, context.Stopping, context.Finished }, Formatting.Indented)); + } + } + } + private sealed class ImportFaultRunContext + { + private readonly ImportLiveFixture fixture; private readonly ImportLiveCheckpoint point; private int opens; + internal MssqlImportSqlSession WorkerSession; internal MssqlImportProgress Stopping, Finished; + internal ImportFaultRunContext(ImportLiveFixture f, ImportLiveCheckpoint checkpoint) { fixture = f; point = checkpoint; } + internal void Observe(MssqlImportProgress p) { if (p.Stage == MssqlImportStage.Stopping) Stopping = p; if (p.Stage == MssqlImportStage.Finished) Finished = p; } + internal MssqlImportService Service() { return new MssqlImportService(() => { + var owned = ImportDraftOpenOwned(fixture.Server, fixture.Target, fixture.Databases); var connection = owned.Connection; + int ordinal = Interlocked.Increment(ref opens); + // Preflight is plain actual adapter. The selected checkpoint belongs only to worker commands. + if (ordinal == 1) return new MssqlImportSqlSession(owned); + WorkerSession = new MssqlImportSqlSession(owned, () => new ImportLiveCommand(connection.CreateCommand(), point), () => connection.State == ConnectionState.Open); + return WorkerSession; + }, job => new MssqlImportLog(job, Path.Combine(fixture.Root, "logs"))); } + } + private sealed class ImportFaultObservedLog : IMssqlImportLog + { + private MssqlImportLog inner; private readonly ImportLiveFixture fixture; internal readonly List Details = new List(); + internal bool ErrorDurableBeforeNext; + internal ImportFaultObservedLog(ImportLiveFixture f) { fixture = f; } + internal IMssqlImportLog Open(Guid job, string root) { inner = new MssqlImportLog(job, System.IO.Path.Combine(root, "logs")); return this; } + public string Path { get { return inner.Path; } } public MssqlImportLogTail Tail { get { return inner.Tail; } } + public void WriteStart(MssqlImportRequest r, MssqlImportScan s, MssqlImportTarget t) { inner.WriteStart(r, s, t); } + public void WriteError(MssqlImportBatch b, MssqlImportBatchException e, string a) { Details.AddRange(e.Errors); inner.WriteError(b, e, a); } + public void Flush() { inner.Flush(); if (Details.Count > 0) { string log; using(var file=new FileStream(inner.Path,FileMode.Open,FileAccess.Read,FileShare.ReadWrite))using(var reader=new StreamReader(file))log=reader.ReadToEnd(); using (var c = fixture.Open(fixture.Target)) ErrorDurableBeforeNext = Details.All(x => log.Contains(x.Message)) && Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM dbo.ImportFault WHERE id=2;")) == 0; } } + public void WriteEnd(MssqlImportResult r) { inner.WriteEnd(r); } public void Dispose() { inner.Dispose(); } + } + private static void ImportFaultLogCase(ImportLiveFixture f, string mode) + { + f.ResetTarget(); + string errors = "DECLARE @message nvarchar(1800)=REPLICATE(N'x',1750);RAISERROR(@message,16,1);RAISERROR(@message,16,2);RAISERROR(@message,16,3);"; + string path = f.WriteScript("log-" + mode + ".sql", "CREATE TABLE dbo.ImportFault(id int);\nGO\nINSERT dbo.ImportFault VALUES(1);" + errors + "\nGO\nINSERT dbo.ImportFault VALUES(2);\nGO\n"); + int sessions = 0; ImportDraftLogFailure logFault = null; + Func logs; + if (mode == "create") { + string occupied = Path.Combine(f.Root, "log-directory-is-file"); File.WriteAllText(occupied, "owned fixture file", new UTF8Encoding(false)); + logs = job => new MssqlImportLog(job, occupied); // Real Directory.CreateDirectory IO failure. + } else logs = job => logFault = new ImportDraftLogFailure(job, Path.Combine(f.Root, "logs"), mode); + var service = new MssqlImportService(() => { sessions++; return new MssqlImportSqlSession(ImportDraftOpenOwned(f.Server, f.Target, f.Databases)); }, logs); + var request = new MssqlImportRequest(Guid.NewGuid(), path, f.Target, true); + using (var preflight = service.Preflight(request, null, CancellationToken.None)) { + if (mode == "create" || mode == "start") { + Assert(!preflight.Passed && sessions == 1 && preflight.Issues.Any(x => !x.IsWarning), "Actual filesystem header/create failure prevents execution ownership claim."); + using (var c = f.Open(f.Target)) Assert(ImportLiveFixture.Scalar(c, "SELECT OBJECT_ID(N'dbo.ImportFault')") == DBNull.Value, "Logging startup failure executes no source SQL."); + // Rejecting Run with this preflight must not open a worker connection. + var rejected = service.Run(request, preflight, null, CancellationToken.None); + Assert(rejected.Status == MssqlImportStatus.Failed && rejected.ValidationSummary == "PreflightInvalidOrConsumed", "Failed real preflight cannot be claimed for execution."); + Assert(sessions == 1, "Failed log preflight did not open a second SQL session."); + } else { + Assert(preflight.Passed, "Log failure is armed at worker error/end phase only."); + var result = service.Run(request, preflight, null, CancellationToken.None); ImportRecordResult(f, "log-" + mode, path, result); + Assert(result.Status == MssqlImportStatus.Failed && result.ValidationSummary.Contains("LogIncomplete"), "Actual log stream failure is visible in final result."); + using (var c = f.Open(f.Target)) Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM dbo.ImportFault WHERE id=2")) == (mode == "end" ? 1 : 0), "Error write/flush failure stops before next GO; terminal failure reports writes already acknowledged."); + Assert(result.LastAcknowledgedBatch == (mode == "end" ? 3 : 1), "Logging failure cannot advance an unacknowledged SQL batch."); + ImportFaultClean(result, true); + } + } + } + private static void ImportFaultPermissions(ImportLiveFixture f, string nativeScript) + { + foreach (string permissionMode in new[] { "no-view-definition", "user-deny", "role-deny", "public-deny" }) { + f.ResetTarget(); + const string user = "ImportFixtureLimited", role = "ImportFixtureLimitedRole"; + using (var c = f.Open(f.Target)) { + ImportLiveFixture.Exec(c, "CREATE TABLE dbo.HiddenFixture(id int);INSERT dbo.HiddenFixture VALUES(1);CREATE USER " + ImportLiveFixture.Q(user) + " WITHOUT LOGIN;CREATE ROLE " + ImportLiveFixture.Q(role) + ";ALTER ROLE " + ImportLiveFixture.Q(role) + " ADD MEMBER " + ImportLiveFixture.Q(user) + ";EXEC(N'CREATE SCHEMA import_owned AUTHORIZATION " + ImportLiveFixture.Q(user) + ";');GRANT CREATE TABLE TO " + ImportLiveFixture.Q(user) + ";"); + if (permissionMode != "no-view-definition") { + ImportLiveFixture.Exec(c, "GRANT VIEW DEFINITION TO " + ImportLiveFixture.Q(user) + ";GRANT VIEW SECURITY DEFINITION TO " + ImportLiveFixture.Q(user) + ";DENY VIEW DEFINITION ON OBJECT::dbo.HiddenFixture TO " + ImportLiveFixture.Q(permissionMode == "user-deny" ? user : permissionMode == "role-deny" ? role : "public") + ";"); + } + } + Func sessions = () => { + var owned = ImportDraftOpenOwned(f.Server, f.Target, f.Databases); + try { ImportLiveFixture.Exec(owned.Connection, "EXECUTE AS USER=" + ImportLiveFixture.L(user) + ";"); return new MssqlImportSqlSession(new ImportFaultRevertOwner(owned), () => owned.Connection.CreateCommand(), () => owned.Connection.State == ConnectionState.Open); } + catch { owned.Dispose(); throw; } + }; + // Direct independent impersonated view proves catalog filtering before adapter decision. + using (var c = f.Open(f.Target)) try { + ImportLiveFixture.Exec(c, "EXECUTE AS USER=" + ImportLiveFixture.L(user) + ";"); + Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM sys.objects WHERE is_ms_shipped=0;")) == 0, "Real effective token filters existing hidden object to zero."); + Assert(Convert.ToInt32(ImportLiveFixture.Scalar(c, "SELECT HAS_PERMS_BY_NAME(DB_NAME(),'DATABASE','ALTER');")) == 0, "Fixture user has no ALTER DATABASE permission."); + } finally { if (c.State == ConnectionState.Open) ImportLiveFixture.Exec(c, "REVERT;"); } + using (var session = sessions()) { + var target = session.ReadTarget(CancellationToken.None); + Assert(!target.MetadataVisible && !target.UserObjectCount.HasValue, "Filtered zero metadata is unknown for complete effective user/role/public token."); + } + var service = new MssqlImportService(sessions, job => new MssqlImportLog(job, Path.Combine(f.Root, "logs"))); + var nativeRequest = new MssqlImportRequest(Guid.NewGuid(), nativeScript, f.Target); + using (var preflight = service.Preflight(nativeRequest, null, CancellationToken.None)) Assert(!preflight.Passed && preflight.Issues.Any(x => x.Code == "TargetVisibilityRequired"), "Native full import rejects incomplete metadata without claiming empty target."); + string generic = f.WriteScript("limited-" + permissionMode + ".sql", "CREATE TABLE import_owned.VisibleFixture(id int);INSERT import_owned.VisibleFixture VALUES(7);\nGO\n"); + var request = new MssqlImportRequest(Guid.NewGuid(), generic, f.Target); + using (var preflight = service.Preflight(request, null, CancellationToken.None)) { + Assert(preflight.Passed && preflight.Issues.Any(x => x.Code == "TargetNotProvenEmpty"), "Generic import under same limited token is allowed with explicit visibility warning."); + var result = service.Run(request, preflight, null, CancellationToken.None); ImportRecordResult(f, "limited-" + permissionMode, generic, result); + Assert(result.Status == MssqlImportStatus.Completed, "Limited schema owner imports without ALTER DATABASE or broader permissions."); ImportFaultClean(result, true); + using (var c = f.Open(f.Target)) Assert(Convert.ToInt64(ImportLiveFixture.Scalar(c, "SELECT COUNT_BIG(*) FROM import_owned.VisibleFixture WHERE id=7;")) == 1, "Independent admin reads actual limited-caller committed data."); + } + } + f.ResetTarget(); + } + private sealed class ImportFaultRevertOwner : IDisposable + { + private MssqlOwnedSession owned; + internal ImportFaultRevertOwner(MssqlOwnedSession session) { owned = session; } + public void Dispose() { + var value = Interlocked.Exchange(ref owned, null); if (value == null) return; + try { if (value.Connection.State == ConnectionState.Open) ImportLiveFixture.Exec(value.Connection, "REVERT;"); } + finally { value.Dispose(); } + } + } + private static void ImportLiveMemory(ImportLiveFixture f) + { + f.Case("memory.drain-million-rows",()=>{ + string path=f.WriteScript("million-results.sql","/*IMPORT_INTEGRATION_DRAIN*/ WITH n AS(SELECT TOP(2500000) ROW_NUMBER() OVER(ORDER BY(SELECT NULL)) n FROM sys.all_objects a CROSS JOIN sys.all_objects b) SELECT n FROM n;\r\nGO\r\n"); + using(var point=new ImportLiveCheckpoint("Drain"))using(var memory=new ImportDraftMemory()){ + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,path,f.Root,point:point,progress:p=>memory.Stage=p.Stage.ToString());ImportRecordResult(f,"million",path,result); + File.WriteAllText(Path.Combine(f.Root,"memory-million.json"),JsonConvert.SerializeObject(memory.Snapshot(),Formatting.Indented)); + Assert(result.Status==MssqlImportStatus.Completed&&point.ReadCalls==2500001&&point.ValueGetterCalls==0,"Actual SqlClient discarded2.5million rows with complete result drain."); + } + }); + f.Case("memory.drain-64m-returned-lob",()=>{ + string path=f.WriteScript("returned-lob.sql","/*IMPORT_INTEGRATION_DRAIN*/ SELECT CONVERT(varbinary(max),REPLICATE(CONVERT(varchar(max),'z'),67108864));SELECT 2;\r\nGO\r\n"); + using(var point=new ImportLiveCheckpoint("Drain"))using(var memory=new ImportDraftMemory()){ + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,path,f.Root,point:point,progress:p=>memory.Stage=p.Stage.ToString());ImportRecordResult(f,"returned-lob",path,result); + var values=memory.Snapshot();File.WriteAllText(Path.Combine(f.Root,"memory-returned-lob.json"),JsonConvert.SerializeObject(values,Formatting.Indented)); + Assert(result.Status==MssqlImportStatus.Completed&&point.ReadCalls==4&&point.NextCalls==2&&point.ValueGetterCalls==0,"Actual64MiB returned LOB and next result drained without getters/materialization."); + long peak=values.Values.Max(x=>x.PeakPrivateBytes),baseline=values.Values.Min(x=>x.FirstPrivateBytes); + Assert(peak-baseline<256L*1024*1024,"Returned64MiB result drain private memory remains bounded."); + } + }); + f.Case("memory.warmup-x64",()=>{Assert(IntPtr.Size==8,"Actual integration process is x64.");ImportMemoryWarmup(f);}); + JObject small=null,large=null; + f.Case("memory.stream-256m",()=>{small=ImportRunMemoryChild(f,1);}); + f.Case("memory.stream-2g5",()=>{large=ImportRunMemoryChild(f,10);}); + f.Case("memory.compare-peaks",()=>{ + Assert((long)large["fileBytes"]==(long)small["fileBytes"]*10&&(long)large["batches"]==(long)small["batches"]*10,"Actual executed file bytes and batch counts scale exactly10x."); + long smallDelta=(long)small["peakPrivateBytes"]-(long)small["baselinePrivateBytes"],largeDelta=(long)large["peakPrivateBytes"]-(long)large["baselinePrivateBytes"]; + long difference=largeDelta-smallDelta; + File.WriteAllText(Path.Combine(f.Root,"memory-comparison.json"),JsonConvert.SerializeObject(new{small,large,smallDelta,largeDelta,difference,thresholdBytes=256L*1024*1024},Formatting.Indented)); + Assert(difference<=256L*1024*1024,"10x actual import cannot add>256MiB private bytes; investigate instead of raising cap."); + Assert((long)large["peakPrivateBytes"]-(long)small["peakPrivateBytes"]<=256L*1024*1024,"Raw private peak difference also remains bounded."); + }); + } + private static void ImportMemoryWarmup(ImportLiveFixture f) + { + string path=Path.Combine(f.Root,"memory-warmup.sql"); + using(var writer=new StreamWriter(path,false,new UTF8Encoding(false),65536)){ + writer.Write("SELECT CAST(1 AS int); /*");string block=new string('m',65536);for(int i=0;i<16;i++)writer.Write(block);writer.Write("*/\r\nGO\r\n"); + } + var result=ImportDraftRunActual(f.Server,f.Target,f.Databases,path,f.Root);Assert(result.Status==MssqlImportStatus.Completed&&result.SuccessfulBatches==1,"Actual importer warmup uses identical bounded batch shape."); + } + private static void ImportLiveMemoryChild(ImportLiveFixture f,int scale) + { + Assert(scale==1||scale==10,"Exact memory child scale only.");Assert(IntPtr.Size==8,"Memory child is x64."); + string path=Path.Combine(f.Root,"memory-input.sql");long bytes=ImportDraftGenerateMemorySql(path,scale); + f.Case("memory.warmup-x64",()=>ImportMemoryWarmup(f)); + f.Case(scale==1?"memory.stream-256m":"memory.stream-2g5",()=>{ + long baseline;using(var process=Process.GetCurrentProcess()){process.Refresh();baseline=process.PrivateMemorySize64;} + MssqlImportResult result;Dictionary values; + using(var memory=new ImportDraftMemory()){ + result=ImportDraftRunActual(f.Server,f.Target,f.Databases,path,f.Root,progress:p=>memory.Stage=p.Stage.ToString());values=memory.Snapshot(); + } + ImportRecordResult(f,"memory-stream",path,result); + Assert(result.Status==MssqlImportStatus.Completed&&result.SuccessfulBatches==256L*scale&&result.ProcessedBytes==bytes,"Actual importer executes all memory batches and full digest/EOF boundary."); + Assert(bytes>=256L*1024*1024*scale&&bytes<256L*1024*1024*scale+1024*1024,"Actual streamed input matches planned size with only bounded per-batch framing."); + Assert(values.Values.Sum(x=>x.Samples)>3,"Memory sampler captured actual scan/execute samples."); + File.WriteAllText(Path.Combine(f.Root,"memory-measurement.json"),JsonConvert.SerializeObject(new{scale,processBits=IntPtr.Size*8,fileBytes=bytes,batches=result.SuccessfulBatches, + baselinePrivateBytes=baseline,peakPrivateBytes=values.Values.Max(x=>x.PeakPrivateBytes),peakWorkingSet=values.Values.Max(x=>x.PeakWorkingSet),stages=values,elapsed=result.Elapsed},Formatting.Indented)); + }); + } + private static JObject ImportRunMemoryChild(ImportLiveFixture f,int scale) + { + string parent=Path.Combine(f.Root,"child-"+scale),identity=Guid.NewGuid().ToString("N"); + var start=new ProcessStartInfo(typeof(SmokeTests).Assembly.Location){UseShellExecute=false,CreateNoWindow=true,RedirectStandardOutput=true,RedirectStandardError=true}; + start.EnvironmentVariables["MYSQLPUNK_MSSQL_IMPORT_MEMORY_SCALE"]=scale.ToString();start.EnvironmentVariables["MYSQLPUNK_MSSQL_IMPORT_OUTPUT"]=parent;start.EnvironmentVariables["MYSQLPUNK_MSSQL_IMPORT_RUN_ID"]=identity; + start.EnvironmentVariables["MYSQLPUNK_SMOKE_FILTER"]="MSSQL import integration"; + int exit=RunOwnedRestore(start,3600000,receipt=>File.WriteAllText(Path.Combine(f.Root,"memory-child-"+scale+".json"),JsonConvert.SerializeObject(receipt,Formatting.Indented))); + Assert(exit==0,"Owned actual-importer memory child exits successfully."); + string root=Path.Combine(parent,"Memory");var receipt=JObject.Parse(File.ReadAllText(Path.Combine(root,"profile-receipt.json"))); + Assert((bool)receipt["complete"]&&(int)receipt["processBits"]==64&&(string)receipt["runId"]==identity&&(string)receipt["cleanupStatus"]=="Confirmed","Actual child identity,completion,x64 and dual ownership cleanup confirmed."); + var cases=(JArray)receipt["cases"];Assert(cases.Count==2&&cases.All(x=>(string)x["status"]=="Passed"&&(bool)x["executed"]),"Actual child warmup and requested stream cases both executed/passed."); + foreach(var item in cases){string file=Path.Combine(root,(string)item["caseId"]+".json");using(var sha=SHA256.Create())using(var stream=File.OpenRead(file))Assert(BitConverter.ToString(sha.ComputeHash(stream)).Replace("-","").ToLowerInvariant()==(string)item["receiptSha256"],"Child case receipt hashes match actual files.");} + return JObject.Parse(File.ReadAllText(Path.Combine(root,"memory-measurement.json"))); + }} \ No newline at end of file diff --git a/tests/SmokeTests.MssqlImport.Parser.cs b/tests/SmokeTests.MssqlImport.Parser.cs new file mode 100644 index 0000000..a5f4891 --- /dev/null +++ b/tests/SmokeTests.MssqlImport.Parser.cs @@ -0,0 +1,218 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Security.Cryptography; +using System.Text; +using System.Threading; +using mySQLPunk.lib.MssqlImport; + +public static partial class SmokeTests +{ + // Breaks caught: GO split inside a lexical context, decoder boundary corruption, + // accepting over-budget SQL, native downgrade, read-ahead progress and handle replacement. + private static void MssqlImportParserPreservesContexts() + { + string first = "SELECT N'羽😀\r\nGO\r\n''x';\r\nSELECT \"a\"\"\r\nGO\r\nb\", [a]]\r\nGO\r\nb];\r\n/* outer\r\n/* nested */\r\nGO\r\n*/\r\n-- GO\r\n"; + string last = "SELECT 2;"; + foreach (Encoding encoding in new Encoding[] { new UTF8Encoding(false,true), new UTF8Encoding(true,true), new UnicodeEncoding(false,true,true), new UnicodeEncoding(true,true,true) }) + foreach (int buffer in new[] {1,2,3,7}) + { + byte[] bytes = ImportBytes(first + " gO -- end\r\n" + last, encoding); + using (var stream = new MemoryStream(bytes)) + using (var reader = new MssqlGoBatchReader(stream, buffer)) + { + var batches = reader.ReadBatches(CancellationToken.None).ToArray(); + Assert(batches.Length == 2, "Context-aware GO reader yields exactly two commands."); + Assert(batches[0].Text == first && batches[1].Text == last, "Original SQL text survives tiny decode buffers and quote escapes."); + Assert(batches[0].StartByte == 0 && batches[0].EndByte == ImportBytes(first + " gO -- end\r\n",encoding).LongLength, "EndByte includes original encoded GO line and BOM."); + Assert(batches[1].StartByte == batches[0].EndByte && batches[1].EndByte == bytes.LongLength, "Byte ranges cover last undelimited command."); + Assert(batches[0].StartLine == 1 && batches[0].EndLine == 13 && batches[1].StartLine == 15 && batches[1].EndLine == 15, "Original line offsets exclude GO delimiter."); + } + } + // A later malformed byte in one decoder buffer must not erase valid + // preceding line/GO transitions, including EOF decoder flush errors. + foreach(Encoding encoding in new Encoding[]{new UTF8Encoding(false,true),new UTF8Encoding(true,true),new UnicodeEncoding(false,true,true),new UnicodeEncoding(true,true,true)}) + foreach(int buffer in new[]{8192,1,2,3,7}) + foreach(int padding in new[]{0,9000}) + foreach(string newline in new[]{"\n","\r\n"}) + { + string prefix=(padding==0?"":"--"+new string(' ',padding)+newline)+"SELECT 1;"+newline+"GO"+newline+"SELECT '"+newline; + byte[] valid=ImportBytes(prefix,encoding); + bool utf8=encoding.CodePage==65001, little=encoding.CodePage==1200; + byte[][] invalid=utf8 ? new[]{new byte[]{0xff},new byte[]{0xe2,0x82}} : little ? new[]{new byte[]{0x00,0xdc},new byte[]{0x00,0xd8},new byte[]{0x41}} : new[]{new byte[]{0xdc,0x00},new byte[]{0xd8,0x00},new byte[]{0x41}}; + foreach(byte[] suffix in invalid) + using(var stream=new MemoryStream(valid.Concat(suffix).ToArray())) + using(var reader=new MssqlGoBatchReader(stream,buffer)) + { + try {reader.ReadBatches(CancellationToken.None).ToArray();Assert(false,"Malformed encoding must reject after valid GO prefix.");} + catch(MssqlImportInputException ex) + { + Assert(ex.Code=="InvalidEncoding" && ex.BatchNumber==2 && ex.Line==(padding==0?4:5) && ex.ByteOffset==valid.LongLength, + "Decoder error must preserve actual batch/line/raw byte coordinates: "+encoding.CodePage+" buffer="+buffer+" padding="+padding+" got "+ex.Message); + } + } + } + using(var input = ImportInput("-- only\r\n/* nested /* ok */ end */\r\nGO\r\n-- tail")) + { + Assert(input.Scan(CancellationToken.None,null).BatchCount==0, "Only comments never become executable commands."); + Assert(!input.ReadBatches(CancellationToken.None).Any() && input.ExecutionSha256!=null, "Comment-only EOF still finishes execution digest."); + } + } + + private static void MssqlImportParserLimits() + { + const int limit = 16777216; + string sql = "SELECT '" + new string('x',limit-10) + "';"; + Assert(sql.Length==limit, "Fixture reaches the documented code-unit boundary."); + using(var input=ImportInput(sql+"\r\nGO\r\n")) + { + // The CRLF preceding GO is SQL text and counts toward the batch budget. + ImportReject(input,"BatchTooLarge"); + } + using(var input=ImportInput(sql)) + { + var scan=input.Scan(CancellationToken.None,null); + Assert(scan.BatchCount==1 && scan.MaxBatchCodeUnits==limit,"Exactly 16,777,216 code units scan successfully."); + } + using(var input=ImportInput(sql+" ")) ImportReject(input,"BatchTooLarge"); + string framed = "SELECT '" + new string('x',limit-12) + "';\r\n"; + using(var input=ImportInput(framed+"GO --delimiter\r\n")) + Assert(input.Scan(CancellationToken.None,null).MaxBatchCodeUnits==limit,"Exact cap remains valid with a GO delimiter excluded from SQL."); + using(var input=ImportInput("SELECT 1;\n"+new string(' ',limit+5)+"GO --"+new string('x',limit+5)+"\nSELECT 2;")) + Assert(input.Scan(CancellationToken.None,null).BatchCount==2,"Oversized GO whitespace/comment candidates are discarded without losing commands."); + using(var input=ImportInput(new string(' ',limit+5)+"SELECT 1;")) ImportReject(input,"BatchTooLarge"); + using(var input=ImportInput("--"+new string(' ',limit-2))) + Assert(input.Scan(CancellationToken.None,null).BatchCount==0,"Huge single comment line is bounded without an empty command."); + } + + private static void MssqlImportParserUnsupported() + { + foreach(string sql in new[]{"SELECT 1\nGO 2\n","SELECT 1\nGO /*comment*/\n","SELECT 1\nGO--ok\n:r file.sql","/*ok*/ :setvar a b","!! whoami","SELECT 1; USE [other];","USE [transfer];"}) + using(var input=ImportInput(sql,"Transfer")) ImportReject(input,sql.Contains("GO 2")||sql.Contains("GO /*")?"UnsupportedGo":sql.Contains("USE")?"DatabaseChange":"SqlCmdUnsupported"); + using(var input=ImportInput("/* same-line comment */ GO\n")) ImportReject(input,"UnsupportedGo"); + foreach(string sql in new[]{"SELECT '\nGO","SELECT [abc","SELECT \"abc","/* /* nested */"}) + using(var input=ImportInput(sql)) ImportReject(input,"UnclosedContext"); + using(var input=ImportInput("USE [Tran]]sfer]; SELECT ':r file.sql GO 2 !!'; -- USE other\n","Tran]sfer")) + Assert(input.Scan(CancellationToken.None,null).BatchCount==1,"Exact decoded same-database USE succeeds; string/comment directives are inert."); + using(var input=ImportInput("USE \"Transfer\";", "Transfer")) Assert(input.Scan(CancellationToken.None,null).BatchCount==1,"Double-quoted exact USE succeeds."); + using(var input=ImportInput("USE Transfer;")) ImportReject(input,"DatabaseChange"); + foreach(Encoding encoding in new Encoding[]{new UTF8Encoding(true,true),new UnicodeEncoding(false,true,true),new UnicodeEncoding(true,true,true)}) + { + string prefix="SELECT N'\u7fbd\U0001f600';\r\n"; + using(var input=ImportInput(ImportBytes(prefix+"USE [other];",encoding),"Transfer")) + { + try {input.Scan(CancellationToken.None,null);Assert(false,"Cross-database USE must reject.");} + catch(MssqlImportInputException ex) {Assert(ex.Code=="DatabaseChange" && ex.Line==2 && ex.ByteOffset==ImportBytes(prefix+"USE ",encoding).LongLength,"Error source byte/line coordinates use original encoding and BOM.");} + } + } + foreach(byte[] bytes in new[]{new byte[]{0xef,0xbb},new byte[]{0xc0,0xaf},new byte[]{0xe2,0x82},new byte[]{0xff,0xfe,0x00},new byte[]{0xff,0xfe,0x00,0xd8},new byte[]{0xfe,0xff,0xdc,0x00},new byte[]{0xff,0xfe,0,0,0,0,0,0}}) + using(var input=ImportInput(bytes)) ImportReject(input,"InvalidEncoding"); + foreach(byte[] bytes in new[]{new byte[]{0x53,0,0x45,0},ImportBytes("SELECT '\0';",new UTF8Encoding(false,true)),ImportBytes("SELECT '\0';",new UnicodeEncoding(false,true,true))}) + using(var input=ImportInput(bytes)) ImportReject(input,"EmbeddedNul"); + foreach(string sql in new[]{"SELECT col$(suffix) FROM dbo.t;","SELECT dbo.table$(suffix).col FROM dbo.t;","USE Transfer$(suffix);","SELECT @col$(suffix);","SELECT #tmp$(suffix);","SELECT "+new string('x',600)+"$(suffix);","SELECT 1;\nGO\nSELECT col$(suffix);"}) + using(var input=ImportInput(sql,"Transfer")) ImportReject(input,"SqlCmdUnsupported"); + using(var input=ImportInput("SELECT N'col$(suffix)', \"col$(suffix)\", [col$(suffix)], [a]]$(suffix)]; -- col$(suffix)\n/* outer /* col$(suffix) */ end */")) + Assert(input.Scan(CancellationToken.None,null).Script.Kind==MssqlImportScriptKind.Generic,"Embedded SQLCMD text remains inert in all protected lexical contexts."); + using(var input=ImportInput("SELECT '$(name) :connect !!';")) Assert(input.Scan(CancellationToken.None,null).BatchCount==1,"No SQLCMD expansion in strings."); + } + + private const string ImportJob="123456781234123412341234567890ab"; + private const string ImportHeader="-- Source read mode: READ COMMITTED\r\n"; + private const string ImportHint="-- Restore into an empty, compatible target selected by the caller; existing objects fail.\r\n"; + private const string ImportFooter="-- mysqlpunk-export-complete job="+ImportJob+" tables=1 rows=2\r\n"; + private static void MssqlImportParserNative() + { + string schema=ImportHeader+ImportHint+"CREATE TABLE [dbo].[x] ([a] int);\r\nGO\r\n"+ImportFooter; + using(var input=ImportInput(schema)) + { + var script=input.Scan(CancellationToken.None,null).Script; + Assert(script.Kind==MssqlImportScriptKind.NativeSchema && script.NativeJobId==Guid.ParseExact(ImportJob,"N") && script.ExpectedTables==1 && script.ExpectedRows==2 && !script.HasLobProtocol && script.SourceReadMode=="READ COMMITTED","Native schema metadata is exact without requiring LOB."); + } + using(var input=ImportInput(ImportHeader+"INSERT INTO dbo.x VALUES(1);\r\nGO\r\n"+ImportFooter)) + Assert(input.Scan(CancellationToken.None,null).Script.Kind==MssqlImportScriptKind.NativeData,"Existing DataOnly header is accepted without a new mode manifest."); + using(var input=ImportInput(ImportHeader+"CREATE TABLE #mp_"+ImportJob+"_s ([Sequence] bigint);\r\nGO\r\n"+ImportFooter)) + Assert(input.Scan(CancellationToken.None,null).Script.HasLobProtocol,"Actual validated native temporary identifiers mark LOB."); + foreach(string sql in new[]{ImportHeader+ImportHint+"CREATE TABLE x(a int);",schema+ImportFooter,schema.Replace("rows=2","rows=9223372036854775808"),schema.Replace("tables=1","tables=-1"),schema.Replace(ImportJob,"bad"),ImportFooter,ImportHeader+"CREATE TABLE x(a int);\nGO\n"+ImportFooter,ImportHeader+ImportHeader+ImportFooter,ImportHeader.Replace("READ COMMITTED","unknown")+ImportFooter,ImportHeader+"CREATE TABLE #mp_ffffffffffffffffffffffffffffffff_s (a int);\nGO\n"+ImportFooter,ImportHeader+"CREATE TABLE #mp_bad_s (a int);\nGO\n"+ImportFooter,"CREATE TABLE #mp_"+ImportJob+"_s (a int);"}) + using(var input=ImportInput(sql)) ImportReject(input,"NativeFormat"); + using(var input=ImportInput("SELECT '-- Source read mode: READ COMMITTED', '-- mysqlpunk-export-complete job=bad', [#mp_bad_s];\nGO\n-- regular comment")) + Assert(input.Scan(CancellationToken.None,null).Script.Kind==MssqlImportScriptKind.Generic,"String/ordinary identifier decoys do not create native markers."); + using(var input=ImportInput("SELECT 1;\n")) Assert(input.Scan(CancellationToken.None,null).Script.Kind==MssqlImportScriptKind.Generic,"Generic SQL requires no native footer."); + using(var input=ImportInput("-- Source read mode READ COMMITTED\n")) ImportReject(input,"NativeFormat"); + string emptyFooter=ImportFooter.Replace("tables=1 rows=2","tables=0 rows=0"); + using(var input=ImportInput(ImportHeader+ImportHint+emptyFooter)) + Assert(input.Scan(CancellationToken.None,null).Script.Kind==MssqlImportScriptKind.NativeSchema,"Existing zero-object schema export remains valid."); + using(var input=ImportInput(ImportHeader.Replace("READ COMMITTED","Schema only (no data transaction)")+ImportHint+"CREATE TABLE x(a int);\nGO\n"+ImportFooter.Replace("rows=2","rows=0"))) + Assert(input.Scan(CancellationToken.None,null).Script.Kind==MssqlImportScriptKind.NativeSchema,"SchemaOnly source mode and zero row footer are coherent."); + using(var input=ImportInput(ImportHeader.Replace("READ COMMITTED","Schema only (no data transaction)")+ImportHint+ + "CREATE TABLE p(id int PRIMARY KEY); CREATE TABLE c(id int,parent int);\nGO\nALTER TABLE c ADD CONSTRAINT FK_c FOREIGN KEY(parent) REFERENCES p(id) ON DELETE NO ACTION ON UPDATE NO ACTION;\nGO\n"+ImportFooter.Replace("tables=1 rows=2","tables=2 rows=0"))) + Assert(input.Scan(CancellationToken.None,null).Script.Kind==MssqlImportScriptKind.NativeSchema,"Actual exporter FK ON DELETE/ON UPDATE are schema action clauses, not SchemaOnly data writes."); + foreach(string dml in new[]{"SET IMPLICIT_TRANSACTIONS ON UPDATE x SET a=1;", "SET ANSI_NULLS ON DELETE FROM x;", + "ALTER TABLE x ADD FOREIGN KEY(a) REFERENCES x(a) ON UPDATE SET NULL; UPDATE x SET a=1;", + "ALTER TABLE x ADD FOREIGN KEY(a) REFERENCES x(a) ON DELETE NO ACTION SET ANSI_NULLS ON DELETE FROM x;"}) + using(var input=ImportInput(ImportHeader.Replace("READ COMMITTED","Schema only (no data transaction)")+ImportHint+"CREATE TABLE x(a int); "+dml+"\nGO\n"+ImportFooter.Replace("rows=2","rows=0"))) + ImportReject(input,"NativeFormat"); + foreach(string ddl in new[]{ + "CREATE TABLE p(a int,b int,PRIMARY KEY(a,b)); CREATE TABLE c(a int,b int); ALTER TABLE c ADD FOREIGN KEY(a,b) REFERENCES dbo.p(a,b) ON DELETE NO ACTION ON UPDATE NO ACTION;", + "CREATE TABLE p(a int,b int,PRIMARY KEY(a,b)); CREATE TABLE c(a int,b int,FOREIGN KEY(a,b) REFERENCES dbo.p(a,b) ON DELETE CASCADE ON UPDATE SET NULL,CONSTRAINT CK_c CHECK(a>=0));"}) + using(var input=ImportInput(ImportHeader.Replace("READ COMMITTED","Schema only (no data transaction)")+ImportHint+ddl+"\nGO\n"+ImportFooter.Replace("tables=1 rows=2","tables=2 rows=0"))) + Assert(input.Scan(CancellationToken.None,null).Script.Kind==MssqlImportScriptKind.NativeSchema,"Composite FK key-list commas are not constraint boundaries in ALTER or CREATE TABLE."); + foreach(string action in new[]{"NO ACTION","CASCADE","SET NULL","SET DEFAULT"}) + using(var input=ImportInput(ImportHeader.Replace("READ COMMITTED","Schema only (no data transaction)")+ImportHint+"CREATE TABLE x(a int); ALTER TABLE x ADD FOREIGN KEY(a) REFERENCES x(a) ON DELETE "+action+" ON UPDATE "+action+";\nGO\n"+ImportFooter.Replace("rows=2","rows=0"))) + Assert(input.Scan(CancellationToken.None,null).Script.Kind==MssqlImportScriptKind.NativeSchema,"Exact FK referential action recognized: "+action); + foreach(string malformed in new[]{ImportHeader+ImportHint+"SELECT 'CREATE TABLE x(a int)';\nGO\n"+ImportFooter,"CREATE TABLE x(a int);\nGO\n"+ImportHeader+ImportHint+ImportFooter,ImportHeader.Replace("READ COMMITTED","Schema only (no data transaction)")+ImportHint+"CREATE TABLE x(a int); INSERT INTO x VALUES(1);\nGO\n"+ImportFooter}) + using(var input=ImportInput(malformed)) ImportReject(input,"NativeFormat"); + } + + private static void MssqlImportParserFileIdentity() + { + byte[] bytes=ImportBytes("SELECT N'羽😀';\r\nGO\r\n-- footer only\r\n",new UTF8Encoding(true,true)); + string path=ImportFixture(bytes); + try + { + using(var input=MssqlImportInput.Open(path)) + { + long progress=0; + var scan=input.Scan(CancellationToken.None,p=>{Assert(p>=progress&&p<=bytes.LongLength,"Scan progress is monotonic bytes.");progress=p;}); + string expected; + using(var hash=SHA256.Create()) expected=BitConverter.ToString(hash.ComputeHash(bytes)).Replace("-","").ToLowerInvariant(); + Assert(scan.Sha256==expected && scan.FileBytes==bytes.LongLength && progress==bytes.LongLength,"Preflight hashes all original bytes including BOM and comment tail."); + Assert(input.ExecutionSha256==null,"Scan cannot claim execution digest."); + var commands=input.ReadBatches(CancellationToken.None).ToArray(); + Assert(commands.Length==1 && commands[0].EndByte(()=>{using(var writer=new FileStream(path,FileMode.Open,FileAccess.Write,FileShare.ReadWrite)){}},"Held input blocks general writers."); + AssertThrows(()=>File.Delete(path),"Held input blocks delete."); + } + using(var writer=new FileStream(path,FileMode.Open,FileAccess.Write,FileShare.None)) writer.WriteByte(0x20); + } + finally {File.Delete(path);} + using(var input=ImportInput("SELECT 1;\nGO\nSELECT 2;")) + { + var cancel=new CancellationTokenSource();cancel.Cancel(); + AssertThrows(()=>input.Scan(cancel.Token,null),"Preflight cancellation performs no full scan."); + input.Scan(CancellationToken.None,null); + using(var enumerator=input.ReadBatches(CancellationToken.None).GetEnumerator()) {Assert(enumerator.MoveNext(),"First command exists.");} + Assert(input.ExecutionSha256==null,"Abandoned enumeration never claims EOF digest."); + } + } + + private static byte[] ImportBytes(string sql,Encoding encoding) {return encoding.GetPreamble().Concat(encoding.GetBytes(sql)).ToArray();} + private static string ImportFixture(byte[] bytes) {string path=Path.Combine(Path.GetTempPath(),"mysqlpunk-import-parser-"+Guid.NewGuid().ToString("N")+".sql");File.WriteAllBytes(path,bytes);return path;} + private static ImportParserFixture ImportInput(string sql,string database=null) {return ImportInput(new UTF8Encoding(false,true).GetBytes(sql),database);} + private static ImportParserFixture ImportInput(byte[] bytes,string database=null) {return new ImportParserFixture(bytes,database);} + private sealed class ImportParserFixture : IDisposable + { + private readonly string path; + private readonly MssqlImportInput input; + internal ImportParserFixture(byte[] bytes,string database) {path=ImportFixture(bytes);try {input=MssqlImportInput.Open(path,database);}catch {File.Delete(path);throw;}} + internal MssqlImportScan Scan(CancellationToken token,Action progress) {return input.Scan(token,progress);} + internal IEnumerable ReadBatches(CancellationToken token) {return input.ReadBatches(token);} + internal string ExecutionSha256 {get {return input.ExecutionSha256;}} + public void Dispose() {input.Dispose();File.Delete(path);} + } + private static void ImportReject(ImportParserFixture input,string code) + { + try {input.Scan(CancellationToken.None,null);Assert(false,"Expected preflight rejection: "+code);} + catch(MssqlImportInputException ex) {Assert(ex.Code==code,"Stable preflight code: expected "+code+", got "+ex.Code);Assert(ex.Line>=1&&ex.ByteOffset>=0&&ex.BatchNumber>=1,"Rejected input has source coordinates.");} + } +} diff --git a/tests/SmokeTests.MssqlImport.TransferTrial.cs b/tests/SmokeTests.MssqlImport.TransferTrial.cs new file mode 100644 index 0000000..3d0fb81 --- /dev/null +++ b/tests/SmokeTests.MssqlImport.TransferTrial.cs @@ -0,0 +1,518 @@ +// SQL2019 Transfer acceptance harness. Isolated fixture authority never applies here. +using System; +using System.Collections.Generic; +using System.Data; +using System.Data.SqlClient; +using System.Diagnostics; +using System.Globalization; +using System.IO; +using System.Text; +using System.Security.Cryptography; +using System.Security.Principal; +using System.Runtime.InteropServices; +using System.Runtime.ExceptionServices; +using System.Threading; +using mySQLPunk.lib; +using mySQLPunk.lib.MssqlImport; +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; + +public static partial class SmokeTests +{ + private const string TransferInput = @"C:\Users\johnho\Desktop\Transfer_dump_20261013_165730.sql"; + private const string TransferAttempt = @"C:\Users\johnho\Documents\mySQLPunk-worktrees\mssql-import\.superpowers\sdd\implementation-plan\ImportTrials\JOHN_MSSQL2019-Transfer-Transfer_dump_20261013_165730.attempt.json"; + + private const string TransferProductExecutable = @"C:\Users\johnho\Documents\mySQLPunk-worktrees\mssql-import\mySQLPunk\bin\CodexVerify\mySQLPunk.exe"; + private const long TransferEntryFree = 68719476736L, TransferCancelFree = 21474836480L; + private const string TransferPin = @"C:\Users\johnho\Documents\mySQLPunk-worktrees\mssql-import\.superpowers\sdd\implementation-plan\task-7-build-pin.json"; + [DllImport("kernel32.dll", CharSet=CharSet.Unicode, SetLastError=true)] + private static extern bool GetVolumePathName(string fileName, StringBuilder volumePathName, uint length); + [DllImport("kernel32.dll", CharSet=CharSet.Unicode, SetLastError=true)] + private static extern bool GetVolumeNameForVolumeMountPoint(string mountPoint, StringBuilder volumeName, uint length); + private static string TransferHash(string path) + { using(var hash=SHA256.Create())using(var input=File.OpenRead(path))return BitConverter.ToString(hash.ComputeHash(input)).Replace("-","").ToLowerInvariant(); } + private static JObject TransferReadBounded(string path) + { + TransferPathNoReparse(path);byte[] bytes; + using(var input=File.OpenRead(path)) + { + TransferRequire(input.Length>0&&input.Length<=1048576,"TrialJsonSize");bytes=new byte[(int)input.Length];int read=0; + while(read0,"TrialJsonTruncated");read+=n;} + TransferRequire(input.ReadByte()==-1,"TrialJsonChanged"); + } + string text=new UTF8Encoding(false,true).GetString(bytes);var names=new HashSet(StringComparer.OrdinalIgnoreCase); + using(var scan=new JsonTextReader(new StringReader(text)){MaxDepth=4,DateParseHandling=DateParseHandling.None}) + { + TransferRequire(scan.Read()&&scan.TokenType==JsonToken.StartObject,"TrialJsonShape"); + while(scan.Read()) + { + TransferRequire(scan.TokenType!=JsonToken.Comment&&scan.TokenType!=JsonToken.StartObject&&scan.TokenType!=JsonToken.StartArray,"TrialJsonShape"); + if(scan.TokenType==JsonToken.PropertyName)TransferRequire(names.Add((string)scan.Value),"TrialJsonDuplicate"); + } + } + var settings=new JsonLoadSettings{DuplicatePropertyNameHandling=DuplicatePropertyNameHandling.Error}; + using(var reader=new JsonTextReader(new StringReader(text)){MaxDepth=4,DateParseHandling=DateParseHandling.None}) + {var value=JObject.Load(reader,settings);TransferRequire(!reader.Read(),"TrialJsonTrailingData");foreach(var property in value.Properties())TransferRequire(property.Value is JValue,"TrialJsonShape");return value;} + } + private static void TransferValidateRequest(JObject value,string output) + { + string[] names={"SchemaVersion","Kind","Nonce","JobId","Server","DatabaseName","InputCanonicalPath","OutputCanonicalPath","RequiredFreeBytes","CancelBelowBytes","SourceCommit","SourceSnapshotSha256","ExecutableSha256","ProductExecutableSha256","ProductExecutablePath","RegistrationSha256"}; + TransferRequire(value.Count==names.Length,"TrialRequestShape"); + foreach(string name in names)TransferRequire(value[name]!=null&&value[name] is JValue,"TrialRequestShape"); + foreach(string name in new[]{"SchemaVersion","RequiredFreeBytes","CancelBelowBytes"})TransferRequire(value[name].Type==JTokenType.Integer,"TrialRequestType"); + foreach(string name in names)if(name!="SchemaVersion"&&name!="RequiredFreeBytes"&&name!="CancelBelowBytes")TransferRequire(value[name].Type==JTokenType.String,"TrialRequestType"); + TransferRequire((int)value["SchemaVersion"]==2&&(string)value["Kind"]=="Request"&&(string)value["Server"]=="localhost"&&(string)value["DatabaseName"]=="Transfer"&&(string)value["InputCanonicalPath"]==TransferInput&&(string)value["OutputCanonicalPath"]==output&&(long)value["RequiredFreeBytes"]==TransferEntryFree&&(long)value["CancelBelowBytes"]==TransferCancelFree&&(string)value["ProductExecutablePath"]==TransferProductExecutable,"TrialRequestPolicy"); + Guid nonce,job;TransferRequire(Guid.TryParseExact((string)value["Nonce"],"N",out nonce)&&nonce!=Guid.Empty&&Guid.TryParseExact((string)value["JobId"],"D",out job)&&job!=Guid.Empty,"TrialRequestIdentity"); + foreach(string name in new[]{"SourceSnapshotSha256","ExecutableSha256","ProductExecutableSha256","RegistrationSha256"})TransferRequire(System.Text.RegularExpressions.Regex.IsMatch((string)value[name],"^[a-f0-9]{64}$"),"TrialBuildPinInvalid"); + TransferRequire(System.Text.RegularExpressions.Regex.IsMatch((string)value["SourceCommit"],"^[a-f0-9]{40}$"),"TrialBuildPinInvalid"); + } + private static void TransferValidateBuildValues(JObject request,JObject pin,string smokeHash,string productPath,string productHash) + { + TransferRequire(pin["FinalPinAccepted"]!=null&&pin["FinalPinAccepted"].Type==JTokenType.Boolean&&(bool)pin["FinalPinAccepted"]&&pin["TransferTrialRegistered"]!=null&&pin["TransferTrialRegistered"].Type==JTokenType.Boolean&&(bool)pin["TransferTrialRegistered"],"TrialBuildPinNotReleased"); + foreach(string name in new[]{"SourceCommit","SourceSnapshotSha256","ExecutableSha256","ProductExecutableSha256","ProductExecutablePath","RegistrationSha256"})TransferRequire(JToken.DeepEquals(pin[name],request[name]),"TrialBuildPinMismatch"); + TransferRequire(smokeHash==(string)pin["ExecutableSha256"],"TrialExecutableChanged"); + TransferRequire(string.Equals(productPath,TransferProductExecutable,StringComparison.OrdinalIgnoreCase)&&(string)pin["ProductExecutablePath"]==TransferProductExecutable,"TrialProductAssemblyPathChanged"); + TransferRequire(productHash==(string)pin["ProductExecutableSha256"],"TrialProductExecutableChanged"); + } + private static void TransferValidateBuild(JObject request) + { + var pin=TransferReadBounded(TransferPin); + string executable=Process.GetCurrentProcess().MainModule.FileName,product=Path.GetFullPath(typeof(MssqlImportService).Assembly.Location); + TransferPathNoReparse(product); + TransferValidateBuildValues(request,pin,TransferHash(executable),product,TransferHash(product)); + foreach(string manifestName in new[]{"SourceSnapshot","Registration"}) + { + string manifest=(string)pin[manifestName+"Path"];TransferPathNoReparse(manifest); + TransferRequire(TransferHash(manifest)==(string)pin[manifestName+"Sha256"],"TrialSourceSnapshotChanged"); + var files=TransferReadBounded(manifest); + foreach(var property in files.Properties()) + {TransferPathNoReparse(property.Name);TransferRequire(property.Value.Type==JTokenType.String&&TransferHash(property.Name)==(string)property.Value,"TrialPinnedFileChanged");} + } + } + private static string TransferFixedFailure(Exception error) + { + if(error is OperationCanceledException)return "TrialCancelled"; + if(error is InvalidOperationException&&error.Message.Length<160&&System.Text.RegularExpressions.Regex.IsMatch(error.Message,"^(Trial|Monitor|Reparse|COnly)[A-Za-z0-9:]+$"))return error.Message; + return "TrialFailure:"+error.GetType().Name; + } + private static Dictionary TransferTrialSettings() + { + // Explicit ephemeral snapshot, never saved settings or account credentials. No SSH/session borrowing. + return new Dictionary { + { "db_kind", "mssql" }, { "host", "localhost" }, { "port", "" }, + { "initial_database", "Transfer" }, { "trusted_connection", "T" }, { "tls_mode", "Disabled" } + }; + } + private static volatile bool TransferOfflineGuard; + private static int TransferOfflineSqlAttempts; + private static void TransferRequireEntryFreeSpace(long bytes) + { TransferRequire(bytes>=TransferEntryFree,"TrialInsufficientSpace"); } + private static void TransferNoOfflineSql() + { if(TransferOfflineGuard){Interlocked.Increment(ref TransferOfflineSqlAttempts);throw new InvalidOperationException("TrialOfflineSqlOpenBlocked");} } + private static IMssqlImportSession TransferOpenSession() + { TransferNoOfflineSql();return new MssqlImportSqlSession(ConnectionOpenService.OpenMssqlImportSession(TransferTrialSettings())); } + private static SqlConnection TransferOpenProbe() + { + TransferNoOfflineSql(); + var connection = new SqlConnection(ConnectionConfigurationService.BuildMssqlImportConnectionString(TransferTrialSettings())); + try { connection.Open(); return connection; } catch { connection.Dispose(); throw; } + } + private static void TransferRequire(bool condition, string code) + { if (!condition) throw new InvalidOperationException(code); } + private static void TransferPathNoReparse(string path) + { + string full = Path.GetFullPath(path); + TransferRequire(full.StartsWith(@"C:\", StringComparison.OrdinalIgnoreCase) && !full.StartsWith(@"\\", StringComparison.Ordinal), "COnlyPathRequired"); + for (string current = full; !string.IsNullOrEmpty(current); current = Path.GetDirectoryName(current)) + if (File.Exists(current) || Directory.Exists(current)) + TransferRequire((File.GetAttributes(current) & FileAttributes.ReparsePoint) == 0, "ReparsePathRejected"); + string existing=full;while(!Directory.Exists(existing)&&!File.Exists(existing))existing=Path.GetDirectoryName(existing); + var mount=new StringBuilder(1024);var localVolume=new StringBuilder(1024);var actualVolume=new StringBuilder(1024); + TransferRequire(new DriveInfo("C").DriveType==DriveType.Fixed&&GetVolumePathName(existing,mount,1024)&&string.Equals(mount.ToString(),@"C:\",StringComparison.OrdinalIgnoreCase)&&GetVolumeNameForVolumeMountPoint(@"C:\",localVolume,1024)&&GetVolumeNameForVolumeMountPoint(mount.ToString(),actualVolume,1024)&&localVolume.ToString()==actualVolume.ToString(),"TrialLocalCVolumeRequired"); + } + private static int TransferDatabaseId(IDataRecord row,int ordinal) { return Convert.ToInt32(row.GetValue(ordinal),CultureInfo.InvariantCulture); } + private static JObject TransferIdentity(CancellationToken cancellation) + { + using (var connection = TransferOpenProbe()) + using (var command = connection.CreateCommand()) + { + command.CommandTimeout = 5; + command.CommandText = @"SELECT CONVERT(nvarchar(128),SERVERPROPERTY('ServerName')),DB_NAME(),DB_ID(),d.create_date, +CONVERT(nvarchar(128),SERVERPROPERTY('ProductVersion')),d.compatibility_level,d.state_desc,d.is_read_only,d.recovery_model_desc, +ORIGINAL_LOGIN(),CONVERT(nvarchar(128),SUSER_SNAME()),CONVERT(nvarchar(32),CONNECTIONPROPERTY('auth_scheme')),SUSER_SID(ORIGINAL_LOGIN()),SUSER_SID() +FROM sys.databases AS d WHERE d.database_id=DB_ID();"; + cancellation.ThrowIfCancellationRequested(); + using(var registration=cancellation.Register(()=>{try{command.Cancel();}catch{}})) + using (var reader = command.ExecuteReader(CommandBehavior.SingleRow)) + { + TransferRequire(reader.Read(), "TrialIdentityUnavailable"); + string login = reader.IsDBNull(9) ? "" : reader.GetString(9); + string current = reader.IsDBNull(10) ? "" : reader.GetString(10); + var row = new JObject { + ["server"] = reader.GetString(0), ["database"] = reader.GetString(1), ["databaseId"] = TransferDatabaseId(reader,2), + ["createDateTicks"] = reader.GetDateTime(3).Ticks, ["version"] = reader.GetString(4), + ["compatibility"] = Convert.ToInt32(reader.GetValue(5), CultureInfo.InvariantCulture), ["state"] = reader.GetString(6), + ["readOnly"] = reader.GetBoolean(7), ["recovery"] = reader.GetString(8), + ["originalLogin"] = login, ["currentLogin"] = current, ["authScheme"] = reader.IsDBNull(11) ? "" : reader.GetString(11) + }; + TransferRequire(string.Equals((string)row["server"], @"JOHN\MSSQL2019", StringComparison.OrdinalIgnoreCase) && (string)row["database"] == "Transfer", "TrialWrongIdentity"); + TransferRequire(((string)row["version"]).StartsWith("15.", StringComparison.Ordinal) && (int)row["compatibility"] == 150 && + (string)row["state"] == "ONLINE" && !(bool)row["readOnly"] && (string)row["recovery"] == "FULL", "TrialWrongState"); + using(var identity=WindowsIdentity.GetCurrent()) + { + TransferRequire(identity.User!=null&&identity.Name.Split((char)92)[identity.Name.Split((char)92).Length-1].Equals("johnho",StringComparison.OrdinalIgnoreCase),"TrialWrongWindowsIdentity"); + byte[] sid=new byte[identity.User.BinaryLength];identity.User.GetBinaryForm(sid,0); + string expected=BitConverter.ToString(sid).Replace("-","").ToLowerInvariant(); + string original=reader.IsDBNull(12)?"":BitConverter.ToString((byte[])reader.GetValue(12)).Replace("-","").ToLowerInvariant(); + string currentSid=reader.IsDBNull(13)?"":BitConverter.ToString((byte[])reader.GetValue(13)).Replace("-","").ToLowerInvariant(); + TransferValidateSidNames(identity.Name,expected,login,original,current,currentSid,(string)row["authScheme"]); + row["windowsSid"]=expected;row["windowsName"]=identity.Name; + } + return row; + } + } + } + private static void TransferSameIdentity(JObject a, JObject b) + { + foreach (string name in new [] { "server", "database", "databaseId", "createDateTicks", "version", "compatibility", "state", "readOnly", "recovery", "originalLogin", "currentLogin", "windowsSid", "windowsName" }) + TransferRequire(JToken.DeepEquals(a[name], b[name]), "TrialIdentityChanged:" + name); + } + private static void TransferWriteNewReceipt(string path, JObject value) + { + // Append-only bounded stage receipts. No overwrite hides an earlier attempted/failed state. + string json=value.ToString(Formatting.Indented);TransferRequire(new UTF8Encoding(false).GetByteCount(json)<=1048576,"TrialReceiptSize"); + using (var stream = new FileStream(path, FileMode.CreateNew, FileAccess.Write, FileShare.Read)) + using (var writer = new StreamWriter(stream, new UTF8Encoding(false), 4096, true)) + { writer.Write(json); writer.Flush(); stream.Flush(true); } + } + private static void TransferClaimAttempt(JObject attempt) + { + TransferClaimAttemptAt(TransferAttempt,attempt,TransferPathNoReparse); + } + private static void TransferClaimAttemptAt(string path,JObject attempt,Action validate) + { + validate(path);Directory.CreateDirectory(Path.GetDirectoryName(path));validate(Path.GetDirectoryName(path));validate(path); + // CreateNew is the actual race-safe authorization claim; any existing file, even corrupt, stops. + TransferWriteNewReceipt(path, attempt); + // A failure/crash here may consume authority before SQL starts. Preserve marker; no implicit retry. + } + private static string TransferQuote(string name) { return "[" + name.Replace("]", "]]") + "]"; } + + private static JObject TransferIndependentVerification(string output, CancellationToken cancellation) + { + long tables=0, rows=0; + using (var connection = TransferOpenProbe()) + using (var detail = new StreamWriter(new FileStream(Path.Combine(output,"independent-counts.jsonl"),FileMode.CreateNew,FileAccess.Write,FileShare.Read),new UTF8Encoding(false))) + { + int lastObjectId=0; + while (true) + { + cancellation.ThrowIfCancellationRequested(); + int id; string schema, table; + using (var next=connection.CreateCommand()) + { + next.CommandTimeout=5; + next.CommandText="SELECT TOP(1) t.object_id,s.name,t.name FROM sys.tables t JOIN sys.schemas s ON s.schema_id=t.schema_id WHERE t.is_ms_shipped=0 AND t.object_id>@id ORDER BY t.object_id;"; + next.Parameters.Add("@id",SqlDbType.Int).Value=lastObjectId; + using(var reader=next.ExecuteReader(CommandBehavior.SingleRow)) + { if(!reader.Read())break;id=reader.GetInt32(0);schema=reader.GetString(1);table=reader.GetString(2); } + } + using(var count=connection.CreateCommand()) + { + count.CommandTimeout=0;count.CommandText="SELECT COUNT_BIG(*) FROM "+TransferQuote(schema)+"."+TransferQuote(table)+";"; + using(cancellation.Register(()=>{try{count.Cancel();}catch{}})) + { + long n=Convert.ToInt64(count.ExecuteScalar(),CultureInfo.InvariantCulture);rows=checked(rows+n);tables++; + detail.WriteLine(new JObject{["schema"]=schema,["table"]=table,["rows"]=n}.ToString(Formatting.None));detail.Flush(); + } + } + lastObjectId=id; + } + } + TransferRequire(tables==126 && rows==3247785,"TrialIndependentCountMismatch"); + // Catalog inventory is separate evidence; no unsupported equality claim versus absent source object manifest. + using(var connection=TransferOpenProbe()) + using(var command=connection.CreateCommand()) + using(var stream=new StreamWriter(new FileStream(Path.Combine(output,"independent-catalog.jsonl"),FileMode.CreateNew,FileAccess.Write,FileShare.Read),new UTF8Encoding(false))) + { + command.CommandTimeout=5; + command.CommandText=@"SELECT o.type_desc,s.name,o.name,CONVERT(nvarchar(100),NULL),CONVERT(nvarchar(100),NULL) +FROM sys.objects o JOIN sys.schemas s ON s.schema_id=o.schema_id WHERE o.is_ms_shipped=0 +UNION ALL SELECT N'INDEX',s.name,t.name,i.name,CONCAT(N'disabled=',i.is_disabled,N';hypothetical=',i.is_hypothetical,N';unique=',i.is_unique) FROM sys.indexes i JOIN sys.tables t ON t.object_id=i.object_id JOIN sys.schemas s ON s.schema_id=t.schema_id WHERE t.is_ms_shipped=0 AND i.index_id>0 +UNION ALL SELECT N'FOREIGN_KEY',s.name,t.name,f.name,CONCAT(N'disabled=',f.is_disabled,N';not_trusted=',f.is_not_trusted) FROM sys.foreign_keys f JOIN sys.tables t ON t.object_id=f.parent_object_id JOIN sys.schemas s ON s.schema_id=t.schema_id WHERE t.is_ms_shipped=0 +UNION ALL SELECT N'CHECK',s.name,t.name,c.name,CONCAT(N'disabled=',c.is_disabled,N';not_trusted=',c.is_not_trusted) FROM sys.check_constraints c JOIN sys.tables t ON t.object_id=c.parent_object_id JOIN sys.schemas s ON s.schema_id=t.schema_id WHERE t.is_ms_shipped=0 +UNION ALL SELECT N'COMMENT_NAME',CONVERT(nvarchar(128),e.class),CONVERT(nvarchar(128),e.major_id),e.name,CONVERT(nvarchar(100),e.minor_id) FROM sys.extended_properties e;"; + using(var reader=command.ExecuteReader(CommandBehavior.SequentialAccess)) + while(reader.Read()) + { cancellation.ThrowIfCancellationRequested();var record=new JObject();for(int i=0;i=0,"TrialLogUsedInvalid"); + snapshot["Bytes"]=bytes;snapshot["Availability"]="LogUsedAvailable"; + } + } + } + catch + { + // Optional read-only evidence: permission/transport/identity failure remains null with one fixed code. + // Never convert unknown to zero, GRANT permissions, change recovery or replace the primary import failure. + snapshot["Bytes"]=JValue.CreateNull();snapshot["Availability"]="LogUsedUnavailable"; + } + return snapshot; + } + private static void TransferCaptureUsedLog(JObject receipt,string phase) + { + var snapshot=receipt["DatabaseId"]!=null&&receipt["DatabaseCreateDateTicks"]!=null?TransferReadUsedLog(receipt):new JObject{["Bytes"]=JValue.CreateNull(),["Availability"]="LogUsedUnavailable",["Utc"]=DateTime.UtcNow.ToString("o")}; + receipt[phase+"LogUsedBytes"]=snapshot["Bytes"].DeepClone(); + receipt[phase+"LogUsedAvailability"]=(string)snapshot["Availability"]; + receipt[phase+"LogUsedUtc"]=(string)snapshot["Utc"]; + } + private static void TransferValidateSidNames(string windowsName,string sid,string originalName,string originalSid,string currentName,string currentSid,string auth) + { TransferRequire(!string.IsNullOrEmpty(sid)&&sid==originalSid&&sid==currentSid&&string.Equals(originalName,windowsName,StringComparison.OrdinalIgnoreCase)&&string.Equals(currentName,windowsName,StringComparison.OrdinalIgnoreCase)&&(auth=="NTLM"||auth=="KERBEROS"),"TrialWrongLogin"); } + private static void TransferRetainFailure(ref Exception primary,Exception failure,ref string secondary,JObject receipt) + { if(primary==null){primary=failure;receipt["FailureCode"]=TransferFixedFailure(failure);}else secondary=TransferFixedFailure(failure); } + private static JObject TransferReadAllocation() + { + var value=new JObject{["dataAllocatedBytes"]=JValue.CreateNull(),["logAllocatedBytes"]=JValue.CreateNull(),["dbMetricAvailability"]="Unavailable"}; + try{ + using(var connection=TransferOpenProbe())using(var command=connection.CreateCommand()){ + command.CommandTimeout=5;command.CommandText="SELECT SUM(CASE WHEN type=0 THEN CONVERT(bigint,size)*8192 ELSE 0 END),SUM(CASE WHEN type=1 THEN CONVERT(bigint,size)*8192 ELSE 0 END) FROM sys.database_files;"; + using(var reader=command.ExecuteReader(CommandBehavior.SingleRow)){TransferRequire(reader.Read(),"MonitorMetricsMissing");value["dataAllocatedBytes"]=reader.GetInt64(0);value["logAllocatedBytes"]=reader.GetInt64(1);value["dbMetricAvailability"]="AllocationAvailable";} + } + }catch{ } + return value; + } + private static void TransferAddProgress(JObject record,MssqlImportProgress progress) + { + record["fileBytes"]=progress==null?JValue.CreateNull():new JValue(progress.FileBytes); + record["scannedBytes"]=progress==null?JValue.CreateNull():new JValue(progress.ScannedBytes); + record["successfulBytes"]=progress==null?JValue.CreateNull():new JValue(progress.SuccessfulBytes); + record["skippedBytes"]=progress==null?JValue.CreateNull():new JValue(progress.SkippedBytes); + record["processedBytes"]=progress==null?JValue.CreateNull():new JValue(progress.ProcessedBytes); + record["batchNumber"]=progress==null?JValue.CreateNull():new JValue(progress.BatchNumber); + record["remainingSeconds"]=progress==null||!progress.Remaining.HasValue?JValue.CreateNull():new JValue(progress.Remaining.Value.TotalSeconds); + record["estimatedFinishUtc"]=progress==null||!progress.EstimatedFinish.HasValue?JValue.CreateNull():new JValue(progress.EstimatedFinish.Value.ToUniversalTime().ToString("o")); + } + private sealed class TransferTrialMonitor : IDisposable + { + private readonly string output; + private readonly ManualResetEventSlim stop = new ManualResetEventSlim(false); + private readonly ManualResetEventSlim ready = new ManualResetEventSlim(false); + private readonly CancellationTokenSource cancellation; + private readonly string nonce; + private readonly Func freeSpace; + private readonly Func readUsed,readAllocation; + private readonly Func openStream; + private readonly bool pollOperator; + private readonly long baselineCpu; + private readonly Thread thread; + private readonly Stopwatch elapsed = Stopwatch.StartNew(); + private long sequence, peakPrivate; + internal volatile Exception Failure; + internal MssqlImportProgress Latest; // Interlocked reference publication only; immutable DTO. + internal TransferTrialMonitor(string directory,CancellationTokenSource ownedCancellation,string runNonce) + : this(directory,ownedCancellation,runNonce,()=>new DriveInfo("C").AvailableFreeSpace,()=>TransferReadUsedLog(null),TransferReadAllocation, + ()=>new FileStream(Path.Combine(directory,"monitor.jsonl"),FileMode.CreateNew,FileAccess.Write,FileShare.Read),true) { } + internal TransferTrialMonitor(string directory,CancellationTokenSource ownedCancellation,string runNonce,Func free,Func used,Func allocation,Func stream,bool operatorProbe) + { output=directory;cancellation=ownedCancellation;nonce=runNonce;freeSpace=free;readUsed=used;readAllocation=allocation;openStream=stream;pollOperator=operatorProbe;using(var p=Process.GetCurrentProcess())baselineCpu=(long)p.TotalProcessorTime.TotalMilliseconds;thread=new Thread(Loop){IsBackground=true,Name="Transfer trial local evidence"};thread.Start(); } + internal void RequireReady(){TransferRequire(ready.Wait(5000),"TrialMonitorNotReady");TransferRequire(Failure==null,"TrialMonitorIoFailure");} + private void Loop() + { + try + { + using(var writer=new StreamWriter(openStream(),new UTF8Encoding(false))) + { ready.Set();while(!stop.IsSet) + { + using(var process=Process.GetCurrentProcess()) + { + process.Refresh();peakPrivate=Math.Max(peakPrivate,process.PrivateMemorySize64); + var p=Interlocked.CompareExchange(ref Latest,null,null); + var record=new JObject{["schemaVersion"]=1,["utc"]=DateTime.UtcNow.ToString("o"),["sequence"]=sequence++, + ["nonce"]=nonce,["processId"]=process.Id,["elapsedMs"]=elapsed.ElapsedMilliseconds,["cpuMs"]=(long)process.TotalProcessorTime.TotalMilliseconds-baselineCpu, + ["peakSampledPrivateBytes"]=peakPrivate,["peakWorkingSetBytes"]=process.PeakWorkingSet64, + ["successfulBatches"]=p==null?0:p.SuccessfulBatches,["failedBatches"]=p==null?0:p.FailedBatches,["stage"]=p==null?"Scanning":p.Stage.ToString(), + ["cFreeBytes"]=freeSpace()}; + TransferAddProgress(record,p); + if(pollOperator&&File.Exists(Path.Combine(output,"cancel.request"))){record["cancelCode"]="TrialOperatorStop";cancellation.Cancel();} + if((long)record["cFreeBytes"]{args.Cancel=true;cancellation.Cancel();};Console.CancelKeyPress+=cancel; + try + { + var before=TransferIdentity(cancellation.Token);receipt["ServerName"]=(string)before["server"];receipt["DatabaseId"]=(long)before["databaseId"];receipt["DatabaseCreateDateTicks"]=(long)before["createDateTicks"];receipt["WindowsSid"]=(string)before["windowsSid"];receipt["WindowsName"]=(string)before["windowsName"]; + monitor=new TransferTrialMonitor(output,cancellation,(string)envelope["Nonce"]);monitor.RequireReady(); + Action progress=p=>Interlocked.Exchange(ref monitor.Latest,p); + var service=new MssqlImportService(TransferOpenSession,id=>new MssqlImportLog(id,output)); + var request=new MssqlImportRequest(Guid.Parse((string)envelope["JobId"]),TransferInput,"Transfer",false); + MssqlImportResult result; + using(var preflight=service.Preflight(request,progress,cancellation.Token)) + { + TransferRequire(preflight.Passed,"TrialPreflightRejected");var scan=preflight.Scan; + TransferRequire(scan.Script.Kind==MssqlImportScriptKind.NativeSchema&&scan.Script.ExpectedTables==126&&scan.Script.ExpectedRows==3247785&&scan.FileBytes==4478414151L&&scan.MaxBatchCodeUnits<=16777216L&&scan.EncodingName=="UTF-8"&&scan.Script.SourceReadMode=="READ COMMITTED","TrialNativeIdentityMismatch"); + receipt["SourceSha256"]=scan.Sha256.ToLowerInvariant();receipt["ScanBatchCount"]=scan.BatchCount;receipt["MaxBatchCodeUnits"]=scan.MaxBatchCodeUnits;receipt["NativeJobId"]=scan.Script.NativeJobId.ToString();receipt["EncodingName"]=scan.EncodingName;receipt["SourceReadMode"]=scan.Script.SourceReadMode;receipt["LogPath"]=preflight.LogPath; + TransferSameIdentity(before,TransferIdentity(cancellation.Token));TransferValidateBuild(envelope);TransferCaptureUsedLog(receipt,"PreRun"); + cancellation.Token.ThrowIfCancellationRequested();TransferRequire(monitor.Failure==null,"TrialMonitorIoFailure");TransferRequireEntryFreeSpace(new DriveInfo("C").AvailableFreeSpace); + var marker=(JObject)receipt.DeepClone();marker["Kind"]="Attempt"; + // Marker carries only immutable request + actual identity/source chain, not fake terminal proof fields. + foreach(string key in new[]{"Executed","AttemptMarkerDurable","Status","FailureCode","SecondaryFailureCode","LogPath","ScanBatchCount","MaxBatchCodeUnits","NativeJobId","EncodingName","SourceReadMode","PreRunLogUsedBytes","PreRunLogUsedAvailability","PreRunLogUsedUtc"})marker.Remove(key); + TransferClaimAttempt(marker);receipt["MarkerSha256"]=TransferHash(TransferAttempt);receipt["AttemptMarkerDurable"]=true; + var started=(JObject)receipt.DeepClone();started["Kind"]="Started";TransferWriteNewReceipt(Path.Combine(output,"started.json"),started); + cancellation.Token.ThrowIfCancellationRequested();TransferRequire(monitor.Failure==null,"TrialMonitorIoFailure");TransferRequireEntryFreeSpace(new DriveInfo("C").AvailableFreeSpace); + receipt["Executed"]=true; + result=service.Run(request,preflight,progress,cancellation.Token); // Sole authorized Run. No retry path. + } + receipt["Status"]=result.Status.ToString();receipt["FailedBatches"]=result.FailedBatches;receipt["SkippedBytes"]=result.SkippedBytes;receipt["TotalErrors"]=result.TotalErrors;receipt["UncertainBatch"]=result.UncertainBatch.HasValue?new JValue(result.UncertainBatch.Value):JValue.CreateNull();receipt["LastAcknowledgedBatch"]=result.LastAcknowledgedBatch;receipt["SuccessfulBatches"]=result.SuccessfulBatches;receipt["ProcessedBytes"]=result.ProcessedBytes; + receipt["CommandEnded"]=result.Cleanup!=null&&result.Cleanup.CommandEnded;receipt["SessionReleased"]=result.Cleanup!=null&&result.Cleanup.SessionReleased;receipt["TransactionCleanupConfirmed"]=result.Cleanup!=null&&result.Cleanup.TransactionCleanupConfirmed;receipt["CleanupErrors"]=result.Cleanup==null?0:result.Cleanup.Errors.Count; + TransferRequire(result.Status==MssqlImportStatus.Completed&&result.FailedBatches==0&&result.TotalErrors==0&&result.SkippedBytes==0&&result.UncertainBatch==null,"TrialSutNotCompleted"); + TransferRequire((bool)receipt["CommandEnded"]&&(bool)receipt["SessionReleased"]&&(bool)receipt["TransactionCleanupConfirmed"]&&(long)receipt["CleanupErrors"]==0,"TrialCleanupUnconfirmed"); + // Accepted Service Completed semantics include EOF digest/session/transaction/native-LOB checks and log durability. + receipt["ServiceCompletionValidated"]=true;receipt["LogComplete"]=true; + TransferSameIdentity(before,TransferIdentity(cancellation.Token));using(var proof=TransferOpenSession())TransferRequire(proof.ReadTarget(cancellation.Token).MetadataVisible,"TrialIndependentMetadataUnknown"); + var verification=TransferIndependentVerification(output,cancellation.Token);receipt["ActualTables"]=(long)verification["tables"];receipt["ActualRows"]=(long)verification["rows"];receipt["IndependentVerificationCompleted"]=true;receipt["IndependentCatalogCaptured"]=(bool)verification["catalogInventoryCaptured"];receipt["ObjectVerificationBoundary"]="InventoryOnly;SourceDefinitionEqualityUnverified";receipt["FieldHashEqualityVerified"]=false; + TransferSameIdentity(before,TransferIdentity(cancellation.Token));cancellation.Token.ThrowIfCancellationRequested(); + } + catch(Exception error){primary=error;receipt["FailureCode"]=TransferFixedFailure(error);} + finally + { + Console.CancelKeyPress-=cancel; + try{if(monitor!=null){monitor.Dispose();if(monitor.Failure!=null)throw new InvalidOperationException("TrialMonitorIoFailure");}receipt["MonitorComplete"]=monitor!=null;} + catch(Exception error){monitorJoined=false;receipt["MonitorComplete"]=false;TransferRetainFailure(ref primary,error,ref secondary,receipt);} + if(cancellation.IsCancellationRequested&&primary==null){primary=new OperationCanceledException();receipt["FailureCode"]="TrialCancelled";} + TransferCaptureUsedLog(receipt,"Terminal"); + receipt["SecondaryFailureCode"]=secondary;receipt["FinishedUtc"]=DateTime.UtcNow.ToString("o"); + try{TransferWriteNewReceipt(Path.Combine(output,"final.json"),receipt);} + catch(Exception error){if(primary==null)primary=error;else Console.Error.WriteLine("TrialFinalReceiptIncomplete;primary="+TransferFixedFailure(primary)+";secondary="+TransferFixedFailure(error));} + if(monitorJoined)cancellation.Dispose(); + } + if(primary!=null)ExceptionDispatchInfo.Capture(primary).Throw(); + } + private static int TransferGuardChecks; + private static void TransferGuardCheck(bool condition,string message) + { TransferGuardChecks++;Assert(condition,message); } + private static void TransferExpectGuard(Action action,string expected=null) + { + Exception failure=null;try{action();}catch(Exception error){failure=error;} + TransferGuardCheck(failure!=null&&(expected==null||failure is InvalidOperationException&&failure.Message==expected),"Transfer guard expected "+expected+"; actual "+(failure==null?"none":failure.Message)); + } + private sealed class TransferMonitorWriteFault : MemoryStream + { public override void Write(byte[] bytes,int offset,int count){throw new IOException("offline monitor write fault");} } + private static JObject TransferGuardRequest(string output) + { + return new JObject{["SchemaVersion"]=2,["Kind"]="Request",["Nonce"]=new Guid("11111111-1111-1111-1111-111111111111").ToString("N"),["JobId"]="22222222-2222-2222-2222-222222222222",["Server"]="localhost",["DatabaseName"]="Transfer",["InputCanonicalPath"]=TransferInput,["OutputCanonicalPath"]=output,["RequiredFreeBytes"]=TransferEntryFree,["CancelBelowBytes"]=TransferCancelFree,["SourceCommit"]=new string('a',40),["SourceSnapshotSha256"]=new string('b',64),["ExecutableSha256"]=new string('c',64),["ProductExecutableSha256"]=new string('d',64),["ProductExecutablePath"]=TransferProductExecutable,["RegistrationSha256"]=new string('e',64)}; + } + private static void MssqlImportTransferGuards() + { + TransferOfflineGuard=true;TransferOfflineSqlAttempts=0;TransferGuardChecks=0; + string[] environmentNames={"MYSQLPUNK_MSSQL_IMPORT_TRIAL","MYSQLPUNK_MSSQL_IMPORT_SERVER","MYSQLPUNK_MSSQL_IMPORT_DATABASE","MYSQLPUNK_MSSQL_IMPORT_INPUT","MYSQLPUNK_MSSQL_IMPORT_OUTPUT","MYSQLPUNK_MSSQL_IMPORT_INTEGRATION","MYSQLPUNK_MSSQL_IMPORT_AUTH","MYSQLPUNK_MSSQL_IMPORT_PROFILE","MYSQLPUNK_MSSQL_IMPORT_FIXTURES","MYSQLPUNK_MSSQL_IMPORT_MEMORY_SCALE","MYSQLPUNK_MSSQL_IMPORT_RUN_ID","MYSQLPUNK_MSSQL_IMPORT_WIZARD_PREVIEW","MYSQLPUNK_MSSQL_WIZARD_PREVIEW"}; + var saved=new Dictionary();foreach(string name in environmentNames){saved[name]=Environment.GetEnvironmentVariable(name);Environment.SetEnvironmentVariable(name,null);} + string fixture=Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData),"mySQLPunk","ImportTrialGuardFixtures",Guid.NewGuid().ToString("N")); + try + { + TransferExpectGuard(()=>TransferOpenProbe(),"TrialOfflineSqlOpenBlocked"); + MssqlImportTransferTrial(); // absent opt-in returns before source/target IO + TransferGuardCheck(TransferOfflineSqlAttempts==1,"NoRun does not call SQL factory."); + Environment.SetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_TRIAL","1"); + foreach(string name in environmentNames)if(name.EndsWith("INTEGRATION",StringComparison.Ordinal)||name.EndsWith("AUTH",StringComparison.Ordinal)||name.EndsWith("PROFILE",StringComparison.Ordinal)||name.EndsWith("FIXTURES",StringComparison.Ordinal)||name.EndsWith("MEMORY_SCALE",StringComparison.Ordinal)||name.EndsWith("RUN_ID",StringComparison.Ordinal)||name.EndsWith("PREVIEW",StringComparison.Ordinal)) + {Environment.SetEnvironmentVariable(name,"1");TransferExpectGuard(MssqlImportTransferTrial,"TrialForeignOptIn");Environment.SetEnvironmentVariable(name,null);} + Environment.SetEnvironmentVariable("MYSQLPUNK_MSSQL_IMPORT_SERVER","not-authorized");TransferExpectGuard(MssqlImportTransferTrial,"TrialInvocationMismatch"); + TransferRequireEntryFreeSpace(TransferEntryFree);TransferExpectGuard(()=>TransferRequireEntryFreeSpace(TransferEntryFree-1),"TrialInsufficientSpace"); + var identityShape=new DataTable();identityShape.Columns.Add("database_id",typeof(short));identityShape.Rows.Add((short)5); + using(var shape=identityShape.CreateDataReader()){shape.Read();TransferGuardCheck(TransferDatabaseId(shape,0)==5,"Actual SQL2019 smallint DB_ID provider shape converts without InvalidCastException.");} + var request=TransferGuardRequest(fixture);TransferValidateRequest(request,fixture); + foreach(string name in new[]{"Server","DatabaseName","InputCanonicalPath","ProductExecutablePath","OutputCanonicalPath"}) + {var bad=(JObject)request.DeepClone();bad[name]="wrong";TransferExpectGuard(()=>TransferValidateRequest(bad,fixture),"TrialRequestPolicy");} + foreach(string name in new[]{"SourceSnapshotSha256","ExecutableSha256","ProductExecutableSha256","RegistrationSha256","SourceCommit"}) + {var bad=(JObject)request.DeepClone();bad[name]="bad";TransferExpectGuard(()=>TransferValidateRequest(bad,fixture),"TrialBuildPinInvalid");} + foreach(string name in new[]{"SchemaVersion","RequiredFreeBytes","CancelBelowBytes"}) + {var bad=(JObject)request.DeepClone();bad[name]="2";TransferExpectGuard(()=>TransferValidateRequest(bad,fixture),"TrialRequestType");} + var extra=(JObject)request.DeepClone();extra["Extra"]=true;TransferExpectGuard(()=>TransferValidateRequest(extra,fixture),"TrialRequestShape"); + var pin=(JObject)request.DeepClone();pin["FinalPinAccepted"]=true;pin["TransferTrialRegistered"]=true; + TransferValidateBuildValues(request,pin,new string('c',64),TransferProductExecutable,new string('d',64)); + TransferExpectGuard(()=>TransferValidateBuildValues(request,pin,new string('c',64),TransferProductExecutable,new string('f',64)),"TrialProductExecutableChanged"); + TransferExpectGuard(()=>TransferValidateBuildValues(request,pin,new string('c',64),@"C:\other.exe",new string('d',64)),"TrialProductAssemblyPathChanged"); + var unaccepted=(JObject)pin.DeepClone();unaccepted["FinalPinAccepted"]="true";TransferExpectGuard(()=>TransferValidateBuildValues(request,unaccepted,new string('c',64),TransferProductExecutable,new string('d',64)),"TrialBuildPinNotReleased"); + TransferValidateSidNames(@"JOHN\johnho","aabb",@"JOHN\johnho","aabb",@"JOHN\johnho","aabb","NTLM"); + TransferExpectGuard(()=>TransferValidateSidNames(@"JOHN\johnho","aabb",@"OTHER\johnho","aabb",@"JOHN\johnho","aabb","NTLM"),"TrialWrongLogin"); + TransferExpectGuard(()=>TransferValidateSidNames(@"JOHN\johnho","aabb",@"JOHN\johnho","aabb",@"JOHN\johnho","ccdd","NTLM"),"TrialWrongLogin"); + TransferExpectGuard(()=>TransferValidateSidNames(@"JOHN\johnho","aabb",@"JOHN\johnho","aabb",@"JOHN\johnho","aabb","SQL"),"TrialWrongLogin"); + TransferPathNoReparse(fixture);Directory.CreateDirectory(fixture);TransferPathNoReparse(fixture); + foreach(string malformed in new[]{"{\"A\":1,\"A\":2}","{\"A\":1,\"a\":2}","{/*comment*/\"A\":1}","{\"A\":{\"B\":1}}","{\"A\":1} {\"B\":2}"}) + {string file=Path.Combine(fixture,Guid.NewGuid().ToString("N")+".json");File.WriteAllText(file,malformed,new UTF8Encoding(false));TransferExpectGuard(()=>TransferReadBounded(file));} + string oversize=Path.Combine(fixture,"oversize.json");using(var file=new FileStream(oversize,FileMode.CreateNew)){file.SetLength(1048577);}TransferExpectGuard(()=>TransferReadBounded(oversize),"TrialJsonSize"); + string invalid=Path.Combine(fixture,"invalid-utf8.json");File.WriteAllBytes(invalid,new byte[]{255});TransferExpectGuard(()=>TransferReadBounded(invalid)); + string valid=Path.Combine(fixture,"request.json");TransferWriteNewReceipt(valid,request);TransferGuardCheck(JToken.DeepEquals(request,TransferReadBounded(valid)),"Strict request round-trip remains exact."); + string marker=Path.Combine(fixture,"attempt.json");int checks=0;TransferClaimAttemptAt(marker,request,p=>{checks++;TransferPathNoReparse(p);}); + string markerHash=TransferHash(marker);TransferExpectGuard(()=>TransferClaimAttemptAt(marker,request,TransferPathNoReparse));TransferGuardCheck(checks==3&&TransferHash(marker)==markerHash,"Existing marker blocks and remains unchanged."); + string concurrent=Path.Combine(fixture,"raced-attempt.json");int winners=0; + Action claim=()=>{try{TransferClaimAttemptAt(concurrent,request,TransferPathNoReparse);Interlocked.Increment(ref winners);}catch(IOException){}}; + System.Threading.Tasks.Task.WaitAll(System.Threading.Tasks.Task.Run(claim),System.Threading.Tasks.Task.Run(claim));TransferGuardCheck(winners==1,"Atomic CreateNew permits one claimant only."); + Func used=()=>new JObject{["Bytes"]=JValue.CreateNull(),["Availability"]="LogUsedUnavailable",["Utc"]="2026-10-14T00:00:00Z"}; + Func allocation=()=>new JObject{["dataAllocatedBytes"]=0L,["logAllocatedBytes"]=0L,["dbMetricAvailability"]="OfflineFixture"}; + using(var cancellation=new CancellationTokenSource())using(var monitor=new TransferTrialMonitor(fixture,cancellation,"offline-low-space",()=>TransferCancelFree-1,used,allocation,()=>new MemoryStream(),false)) + {monitor.RequireReady();TransferGuardCheck(cancellation.Token.WaitHandle.WaitOne(2000),"Low-space monitor cancels the owned token.");} + using(var cancellation=new CancellationTokenSource())using(var monitor=new TransferTrialMonitor(fixture,cancellation,"offline-io",()=>TransferEntryFree,used,allocation,()=>new TransferMonitorWriteFault(),false)) + {try{monitor.RequireReady();}catch(InvalidOperationException error){TransferGuardCheck(error.Message=="TrialMonitorIoFailure","Monitor reports fixed IO failure.");}TransferGuardCheck(cancellation.Token.WaitHandle.WaitOne(2000),"Monitor IO fault cancels the owned token.");} + var progressRecord=new JObject(); + var longProgress=new MssqlImportProgress(Guid.NewGuid(),7000000001L,MssqlImportStage.Executing,scannedBytes:6000000000L,successfulBytes:4000000000L,skippedBytes:3000000000L,batchNumber:5000000001L); + TransferAddProgress(progressRecord,longProgress); + TransferGuardCheck((long)progressRecord["fileBytes"]==7000000001L&&(long)progressRecord["scannedBytes"]==6000000000L&&(long)progressRecord["processedBytes"]==7000000000L&&(long)progressRecord["successfulBytes"]==4000000000L&&(long)progressRecord["skippedBytes"]==3000000000L&&(long)progressRecord["batchNumber"]==5000000001L,"Streamed progress retains Int64 bytes and batch values."); + TransferGuardCheck(progressRecord["remainingSeconds"].Type==JTokenType.Null&&progressRecord["estimatedFinishUtc"].Type==JTokenType.Null,"Unknown ETA stays null rather than zero or invented finish."); + TransferAddProgress(progressRecord,null); + TransferGuardCheck(progressRecord["fileBytes"].Type==JTokenType.Null&&progressRecord["processedBytes"].Type==JTokenType.Null&&progressRecord["remainingSeconds"].Type==JTokenType.Null,"No snapshot remains unknown."); + Exception original=new InvalidOperationException("TrialSutNotCompleted"),primary=original;string secondary="";var receipt=new JObject();TransferRetainFailure(ref primary,new IOException("secondary"),ref secondary,receipt); + TransferGuardCheck(object.ReferenceEquals(primary,original)&&secondary=="TrialFailure:IOException","Secondary error cannot replace the original failure."); + TransferGuardCheck(TransferOfflineSqlAttempts==1,"All negative/monitor/JSON/marker guards stop before SQL factory; the sole blocked call was deliberate."); + Console.WriteLine("Transfer C# offline guard assertions: "+TransferGuardChecks+"; SQL opens0; fixture="+fixture); + } + finally{foreach(string name in environmentNames)Environment.SetEnvironmentVariable(name,saved[name]);TransferOfflineGuard=false;TransferOfflineSqlAttempts=0;} + } +} diff --git a/tests/SmokeTests.MssqlImport.Wizard.cs b/tests/SmokeTests.MssqlImport.Wizard.cs new file mode 100644 index 0000000..73d4e2f --- /dev/null +++ b/tests/SmokeTests.MssqlImport.Wizard.cs @@ -0,0 +1,693 @@ +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.Drawing; +using System.IO; +using System.Linq; +using System.Text; +using System.Threading; +using System.Windows.Forms; +using mySQLPunk; +using mySQLPunk.lib.MssqlImport; + +public static partial class SmokeTests +{ + private static void ImportWizard_GatesAndInvalidation() + { + using (var fixture = new ImportWizardFixture()) + using (var form = fixture.CreateForm()) + { + form.Show(); Application.DoEvents(); + Assert(fixture.SessionCalls == 0, "Showing selected-target setup must not open DB/session."); + Assert(!IWFind(form, "importSkipErrors").Checked, "Skip failed GO batches defaults off."); + Assert(!IWFind(form, "importConfirmTarget").Checked, "Target confirmation is never assumed."); + Assert(IWFind(form, "importTarget").ReadOnly, "Existing tree target is display-only."); + IWClick(form, "importNext"); + IWPump(() => IWFind