Skip to content

feat: qualify copilot native project E2E - #527

Draft
777genius wants to merge 49 commits into
feat/project-native-e2e-remaining-20261025from
feat/project-native-copilot-20261025
Draft

777genius wants to merge 49 commits into
feat/project-native-e2e-remaining-20261025from
feat/project-native-copilot-20261025

Conversation

@777genius

@777genius 777genius commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Current complete-cycle implementation and measured correction — 2026-10-28

Published source 59c1cf29842362eb04cdfaa17b3b78ee021ad65b, tree b164999c9ba7f713684a8b7700e44a8c749b72ca.

The bounded User/A/B lifecycle is implemented: 17 public commands, 15 native registry listings, 14 fresh native sessions, 20 provider exchanges and six independently joined project MCP initialize/list/call triples. It covers unchanged reconciliation, truthful intact repair, A-two update, frozen-source missing-component repair, removal/repeated removal, retained data and final native absence. User is the skill-only control; A/B carry managed MCP. The full success contract requires all 60 evidence members, exact source/ownership/data joins, ordinary cleanup and original deadlines.

The first full-cycle run40734424271, on d32d8fd6, fails in the first installed-User catalog. Its native process exits cleanly in 784 ms with 15 exact events, one provider exchange, ordinary shutdown and zero probe bytes. Two independent complete raw audits verify API-bound native/frontend archives, 21/25 members, all 24 frontend stream joins, build/source/pins, User state/source/native/no-data receipts and the full event/provider graph. The only catalog mismatch is exact User plugin commandName: actual native-proof:uapcopilotuser; the old prediction omitted the declared plugin namespace. No project session executed in that failed prefix, and all qualification flags remain false.

The current four-file correction changes only that fixed User expectation in Go/Python. Project, foreign and builtin names remain strict and unqualified. Two independent plan and code reviews passed; literal positive catalogs plus seven hostile namespace/name cases guard the boundary. The reviewed manifest is 57a5ffd6e528ac5a5f7a4d23a6810d5eaa6c0b8b7480889965c0969586eba82a. Both focused Go contracts pass without skips; 39 focused Python contracts pass; affected all-enabled and complete root six-linter gates report zero issues. The full old archive still replays to its identical failed/all-false result. Earlier complete-cycle source gates included 55/55 Go contracts and 57 Copilot Python contracts.

Fresh runtime40741435108, own job123026438903, executed19:19:38–19:21:11 UTC and failed at native:stable-a after intact repair. Five native sessions and two project MCP sessions passed. The sixth process exited0 with27 events; its first seven kinds were exactly [2,0,1,3,4,5,6] (tools_loaded before pending), which the old startup ordering guard refused. Its append-only probe also contains initialize, three tools/list requests and one tools/call in one fresh owned session; the old exact-three-record guard would refuse that next. Two independent full RAW audits cover all36/25 members,24 frontend and103 embedded stream joins, exact source/build/pins,17 proofs,9 public commands,7 registry listings,6 ordinary native processes,9 provider pairs and11 probe records. Manual diagnostic joins of the sixth exchange use the held intact-repair proof; no missing after-native proof or successful lifecycle is invented. All qualification flags stay false. Root audit SHA256 456b6338692b46885f15cecf0252b3223c55c434bdbdba6add0fd3f7805def61; independent audit 9a97bc26ce30275c920a32b7cc3836f4b1be4d1282de98670edd483bdb50286e. Terminal40741435139, source transports, Polyglot Smoke, Skills picker and the Cline read-only gate passed. Core40741435177 and milestone40741435147 remain queued. The current Native Client Runtime40741435080 failed its Darwin inert stream-overflow safety test with PermissionError: EPERM in process-group cleanup before vendor execution. That baseline failure remains unresolved; historical Windows timeout/APFS failures stay recorded separately.

A separate exact-source in-process Cobra measurement confirms repeat-add is singleton/no group with no_change=true, while repair/remove report source local. The minimal three-file Python-reader/contract/documentation correction has two independent code reviews and60 passing focused Python contracts; it accepts all six real hosted public JSON responses against their actual security/proof facts. A separate narrow plan admits only the exact additional startup order and measured three/five-record probe sequences while retaining all identity, timing, catalog, provider, data and cleanup checks. No cache cause, production change or new native permission is inferred. The current actual failed source and archives remain preserved; the reviewed follow-up will require fresh exact-source hosted closure.

Current-source complete-cycle success, two fresh raw audits and applicable baseline gates remain required before qualification or merge. No local vendor execution, release, VS Code change, vendor-authentication or real-model claim is included.

Preserved source-bound implementation and evidence history

Current user-registration observation and pending capture — 2026-10-27

Published source: 922b705222927897fe54bb45e8665d8873fd790d; tree: bceb65e8cb04029e48012742a6975a22c6431481.

The earlier genuine installed-tool readiness success below is preserved. The new separate first-observation mode runs only the existing bounded setup/version/public user-add prefix from outside both projects. A closed metadata-only inventory reports at most sixteen hashed records inside a complete 2 KiB object with an exact omitted count; it does not open unknown file bodies. Original profile preservation stays fatal, the original deadlines and cleanup remain in force, and the closed failed-only reader cannot promote the observation to lifecycle readiness.

Two pre-code plan reviews and independent exact code reviews passed. Focused inert Go, 42 Python contracts, both full-root lint gates and unchanged replay of the prior successful readiness archive passed. Current runtime 40584763860 passed admission and awaits actual native execution. Full user/A/B lifecycle, maintenance, restart/removal and native registry authority remain outstanding; all five broader qualification flags remain false and this PR remains a draft.

Earlier source-bound readiness and historical records

The following observations retain their own source identities and do not qualify the new user-registration capture.

Genuine installed-tool readiness passed — 2026-10-27

Current source: 189e00d364ccde805c9ffc6738e61fc58ae277fc; tree: 3f6dd05e28e7c6ab009c37d41cf804d6c3907743.

The actual runtime run 40572613104, Copilot job 122437209141, passed the selected installed-tool readiness test in 4.92 seconds without a skip. Independent audit verified native artifact 12766238332 (21 closed ordinary members), frontend 12766558145 (25), exact clean source/tree, API hashes, all 24 frontend stream joins, and unchanged admission through the immutable current-source reader.

The captured native command completed in 940 ms. Its 27 frames and 26 unique event IDs join two provider requests, one selected workspace tool, three native result representations, and exactly three probe events: initialize, tools/list, tools/call. The returned result matches all 11 independent probe facts. Source, physical, client and data identities join; all five commands exit zero without capture/context error, and provider return/closure pass.

readiness_observed, runtime_attempt_complete and frontend collection_complete are true. All five broader program qualification flags remain false. Activation remains Prepared; vendor authentication and real-model qualification remain false. Original config/full live filesystem preservation is enforced by the source's live guards; those original bytes are not separately exported.

The remaining full-cycle plan must establish the original user native plugin registry authority, held metadata preservation, supported user/A/B maintenance and removal, and actual user, empty, multi-tool and fallback session grammars. Single-project readiness does not complete those obligations. This PR remains a draft until its applicable lifecycle/review/merge gates pass.

Earlier source-bound continuation records

Why and scope

Copilot's --scope project installation needs source-bound native discovery, installed runtime evidence and preservation of the original user profile. This client PR is stacked on #520 and contains a public-install headless BYOK attempt plus bounded diagnostics. No release is included.

Current source and changes

Published source: 8b2989dd7ede533f4f07f0a50a97cde55488e5ec.
Exact tree: 033710ee5a863f6f48fa8591df2d246f899a577e.

The new diagnostic privately summarizes the already parsed strict-JSON suffix after the measured leading comment prefix. It reports exact before/after/changed field counts and at most three deterministic key hashes, presence-aware JSON types and canonical value hashes, with an explicit omitted count. The complete new suffix diagnostic is bounded to 2 KiB. Unknown names, values, paths and error text are not emitted. Original invalid-JSON status, exact trusted-folder comparison and fatal original-profile digest guard remain unchanged; no vendor-produced configuration becomes a new baseline.

The fixture pins Copilot CLI 1.0.103, installs a real local package through the public UAP CLI, and independently checks its selected source, Prepared binding, native ownership and package/data/cwd authority. It uses the supported keyless OpenAI-compatible loopback provider, disables native updates/telemetry and built-in MCPs, and selects only the exact installed named tool with the existing narrow per-session permission. No vendor credential or injected server definition is used.

Strict provider checks require the selected declaration, matching native tool-result history, fresh probe facts and observed probe events before returning the sentinel. The measured environment expectation preserves literal ${UNKNOWN} while binding actual $HOME expansion to the held original home; authored installed bytes remain unchanged. Foreign declarations, wrong IDs, aliases and malformed JSON remain refusals. Original process supervision, immutable identities, 180-second budget and closed evidence limits remain intact.

The installed snapshots use the existing exact allowed names. Original authored config bytes and their fatal digest guard are preserved. Existing mismatch diagnostics expose only bounded fixed facts and hashes from one held, correlated read of at most 16 KiB.

The latest diagnostic investigates a possible leading comment prefix only when the original baseline is a strict JSON object and the changed observed body has a strict syntax error. It scans at most sixteen LF-terminated leading comment lines and 4096 prefix bytes, then privately applies the unchanged strict JSON parser to the object suffix. Output contains only fixed statuses, counts, exact prefix hash and optional exact object/trusted-folder equality booleans. No raw prefix, config key/value, private path or parser error text is emitted. Even an equal suffix retains the outer invalid-JSON result and the original fatal digest mismatch. This is observation, not config acceptance or general JSONC support.

Actual evidence

Runtime 40456499106, native job 122033009727, exercised preceding source f7421c5f1d6252dee7991b6bc34765663e277936, tree 65345a3d38219264db0f798ef3e3c491a216148a. Its scripted provider-returned flag became true, but the original config digest guard failed. The baseline was 277 bytes of valid JSON; the changed body was 423 bytes and failed strict parsing with a syntax error. Its actual bytes, comment prefix and semantic equivalence remain unobserved.

Native artifact 12712639477 has outer SHA256 9ac80b05aea67079697538fed26bf5ac1986294c747bef647b35a99bb11dceb2; frontend 12712809326 has bc5140f5d6e0aaf57c57e2925b9890baead353261157e149eba4794a928ded36. API sizes/hashes, canonical ordinary closed membership and CRCs, exact source/tree/clean receipts and all 24 frontend command streams were verified. The native 21-member envelope re-admits as failed/incomplete with all five qualification, authentication and real-model flags false.

Current-source runtime 40462465006, job 122051849360, now confirms exactly two leading comment lines (81 bytes, SHA256 a432cd7c816a2ac01cdbd1b028e79f34d591d33e50346e8f2e2470e71eb56487) followed by a valid 342-byte strict JSON object. The suffix is semantically different from the original; the presence-aware trustedFolders comparison is true. The original digest guard still fails, so this is not a formatting-only acceptance. The changed member remains unidentified and requires a separately reviewed finite diagnostic before any interpretation.

Current native artifact 12716315019 has outer SHA256 b1302e041c8c9a28e3e8584c48bed96b7903936dba1289feee22557ebf604893; frontend 12715209304 has 9713c4cd4a3969c3f7c07c40cd860a5af3ac1d7097c80222ec2c1785f6be3668. Root verified API sizes/hashes, 21/25 canonical ordinary unique members and CRCs, exact source/tree/empty patches, all 24 frontend stream joins and current failed/all-false admission. Research 40462465021 completed successfully; it does not qualify the installed lifecycle.

Independent review and verification

Each amendment has separate plan and implementation review. Shell and desktop independently approved the latest plan and full two-file implementation with no findings. Focused Copilot and retained strict-runtime contracts passed in 0.047 seconds, including actual inert child executions proving whitespace and equal-object leading comments still trip the original fatal guard, private canaries, exact trust semantics and hostile prefix/suffix bounds. All-enabled affected lint and unfiltered whole-file size/complexity/architecture lint both reported zero issues. Formatting, whitespace and the actual prior failed-envelope replay pass. No vendor process or positive hosted eligibility guard ran locally.

The earlier parent frontend/Darwin cleanup alignment, measured environment correction, strict JSON refusal categories and independent source/receipt/probe checks remain preserved.

The suffix member slice received two independent plan and code approvals. Focused Copilot/shared inert contracts passed in 0.054 seconds, including the actual original-profile fatal child test. Both all-enabled affected lint and the unfiltered whole-file gate reported zero issues. Actual protected-file tests cover independent hashes for every JSON type, absent versus null, numeric token differences, reordered 100-member changes, deterministic selection/omission and the complete 2 KiB bound. The retained 9cd 21-member envelope and all 24 frontend stream joins were revalidated unchanged as failed/incomplete with all qualification flags false. A fresh hosted capture must identify the changed member before any separately reviewed fixture correction.

Historical evidence and remaining gate

Run 40363828314, job 121755712251 at source 4566b551b7c225a30b40c149a44f69d96cc2d0ce first recorded the protected-config mismatch. Only its closed frontend artifact 12678603433 was available, SHA256 1020004a73f256b40287914bdb3925f0899997f7226cfbac72c80f6613ee96a4; the rejected raw archive was unavailable. The source-defined snapshot naming defect was fixed without widening the member set. Run 40389710837 and run 40408790152 retain their own historical sources and supply no current-head qualification.

The native JSONL event/session grammar, original-config change and complete user/A/B install, reconciliation, update, repair, fresh restart, removal and repeated-removal sequence still require genuine evidence and independent review. The first attempt remains deliberately unqualified even if provider/probe exchange completes; provider IDs cannot replace native session identity. All five lifecycle/restart/MCP flags remain false. Vendor authentication and real-model use remain unqualified; the scripted provider supplies neither. Fresh current-source checks and the complete applicable evidence are required before merge.

Integrates independently reviewed leaf 8fe563d87857837b0c5d1237c85c48c32fb79b90. Lifecycle and MCP runtime qualification remain separate.
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…agnostics

Emit installed snapshots under the existing exact closed artifact names. Keep the original config baseline fatal and add only bounded hash/type/status facts for a changed config; never export raw profile values or adopt a later baseline.

Align the reviewed shared runtime frontend and Darwin capture cleanup. Native arguments, provider behavior, Copilot artifact membership, deadlines, and qualification remain strict.

Two independent plan/code reviews and exact integration review passed. Focused contracts, actual fatal-guard regression, affected lint and all 18 shared admission tests pass. Fresh genuine native evidence remains required.

Copy link
Copy Markdown
Owner Author

@coderabbitai full review

Owner requested independent review before merge. Observed head: 5afd897d4b10de6286d1e0c97d358ec4e15f0250, stacked on #520. Review only: no autofix, branch edits, ready-state transition, merge or release.

Prioritize original-user .copilot/config.json preservation, bounded hash/type-only mismatch diagnostics, producer/closed-artifact member parity, strict BYOK provider/tool-result and native/probe identity validation, and receipt-selected ownership. Do not accept a native-mutated config as a replacement baseline or conflate provider IDs with native session proof. Preserve offline mode, exact permissions, hosted-only/pin/process/cleanup/budget/redaction guards and false unproved qualification flags. Report concrete findings anchored to the reviewed source.

@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.


Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 59 minutes.

@777genius 777genius left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent current-source evidence review for f7421c5 / tree65345a3d: the measured native environment correction is now observed to work; the separate original-profile guard remains a real NO-GO. This is not full lifecycle or merge approval.

I read the accepted plan and the finite diagnostic implementation, then inspected genuine run40456499106/job122033009727 raw logs: GitHub-hosted Azure Ubuntu24.04.5 image20261012.334.1, exact clean source f7421c5, current-source build and admission. Native artifact12712639477 was independently downloaded and rehashed to 9ac80b05aea67079697538fed26bf5ac1986294c747bef647b35a99bb11dceb2; unique bounded ordinary ZIP entries and CRCs checked. No vendor ran locally.

The native process exited0 in1048ms with empty stderr and no command/context error. The 27 actual JSONL records include selected workspace MCP connection and skill, one tool request/completion, the final expected sentinel and exitCode0. I independently compared native tool.execution_complete.result.content, the second provider-request tool message and the actual probe tools/call facts: all match, including nonce371b69d91b28629f3aaf48c88debe122, probe PID6078/session6078-1793081225666287339, selected managed/data paths, argv and the exact original-HOME native expansion. initialize/tools-list/tools-call share that probe identity. Provider returned=true on this source; this was false before the env correction. The native session ID02dca00a-4586-4f55-9787-a86f6628a97a remains distinct from the probe session.

The run still correctly fails protected original-config equality. New diagnostic proves baseline277bytes is syntactically valid JSON, observed423bytes is not, with observed reason=syntax (typed json.SyntaxError), not merely an unknown field or duplicate-key rejection. Expected SHAa8a85b0ae210181acfedbf8d99319c00c4dde6792c3f9d7d04d7a9bb57983f62; observed SHA6ebd1de8072bf24f5d3ba857cf680caf17a19bd6eb57bb1b7591de9226a45eb0. Raw changed profile bytes are intentionally absent. This does NOT yet prove JSONC/comments, BOM, migration, particular changed keys or a safe new baseline. The guard and all five false qualification flags must remain.

Current official config-directory documentation distinguishes runtime-managed config.json from user settings.json and explicitly documents JSONC only for settings.json (https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-config-dir-reference). This is a research lead, not version1.0.103 source proof or permission to relocate trust, parse permissively or ignore profile changes. The next independently reviewed amendment needs exact pinned writer/initialization evidence or a bounded non-value diagnostic before deciding how to preserve the original authored authority. Do not replace the held baseline with native-written bytes or exempt arbitrary runtime keys. No branch/flag/CI/release changes made here.

Copy link
Copy Markdown
Owner Author

Fresh independent evidence on current9cd2 (not a repeat of the earlier unobserved-prefix hypothesis): genuine run40462465006 native artifact12716315019 independently downloaded and SHA256 verified b1302e041c8c9a28e3e8584c48bed96b7903936dba1289feee22557ebf604893;21 unique bounded ordinary members/CRCs checked; source9cd2/treeb35dc7257820ca1817ea9b0093cad0c56fd29bfc.

The prefix diagnostic now records actual status=captured,prefix_bytes=81,comment_lines=2,prefix_sha256=a432cd7c816a2ac01cdbd1b028e79f34d591d33e50346e8f2e2470e71eb56487,suffix_bytes=342,suffix_json_reason=valid,semantically_equal=false,trusted_folders_match=true. Original baseline is277bytes; native observed423bytes. Thus leading comments are now genuinely observed, but they are NOT the entire change: the strict parsed suffix also differs semantically from the authored object. Do not strip the prefix and adopt the suffix or call this formatting-only. The exact changed member/value remains unobserved in the closed artifact.

Native exit0 in944ms, empty stderr, no command/context error and provider_returned=true persist on this source. The original protected-config refusal correctly keeps runtime_attempt_complete=false and all5qualification/auth/model flags false. Actual original SHA198b73f35ba1108dfe7c4f493a52f4e77e9537b5da26f4b78985e03303451feb;observedSHAc7b61763c29866a12fab079397bbabfde8743a086c1470f3e8f260f52d7049c3. Any subsequent initialization/schema amendment still needs exact pinned writer or finite held-object delta evidence, independent reviews and a fresh run; preserved trust alone does not establish complete user-profile preservation. No code/ref/CI/qualification changes here.

codex and others added 25 commits October 27, 2026 20:42

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants