Repository navigation
fix: recover protocol switches and keep user permissions current - #80
Open
marvinli001 wants to merge 2 commits into
Open
marvinli001 wants to merge 2 commits into
marvinli001 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
There are correctness issues in the updated hashing and Xray user-update success accounting that can desynchronize applied state or report successful authorization changes when underlying operations failed.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR restructures node protocol switching into a reconciliation flow between “desired” and “successfully applied” snapshots, aiming to make protocol changes retryable while keeping authorization and certificates consistent during failures.
Changes:
- Introduces a unified target preparation pipeline (shape validation → kernel selection → cert planning/prep → kernel config validation) to avoid partially applied protocol switches.
- Hardens lifecycle recovery: retry/backoff for failed applies and machine node restarts, plus generation-aware handle tracking.
- Tightens certificate defaults and ETag invalidation behavior, with expanded regression coverage and documentation.
File summaries
| File | Description |
|---|---|
| README.md | Adds link to new protocol-switching/recovery documentation. |
| Makefile | Centralizes build/test tags and runs full-suite tests with tags. |
| internal/service/target.go | Adds target preparation pipeline including cert planning and kernel validation pre-bind. |
| internal/service/target_test.go | Tests cert policy matrix, identity naming, and auto-cert isolation. |
| internal/service/realkernel_test.go | Adds real-kernel end-to-end regression scenarios for switching and recovery. |
| internal/service/bootstrap_http_test.go | Regression test for bootstrap retry semantics and user refetch behavior. |
| internal/service/authorization_regression_test.go | Regressions for TLS downgrade prevention and revocation correctness under failed switches. |
| internal/service/authentication_concurrency_test.go | Race-oriented test around HY2 auth vs user refresh replacement. |
| internal/panel/client.go | Adds generation-aware ETag caching to prevent stale in-flight restores. |
| internal/panel/cache_test.go | Tests ETag invalidation vs in-flight responses and concurrency. |
| internal/model/validate.go | Splits validation into kernel-independent “shape” vs kernel-specific validation. |
| internal/model/types.go | Adds deep cloning helpers for NodeSpec and user slices. |
| internal/model/panel.go | Validates panel snapshots with shape-only validation to allow kernel reselection later. |
| internal/model/panel_test.go | Updates tests for shape-vs-kernel validation behavior. |
| internal/model/capability.go | Adds capability/TLS requirement classification and kernel selection logic. |
| internal/model/capability_test.go | Pins compatibility tables and tests kernel selection and config hints. |
| internal/machine/machine.go | Adds restartable backoff lifecycle, serialized discovery triggers, and generation-aware exits. |
| internal/machine/machine_test.go | Covers rebinding, backoff restarts, wakeups, and stale-exit handling. |
| internal/kernel/xray/xray.go | Adds Validate(), synchronous retire/bind sequencing, and UUID-rotation removal handling. |
| internal/kernel/xray/config.go | Avoids mutating desired state when forcing Trojan TLS; always applies stream settings for HTTP. |
| internal/kernel/xray/config_test.go | Adds tests for non-mutation and proxy-protocol behavior on plain HTTP. |
| internal/kernel/singbox/singbox.go | Adds Validate(), inbound removal on switch, rollback logic, and HY2 user replacement strategy. |
| internal/kernel/singbox/reload_test.go | Tests stale inbound removal, rollback on reload failure, and Validate behavior. |
| internal/kernel/singbox/export_test.go | Exposes inbound manager to tests. |
| internal/kernel/kernel.go | Adds Kernel.Validate() and improves UserDiff for UUID rotation. |
| internal/kernel/kernel_test.go | Tests UserDiff rotation vs limit-only changes. |
| internal/controlplane/types.go | Adds Source.ResetCache() for ETag invalidation. |
| internal/controlplane/panel.go | Implements ResetCache via panel client ETag reset. |
| internal/controlplane/machine.go | Implements ResetCache for machine control plane. |
| internal/controlplane/local.go | Implements no-op ResetCache for local control plane. |
| internal/controlplane/panel_test.go | Updates WS translation tests to be kernel-independent where appropriate. |
| internal/controlplane/mailbox.go | Switches mailbox cloning to shared model.CloneNodeSpec(). |
| internal/cert/cert.go | Refactors cert manager into Prepare/Commit workflow; adds reusable self-signed behavior and ACME instance tracking. |
| internal/cert/selfsigned_test.go | Adds comprehensive self-signed persistence/reuse/rotation tests. |
| docs/protocol-switching.md | Documents protocol switching semantics, certificate defaults, lifecycle behavior, and test guidance. |
| .github/workflows/ci.yml | Uses go.mod Go version; broadens push branches and gates publish steps to canonical pushes. |
Review details
Suppressed comments (1)
internal/kernel/xray/xray.go:563
UpdateUserslogs and commitsx.users = userseven whenAddUserfails (and it also reportsadded/removedbased on the diff, not actual successful operations). That can leave the in-memory snapshot out of sync with the real xray UserManager state and reintroduce the misleading “users updated” success scenario from #53 when an add fails.
for _, u := range toAdd {
mu, err := toMemoryUser(proto, nc, u)
if err != nil {
nlog.Core().Warn("xray: skip user in UpdateUsers", "user", u.ID, "error", err)
continue
- Files reviewed: 38/38 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+98
to
+102
| kernelType: selection.Kernel, | ||
| kernelReason: selection.Reason, | ||
| plan: plan, | ||
| configHash: computeConfigHash(spec), | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Changing a node's protocol changes more than its inbound type. A machine reassigned from a VLESS REALITY node to Hysteria2 can successfully stop the old node, then fail the replacement with:
The certificate manager defaults to
none, while the machine registry can retain the exited service's handle. Subsequent discovery then skips it. Related paths also advance the received config hash before application succeeds, validate a new transport against the previous kernel, retain obsolete sing-box inbound tags, or leave Xray listeners accepting while shutdown drains asynchronously.This PR makes protocol changes a retryable reconciliation between the desired snapshot and the successfully applied snapshot. It covers both an existing node ID changing configuration and a machine binding moving to a different node ID.
There are intentional compatibility changes to certificate defaults and Hysteria2 user updates; these are described below rather than hidden in the implementation.
What changes
One preparation/application path
Initialization, WS configuration messages, REST snapshots and retries now feed the same sequence:
The original configured kernel remains the preference. Automatically selecting Xray for XHTTP does not permanently change it. Unsupported combinations are reported instead of silently ignoring settings. No new underlying protocol or dependency version is introduced.
Recoverable lifecycle
Authorization is independent of a failed target
Retaining the old listener during a bad configuration update must not retain old permissions. User removals/rotations are now applied to the kernel actually serving, even if the new target cannot be prepared. If that listener cannot enforce the current user set, it stops and retries instead of indefinitely accepting revoked credentials.
A stale REST user response can no longer undo a newer WS revocation just because the config response was 304/nil. ETag invalidation is synchronized and generation-aware, so an older in-flight HTTP response cannot restore invalidated tags. Bootstrap clears both tags before retrying an incomplete initial snapshot.
UserDiffalso removes the old identity when the same user ID acquires a new UUID, preserving Xray's remove-before-add requirement.Snapshot ownership and Xray user-update failures
The service captures and owns the desired configuration before hashing it. The prepared target hashes the snapshot it actually validates and applies. Deep copies preserve the JSON representation, including empty containers and nested typed nil slices, so duplicate WS messages or REST/304 responses do not trigger repeated application through mismatched hashes.
Xray converts all credentials to add before its first native change. Native additions/removals stop at the first failure, and bookkeeping records the completed operations rather than the requested whole set. A failed native update rebuilds the instance from the latest desired users; if recovery also fails, an error reaches the service, which stops the listener and retains that desired set for retry. Successful recovery is reported as a completed transition; unrecovered failures never advance the applied user hash. Removal errors are considered idempotent only when
GetUserconfirms the account is absent.UUID rotation removes the old identity once before adding its replacement. Service-level delta rotation uses the kernels' full user-set replacement rather than a separate pre-removal whose error could be lost. Limit-only changes retain their existing non-restarting path.
This covers both the snapshot-hash inline comment and the Xray user-update error accounting concern in Copilot's review. Fault-injection tests exercise the production UserManager lookup, partial changes, recovery into a real Xray instance, and failed recovery. Real VLESS clients verify credential rotation and service recovery after rejected credentials. The service-level real-kernel failure case exercises credential conversion; mid-operation AddUser/RemoveUser failures are injected at the Xray adapter layer.
Certificate behavior and compatibility
self/file/content/http/dnsnonecombined with required/requested TLSauto_tls: trueAutomatic certificates use a namespace derived from panel URL and node ID beneath the configured cache root. This prevents entries inheriting one
cert_dirfrom overwriting each other. Explicit certificate paths remain unchanged. Reuse validates the key pair, time validity and names/SANs; keys are persisted with mode 0600.Important consequences:
tls=1never silently becomes a plaintext listener merely because a certificate was omitted. Existing deployments relying on implicit TLS offload need an explicit backend TLS contract. This PR does not infer the presence of a CDN/nginx; panels that couple public and backend TLS may need a separate offload setting.cert_configwhen its UI saysnone, the node cannot distinguish that omission from an absent policy. An explicit opt-out must actually be sent.Hysteria2 concurrency tradeoff
The pinned
sing-quic v0.6.0updater writes authentication maps concurrently with its HTTP/3 authentication reader. A lock around only the caller does not protect those readers.Hysteria2 user changes therefore replace that inbound instead of calling the unsafe updater. Existing QUIC sessions may be interrupted and clients may need to reconnect. Other supported inbounds keep their native user-update path. The regression test permits this documented reconnect window while checking the final authorization state and running under the race detector.
This avoids introducing an additional dependency fork in this PR. A future concurrency-safe dependency API could restore an in-place update for Hysteria2.
Review guide
internal/service/service.go,target.gointernal/model/capability.go,validate.gointernal/cert/cert.gointernal/machine/machine.gointernal/kernel/singbox/singbox.go,internal/kernel/xray/xray.gointernal/controlplane/*,internal/panel/client.godocs/protocol-switching.mdThe patch deliberately excludes downstream provisioning, installation-source, monitoring and release integrations.
Local validation
The final review revision (
dbf1970) was independently verified on macOS arm64 / Go 1.26.8:git diff --checkis clean.go vetwith production tags andgo mod verifypassed.make build-linuxandmake build-linux-arm64passed for both the node and xbctl binaries. These are cross-builds, not VPS acceptance tests.Linux runtime validation of the final commit is recorded separately under CI below. Temporary local build binaries were removed; this independent review did not create a VM or container.
Full suite command (the package set and tags match
make test):The real-kernel regressions exercise:
Fixtures use local servers, temporary certificates, test users and random ports. No production subscription, VPS credentials or external ACME/DNS account was used.
CI
Verified fork CI for
dbf1970: run 34229043987 passed 477 tests/subtests with the race detector on Go 1.26.7 / linux/amd64, plus both Linux amd64 and arm64 build jobs. No individual tests failed or were skipped. Image publication and release jobs were correctly skipped on the fork.The upstream PR run for the same commit is action_required, with no jobs executed. That workflow requires maintainer approval; it has not run the tests or builds.
go.mod.devPRs and run tests/builds on feature branches.Related work
UserDiffremove-on-UUID-rotation condition overlaps that narrow fix and is needed by full-snapshot/rollback authorization here. If fix: replace stale account when user UUID rotates #70 lands first, that overlap can be dropped on rebase.