Skip to content

fix(manage): retry dataplane setup after restart - #1939

Open
AnouarMohamed wants to merge 2 commits into
kmesh-net:mainfrom
AnouarMohamed:fix/1204-authz-restart-reconcile
Open

AnouarMohamed wants to merge 2 commits into
kmesh-net:mainfrom
AnouarMohamed:fix/1204-authz-restart-reconcile

Conversation

@AnouarMohamed

@AnouarMohamed AnouarMohamed commented Aug 21, 2026

Copy link
Copy Markdown

What type of PR is this?

/kind bug

What this PR does / why we need it:

Informer replay after a Kmesh restart treated pod dataplane setup as a best-effort side effect: network namespace, XDP, and TC failures were discarded, and the managed annotation could be queued before those programs were attached. A transient restart-time failure could therefore leave a pod marked as managed without the authorization dataplane.

This change:

  • makes pod dataplane enrollment and removal explicit reconciliation actions;
  • propagates network namespace, control-state, XDP, and TC errors through the controller queue;
  • retries quickly first and then continues periodic retries instead of permanently giving up;
  • revalidates the current enrollment intent before every retry;
  • updates the managed annotation only after dataplane reconciliation succeeds;
  • adds deterministic coverage for each setup/teardown failure, transient recovery, recovery after the fast retry budget, and post-restart XDP attachment.

Which issue(s) this PR fixes:
Fixes #1204

Special notes for your reviewer:

verified with:

  • make gen-check
  • repository-configured golangci-lint for ./pkg/controller/manage/...
  • go vet ./pkg/controller/manage
  • the complete manage package under -race -gcflags=all=-l in a privileged container
  • focused retry tests repeated 20 times and race-tested 5 times
  • integration test compilation with the integ build tag
  • make copyright-check from a clean checkout

The full repository test target was also run. The affected manage package passed; the overall target encountered unrelated environment-sensitive failures in existing authorization/BPF, cache, CNI, and IPsec tests (including the host kernel rejecting the authorization BPF program at the verifier instruction limit and absent mounted CNI fixtures).

Does this PR introduce a user-facing change?:

Fix authorization enforcement recovery after Kmesh daemon restarts by retrying pod dataplane attachment failures.

Signed-off-by: AnouarMohamed <m.anouar@mundiapolis.ma>
Copilot AI lite review requested due to automatic review settings August 21, 2026 01:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@kmesh-bot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign supercharge-xsy for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kmesh-bot

Copy link
Copy Markdown
Collaborator

Welcome @AnouarMohamed! It looks like this is your first PR to kmesh-net/kmesh 🎉

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 40.08%. Comparing base (c6357aa) to head (6a90ec9).
⚠️ Report is 14 commits behind head on main.

Files with missing lines Coverage Δ
pkg/controller/manage/manage_controller.go 64.81% <100.00%> (+12.17%) ⬆️

... and 3 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ec501a9...6a90ec9. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: AnouarMohamed <m.anouar@mundiapolis.ma>
Copilot AI review requested due to automatic review settings August 21, 2026 11:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@AnouarMohamed

Copy link
Copy Markdown
Author

Hi, it looks like the two failed E2E jobs both hit the same timeout in TestMixNsAndServiceWaypoint (X-Request-Id was not set):

TestKmeshRestart passed in both jobs, including the new XDP checks, and the other four E2E matrices passed as well. The latest commit only adds unit tests, so these seem unrelated to this PR.

I don't have permission to rerun the workflows from my fork. Could a maintainer please rerun these two failed jobs when you have a chance? Thanks!

@AnouarMohamed

Copy link
Copy Markdown
Author

These failures look like the same issue already tracked in #1859.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Offload authorization may fail after Kmesh restart

3 participants