Skip to content

Stale-cache advertisement can stay advertised after its NetworkRule is deleted #763

Closed

Description

@privateip

Summary

NetworkRuleReconciler.reconcileDelete lists a deleting rule's BGPAdvertisements by galactic.datum.net/network-rule label from the informer cache, deletes them, and removes the finalizer. A gateway node working from a stale cache can create an advertisement for the rule after that list runs.

Since #758 that advertisement carries an owner reference to the NetworkRule, so Kubernetes garbage collection deletes it once the rule is gone. Until garbage collection runs, the route stays advertised while no node serves the rule. Traffic to the VIP is dropped for that window, typically a few seconds.

Where

  • reconcileDelete: the cached list at line 216 and the finalizer removal at line 242.
  • applyBGPAdvertisements skips a rule with a deletion timestamp, but a cache that has not yet seen the timestamp does not.

Proposed fix

Before removing the finalizer, re-list the rule's advertisements through an uncached reader (the manager's API reader). If any exist, delete them and requeue instead of removing the finalizer.

Follow-up from #715 (#758).

Activity

  1. added
    bugSomething isn't working
    needs triageIssues that need to be triaged before implementing
    on Oct 7, 2026
  2. removed
    needs triageIssues that need to be triaged before implementing
    on Oct 7, 2026
  3. added theissue type on Oct 7, 2026
  4. self-assigned this
    on Oct 7, 2026
  5. added 2 commits that reference this issue on Oct 7, 2026
    9ed8199
    573ad37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinggalactic-gatewayRelated to galactic-gateway

Type

Fields

Priority

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions