fix(controller): resolve owners by controller reference and sort Jobs - #73
Merged
Merged
Conversation
Three correctness problems in the path that decides which workload to act on, all of which could make the operator scale down the wrong thing. resolveOwnerWorkload read OwnerReferences[0]. At most one reference has controller: true and the ordering is not defined, so an object carrying an extra non-controller owner could resolve to the wrong parent or to nothing. Uses metav1.GetControllerOf now, at all three levels. The owner UID was never compared. A ReplicaSet or Job deleted and recreated under the same name would be acted on despite not owning the pod. A mismatch is now treated as no owner. allReplicasFailing took the newest Job of a CronJob as the last element of an unsorted List. That only worked while the unix-minute suffixes had a stable digit count. Sorted by creation timestamp instead. The test fixtures set neither controller: true nor matching UIDs, which real Kubernetes always does, so they had to become realistic before the fixes could pass. Adds cases for a non-controller reference listed first, a stale reference, and Jobs returned out of order. Closes #65 Signed-off-by: Simon Lauger <simon@lauger.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three correctness problems in the path that decides which workload to act on. All three can make the operator scale down the wrong thing, which is the worst outcome this operator has, since it never scales anything back up.
OwnerReferences[0]is not necessarily the controller. At most one reference carriescontroller: trueand the ordering is not defined, so an object with an extra non-controller owner could resolve to the wrong parent or to nothing at all. Now usesmetav1.GetControllerOfat all three levels.<name>-<unixminutes>suffixes all had the same digit count and the server returned them in name order. Sorted by creation timestamp now.The fixtures had to become realistic first. They set neither
controller: truenor matching UIDs, both of which real Kubernetes always does, so the corrected code initially failed nine tests. That is worth noting rather than hiding: the tests were passing because they modelled a state the cluster does not produce, which is the same reason the duration bug in #64 went unnoticed.Closes #65
Test plan
ConfigMapstill resolves to its Deployment; a pod whose ReplicaSet was recreated under the same name resolves to nothing; and a CronJob whose Jobs are seeded newest-first still evaluates the newest by timestamp, which in that fixture is the healthy one.make cipasses; coverage 61.2 percent.unparamfindings surfaced while adding fixtures. Rather than silence them, the constant CronJob name moved into atestCronJobNameconstant and the redundant parameters are gone.