Repository navigation
fix: Report DNS records that wait on their zone as pending - #657
Merged
Merged
Conversation
vvoytovych
marked this pull request as draft
October 15, 2026 20:42
vvoytovych
marked this pull request as ready for review
October 16, 2026 08:40
vvoytovych
marked this pull request as draft
October 16, 2026 08:46
vvoytovych
force-pushed
the
fix/537-dns-pending-aggregate
branch
from
October 16, 2026 11:44
cd720df to
e355c4f
Compare
vvoytovych
marked this pull request as ready for review
October 18, 2026 15:02
scotwells
approved these changes
Oct 18, 2026
This was referenced Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An HTTPProxy reported every hostname programmed while one of its DNS records was refused (#537). #650 fixed that, and #655 reverted it: a new hostname raised a false "Failed to program DNS" while its DNS zone settled. This change carries the condition again, and the Gateway now reports that wait as pending.
PartialFailurePendingPartialFailurePartialFailurePartialFailurePartialFailurePartialFailurePartialFailureDNSRecordsProgrammedon the HTTPProxyNotes
DNSZoneNotReadywhile its zone settles,Pendingwhile Datum checks its Domain or reads its nameservers. The customer:DomainNotVerifiedorDNSAuthorityMissing.Pendingamong the reasons for a FalseProgrammed, besideAddressNotAssignedfor an address not yet allocated.Pendingonce no DNS record was left, for andrewb3000's review. That drop goes too. Its cause was the Gateway keeping an old condition, which the Gateway now removes.dnsRecordFailedis unchanged. Like its siblingcertificateFailed, it warns when the stored condition is absent or holds another reason.Blast radius
dnsRecordFailedactivity eventalb_activity_events.gointernal/agent/catalog.goPending, as in #650, so a pending HTTPProxy does not read as brokenwaitForHttpProxyReadyrejects False unless the reason isPending; it has no callersPendingDNSRecordProgrammedgetDnsRecordProgrammedIssueand infra'shostname-dns-reapread the HTTPProxy'sOn 2026-10-15, 15 production Gateways kept a False from an earlier state with no hostname left for Datum-managed DNS; they lose it. 8 reported a DNS failure that this change keeps, so each HTTPProxy over one of them raises
dnsRecordFailedonce on upgrade.Test plan
TestGatewayDNSWhileZoneAndDomainSettlereplays the order of infra'shostname-dns-reaptest.TestGatewayDNSRecordsProgrammedByClasscovers each class of hostname, its own reason, and the mixes.TestGatewayWithoutDatumDNSHostnameCarriesNoDNSRecordsProgrammedcovers the Gateway with no hostname left.TestGatewayDNSReturnsAFailedZoneReadcovers the failed read. Each fails onmain; the customer and failure rows pass there.TestHTTPProxyCarriesAPendingDNSRecordsConditionWithoutARecord: the HTTPProxy carries the Gateway's False/Pendingwhile no hostname has a DNS record.TestDNSRecordFailedWarnsAsCertificateFailedDoespins the warning rule: a pending aggregate does not warn. fix: Report DNSRecordsProgrammed on the HTTPProxy as its Gateway does #650's tests pass.go test ./internal/controller/ ./internal/util/dns/ ./internal/agent/ ./api/..., onmainat ff07521. Eighteen planted defects, each caught. Among them: a wait read as a failure, or the reverse. A refused or held zone read as settling. A hostname with the wrong reason.Pendinghiding a failure. A failed read taken for a Domain without a zone. A pending aggregate that warns.Pendingwhile its zone settles. It raises noDNSRecordFailed.Related to #537