Repository navigation
fix: skip an undecodable IPFIX Set instead of dropping the Sets behind it - #332
Open
mikemiles-dev wants to merge 1 commit into
Open
mikemiles-dev wants to merge 1 commit into
mikemiles-dev wants to merge 1 commit into
Conversation
…d it This is the remaining half of #310. The Set-framing check added in #330 could not reach it: a Set whose declared length is sound but whose body cannot be decoded is perfectly well framed, so framing validation passes and many0 still stopped there, dropping every Set behind it while the message reported success. A Template Set declaring more fields than its octets carry silently destroyed the valid Data Sets that followed. The framing pre-pass is what makes the fix possible. Every declared length in the chain is proved before any Set is parsed, so the next Set's offset is known even when the current one fails to decode. Replace many0 with a walk over that validated chain which steps over a failing Set and keeps going. validate_set_framing now returns the offset where the chain ends rather than unit. The walk needs that boundary instead of re-deriving the trailing-padding rule, because a disagreement between the two would be an infinite loop: a tolerated 4-to-7 octet zero tail declares length 0, and splitting at 0 never advances. Reported by the walk's own guard as well, but the boundary removes the duplication that would make them drift. The skipped Set is reported as FlowSetBody::NoTemplate carrying a new NoTemplateInfo::reason of NoTemplateReason::UndecodableBody, so the loss is visible in the data model without consuming the single ParseResult::error slot. reason also names the pre-existing case: a flowset whose template is merely absent reports MissingTemplate. That distinction is load-bearing, not cosmetic. Both V9 and IPFIX cache NoTemplate flowsets for pending-flow replay, and an undecodable Set must never enter that cache -- no template arriving later changes its octets, so it would be replayed and fail forever. Both caching paths now skip anything that is not replayable, and is_replayable() exposes the rule. NoTemplateInfo is #[non_exhaustive], so adding a field is not a breaking change and this stays on the unreleased 1.1.0 alongside the framing fix. A FlowSetBody variant would have forced 2.0.0. The walk also stops early once the decoded-output budget is exhausted. The PostExec hook rejects such a message wholesale, so continuing to decode Sets would spend work on output about to be discarded -- and would let a crafted message with many Sets amplify that work. Three bundled snapshots recorded flowsets: [] before this change and now show the Sets that were being dropped, so none of them was testing what its name claimed. Reviewed each diff before accepting. Verified with the main fuzz target: 2.6M runs, no crashes. That matters here because many0 was replaced with a hand-rolled loop, where a non-advancing split would hang rather than fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The remaining half of #310.
The Set-framing check in #330 could not reach this case. A Set whose declared length is sound but whose body cannot be decoded is perfectly well framed, so framing validation passes and
many0still stopped there, dropping every Set behind it while the message reported success. A Template Set declaring more fields than its octets carry silently destroyed the valid Data Sets that followed. Verified identical before #330 and after it, which is why that PR scoped this out explicitly.The framing pre-pass is what makes the fix possible. Every declared length in the chain is proved before any Set is parsed, so the next Set's offset is known even when the current one fails to decode.
many0is replaced by a walk over that validated chain which steps over a failing Set and keeps going.validate_set_framingnow returns the offset where the Set chain ends rather than unit. The walk needs that boundary instead of re-deriving the trailing-padding rule, because a disagreement between the two would be an infinite loop rather than a wrong answer: a tolerated 4-to-7 octet zero tail declares length 0, and splitting at 0 never advances. The walk carries its own guard as well, but sharing the boundary removes the duplication that would let them drift.Surfacing the skipped Set
The skipped Set is reported as
FlowSetBody::NoTemplatecarrying a newNoTemplateInfo::reasonofNoTemplateReason::UndecodableBody, so the loss stays visible in the data model without consuming the singleParseResult::errorslot.reasonalso names the pre-existing case: a flowset whose template is merely absent reportsMissingTemplate.That distinction is load-bearing, not cosmetic. Both V9 and IPFIX cache
NoTemplateflowsets for pending-flow replay, and an undecodable Set must never enter that cache — no template arriving later changes its octets, so it would be replayed and fail forever. Both caching paths now skip anything not replayable, andNoTemplateInfo::is_replayable()exposes the rule. There is a test for exactly this.The walk also stops early once the decoded-output budget is exhausted. The
PostExechook rejects such a message wholesale, so continuing to decode Sets would spend work on output about to be discarded — and would let a crafted message with many Sets amplify that work.Not a breaking change
NoTemplateInfois#[non_exhaustive], so addingreasonis additive and this stays on the unreleased 1.1.0 alongside the framing fix, where the two halves of #310 now land together. A newFlowSetBodyvariant would have been the cleaner modelling, but that enum is not#[non_exhaustive], so it would have forced 2.0.0 and broken every downstream exhaustive match.Verification
./scripts/check-all.sh— all 7 checks passmany0was replaced with a hand-rolled loop, where a non-advancing split hangs rather than fails.MissingTemplate; an undecodable Set is never cached for replay (and is not counted as a drop either); well-formed messages are untouched with nothing reported; consecutive undecodable Sets are each skipped independently.Three bundled snapshots recorded
flowsets: []before this change and now show the Sets that were being dropped — so none of them was testing what its name claimed, the same pattern #331 found incan_read_ipfix. Each diff was reviewed before accepting rather than blanket-accepted.Closes #310