Skip to content

fix: skip an undecodable IPFIX Set instead of dropping the Sets behind it - #332

Open
mikemiles-dev wants to merge 1 commit into
mainfrom
fix/ipfix-skip-undecodable-set
Open

mikemiles-dev wants to merge 1 commit into
mainfrom
fix/ipfix-skip-undecodable-set

Conversation

@mikemiles-dev

Copy link
Copy Markdown
Owner

The remaining half of #310.

The Set-framing check in #330 could not reach this case. A Set whose declared length is sound but whose body cannot be decoded is perfectly well framed, so framing validation passes and many0 still stopped there, dropping every Set behind it while the message reported success. A Template Set declaring more fields than its octets carry silently destroyed the valid Data Sets that followed. Verified identical before #330 and after it, which is why that PR scoped this out explicitly.

The framing pre-pass is what makes the fix possible. Every declared length in the chain is proved before any Set is parsed, so the next Set's offset is known even when the current one fails to decode. many0 is replaced by a walk over that validated chain which steps over a failing Set and keeps going.

validate_set_framing now returns the offset where the Set chain ends rather than unit. The walk needs that boundary instead of re-deriving the trailing-padding rule, because a disagreement between the two would be an infinite loop rather than a wrong answer: a tolerated 4-to-7 octet zero tail declares length 0, and splitting at 0 never advances. The walk carries its own guard as well, but sharing the boundary removes the duplication that would let them drift.

Surfacing the skipped Set

The skipped Set is reported as FlowSetBody::NoTemplate carrying a new NoTemplateInfo::reason of NoTemplateReason::UndecodableBody, so the loss stays visible in the data model without consuming the single ParseResult::error slot. reason also names the pre-existing case: a flowset whose template is merely absent reports MissingTemplate.

That distinction is load-bearing, not cosmetic. Both V9 and IPFIX cache NoTemplate flowsets for pending-flow replay, and an undecodable Set must never enter that cache — no template arriving later changes its octets, so it would be replayed and fail forever. Both caching paths now skip anything not replayable, and NoTemplateInfo::is_replayable() exposes the rule. There is a test for exactly this.

The walk also stops early once the decoded-output budget is exhausted. The PostExec hook rejects such a message wholesale, so continuing to decode Sets would spend work on output about to be discarded — and would let a crafted message with many Sets amplify that work.

Not a breaking change

NoTemplateInfo is #[non_exhaustive], so adding reason is additive and this stays on the unreleased 1.1.0 alongside the framing fix, where the two halves of #310 now land together. A new FlowSetBody variant would have been the cleaner modelling, but that enum is not #[non_exhaustive], so it would have forced 2.0.0 and broken every downstream exhaustive match.

Verification

  • ./scripts/check-all.sh — all 7 checks pass
  • Main fuzz target — 2.6M runs, no crashes. This matters here specifically: many0 was replaced with a hand-rolled loop, where a non-advancing split hangs rather than fails.
  • 6 new tests: the Data Set behind an undecodable Set survives; the skipped Set is reported as undecodable; a missing template still reports MissingTemplate; an undecodable Set is never cached for replay (and is not counted as a drop either); well-formed messages are untouched with nothing reported; consecutive undecodable Sets are each skipped independently.

Three bundled snapshots recorded flowsets: [] before this change and now show the Sets that were being dropped — so none of them was testing what its name claimed, the same pattern #331 found in can_read_ipfix. Each diff was reviewed before accepting rather than blanket-accepted.

Closes #310

…d it

This is the remaining half of #310. The Set-framing check added in #330
could not reach it: a Set whose declared length is sound but whose body
cannot be decoded is perfectly well framed, so framing validation passes
and many0 still stopped there, dropping every Set behind it while the
message reported success. A Template Set declaring more fields than its
octets carry silently destroyed the valid Data Sets that followed.

The framing pre-pass is what makes the fix possible. Every declared length
in the chain is proved before any Set is parsed, so the next Set's offset
is known even when the current one fails to decode. Replace many0 with a
walk over that validated chain which steps over a failing Set and keeps
going.

validate_set_framing now returns the offset where the chain ends rather
than unit. The walk needs that boundary instead of re-deriving the
trailing-padding rule, because a disagreement between the two would be an
infinite loop: a tolerated 4-to-7 octet zero tail declares length 0, and
splitting at 0 never advances. Reported by the walk's own guard as well,
but the boundary removes the duplication that would make them drift.

The skipped Set is reported as FlowSetBody::NoTemplate carrying a new
NoTemplateInfo::reason of NoTemplateReason::UndecodableBody, so the loss
is visible in the data model without consuming the single
ParseResult::error slot. reason also names the pre-existing case: a
flowset whose template is merely absent reports MissingTemplate.

That distinction is load-bearing, not cosmetic. Both V9 and IPFIX cache
NoTemplate flowsets for pending-flow replay, and an undecodable Set must
never enter that cache -- no template arriving later changes its octets,
so it would be replayed and fail forever. Both caching paths now skip
anything that is not replayable, and is_replayable() exposes the rule.

NoTemplateInfo is #[non_exhaustive], so adding a field is not a breaking
change and this stays on the unreleased 1.1.0 alongside the framing fix.
A FlowSetBody variant would have forced 2.0.0.

The walk also stops early once the decoded-output budget is exhausted.
The PostExec hook rejects such a message wholesale, so continuing to
decode Sets would spend work on output about to be discarded -- and would
let a crafted message with many Sets amplify that work.

Three bundled snapshots recorded flowsets: [] before this change and now
show the Sets that were being dropped, so none of them was testing what
its name claimed. Reviewed each diff before accepting.

Verified with the main fuzz target: 2.6M runs, no crashes. That matters
here because many0 was replaced with a hand-rolled loop, where a
non-advancing split would hang rather than fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant