Skip to content

docs: document the 1.1.0 error semantics in README #333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mikemiles-dev wants to merge 1 commit into
base: fix/ipfix-skip-undecodable-set
Choose a base branch
from
Open

docs: document the 1.1.0 error semantics in README #333

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 53 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ A Netflow Parser library for Cisco V5, V7, V9, and IPFIX written in Rust. Suppor
- [Pending Flow Caching](#pending-flow-caching)
- [Filtering Versions](#filtering-versions)
- [Error Handling Configuration](#error-handling-configuration)
- [Chained messages and error recovery](#chained-messages-and-error-recovery)
- [Custom Enterprise Fields (IPFIX)](#custom-enterprise-fields-ipfix)
- [Netflow Common](#netflow-common)
- [Re-Exporting Flows](#re-exporting-flows)
Expand Down Expand Up @@ -353,6 +354,20 @@ let parser = NetflowParser::builder()

When using template-based protocols (V9/IPFIX), data packets sometimes arrive before their corresponding template. By default these flows are reported as `NoTemplate` and lost. With pending flow caching enabled, these flows are automatically cached and replayed when the template arrives.

**Not every `NoTemplate` flowset is replayable.** `NoTemplateInfo::reason`
distinguishes the two cases, and `NoTemplateInfo::is_replayable()` answers the
question directly:

| `NoTemplateReason` | Meaning | Cached for replay |
| --- | --- | --- |
| `MissingTemplate` | The template ID is not in the cache. The records may decode once it arrives. | yes |
| `UndecodableBody` | The Set is well framed but its body does not match the octets present. | no |

An `UndecodableBody` flowset is never cached, because no template arriving later
changes its octets — caching it would retry it forever. It is still reported in
`flowsets` so the loss stays visible. Such a Set is skipped rather than ending
the walk, so the Sets behind it still decode.

**Disabled by default.** Enable via the builder pattern:

```rust
Expand Down Expand Up @@ -469,10 +484,18 @@ for packet in result.packets {

// Check for errors
if let Some(e) = result.error {
eprintln!("Error after {} packets: {}", packet_count, e);
eprintln!("{} packets parsed, first error: {}", packet_count, e);
}
```

**`error` is the first error, and packets may follow it.** A malformed IPFIX
message is discarded on its own rather than ending the parse: because the IPFIX
Message Header carries its own length, the parser resumes at the next message in
the datagram (see [Chained messages](#chained-messages-and-error-recovery)). So
`result.packets` can contain packets from both before and after the reported
error, and when several messages fail, `error` holds the first — the one closest
to the root cause.

**iter_packets()** yields `Result<NetflowPacket, NetflowError>` for per-packet error handling:

```rust,ignore
Expand All @@ -485,7 +508,35 @@ for result in parser.iter_packets(&buffer) {
}
```

**Error types**: `Incomplete`, `UnsupportedVersion`, `FilteredVersion`, `ParseError`, `Partial`. All implement `Display` and `std::error::Error`.
**Iteration continues past a recoverable error.** When a discarded IPFIX message
locates the next one, the iterator yields the error and resumes, so a single bad
message does not end the stream. It stops only on an error that leaves no
trustworthy boundary to resume from — a declared length below the 16-octet
Message Header, one that leaves nothing behind it, or a filtered version. v5, v7
and v9 cannot locate the next message after a failure and always stop.

### Chained messages and error recovery

A single datagram may carry several chained IPFIX messages.
[RFC 7011 section 9.1](https://www.rfc-editor.org/rfc/rfc7011.html#section-9.1)
requires discarding a malformed *message*, not the rest of the datagram, so the
parser resumes at the next message whenever the header locates one. One bad
message therefore costs only its own records.

`AutoScopedParser::parse_from_source` resumes the same way, and re-derives the
scope from each surviving message's header, so a message is never decoded with
another observation domain's templates.

**Error types**: `Incomplete`, `UnsupportedVersion`, `FilteredVersion`, `ParseError`, `Partial`, `MalformedSetFraming`, `DecodedOutputLimitExceeded`. All implement `Display` and `std::error::Error`.

`MalformedSetFraming` is reported when an IPFIX Set Header declares a length
that cannot be walked past — below the 4-octet Set Header, or beyond the end of
the Set region. It carries the offending Set's `offset`, its `declared` length
and the octets `available`, plus the discarded message's `export_time`,
`sequence_number` and `observation_domain_id`, so a collector can identify the
exporter at fault and account for the records it did not receive. It is
deliberately distinct from `Partial`, which continues to mean a datagram
truncated in flight.

#### Error Sample Size Configuration

Expand Down
Loading