fix: allow opt-in recovery of stale IPv4 fragment IDs

README.md

@@ -156,6 +156,7 @@ dpp --report-format json input.pcap output.csv > dpp-summary.json
 | `--max-dns-compression-jumps <N>` | Maximum compression-pointer jumps per DNS name; defaults to `32`; `0` disables the jump limit |
 | `--allow-fragments` | Opt in to matching complete queries with first IPv4 response fragments when the DNS header and question are present; no reassembly |
 | `--full-fragments` | Reassemble IPv4 fragments into complete UDP datagrams; also enables `--allow-fragments` for incomplete responses |
+| `--ipv4-fragment-reuse-gap-ms <MS>` | Recover a pending IPv4 ID when a new first fragment is separated by more than this capture-time interval; requires `--full-fragments`, conflicts with `--monotonic-capture`; range `1..=9223372036854775`, unset by default |
 | `--anonymize <path>` | Path to the pseudonymization key file |
 | `-h, --help` | Print help |
 | `-V, --version` | Print version |
@@ -182,6 +183,7 @@ provide supported per-core thread pinning through the affinity backend.
 | `DPP_MAX_DNS_COMPRESSION_JUMPS` | Maximum compression-pointer jumps per DNS name when `--max-dns-compression-jumps` is not set; defaults to `32`; `0` disables the jump limit |
 | `DPP_ALLOW_FRAGMENTS` | Enable first-fragment IPv4 response matching without reassembly. |
 | `DPP_FULL_FRAGMENTS` | Enable IPv4 reassembly and first-fragment response matching. |
+| `DPP_IPV4_FRAGMENT_REUSE_GAP_MS` | IPv4 pending-ID recovery interval in milliseconds when `--ipv4-fragment-reuse-gap-ms` is absent; same range and mode requirements as the CLI option; unset by default. |
 | `DPP_ANONYMIZE` | Path to the key file used for pseudonymization |
 | `DPP_SILENT` | Suppress info-level log output |
 
@@ -284,10 +286,27 @@ mode. Fallback uses the first fragment's timestamp.
 Fragmented queries need complete reassembly;
 overlapping, inconsistent, or UDP-length-mismatched fragment sets are rejected.
 
+In ordinary full-fragment mode, an unresolved or rejected fragment set can retain its IPv4 ID
+until capacity eviction or EOF. For captures where a chosen time interval can separate reused
+IDs, opt in with `--ipv4-fragment-reuse-gap-ms <MS>` or `DPP_IPV4_FRAGMENT_REUSE_GAP_MS`.
+The option requires full reassembly and cannot be combined with monotonic-capture mode.
+
+Recovery starts only when a valid DNS first fragment arrives more than the configured interval
+after the maximum timestamp admitted to the old pending set, including duplicates and rejected
+fragments. It also works after a collision has rejected that set. The old usable response prefix
+may fall back once. The retained cutoff is the old maximum plus the interval; fragments with
+timestamps at or before it are ignored while the key's history remains retained. Completed
+datagrams alone do not trigger recovery, and the interval does not expire a set by itself.
+
+The interval expresses an operator's hypothesis about capture timestamps and generation
+separation; it is not a universal maximum datagram lifetime. Recovery does not guarantee
+tail-before-first handling across generations, and protection against old replay ends after all
+bounded history for the key has been evicted. Without the option, existing behavior is unchanged.
+
 Duplicate suppression uses bounded history and compares the fragment key and UDP payload bytes:
 
-- A reused IPv4 ID with different content can form a new datagram, even with an identical first
-  fragment.
+- After completion, a reused IPv4 ID with different content can form a new datagram, even with an
+  identical first fragment.
 - A fully identical new datagram, or an incomplete one whose observed fragments match a recent
   completion, may be suppressed because it is indistinguishable from a duplicate.
 - A separate bounded history drops tails after a non-DNS first fragment. If a new DNS datagram

docs/architecture.md

@@ -356,8 +356,8 @@ The datagram bounds must agree:
 
 Incomplete responses may fall back to first-fragment inference on capacity eviction or EOF.
 With monotonic capture, entries can also expire after the match timeout. Without it, timestamp
-gaps between fragments of the same datagram do not expire the entry; capacity and EOF still bound
-its lifetime.
+gaps alone do not expire an entry; capacity and EOF still bound its lifetime. Explicit pending-ID
+recovery is described below.
 
 - Complete datagrams use the final fragment's (`MF=0`) capture timestamp.
 - Inferred responses use the first fragment's timestamp.
@@ -372,7 +372,7 @@ no record.
 ```mermaid
 flowchart TB
     Input["Packet batches"] --> Reassembly["IPv4 reassembly<br/>pending fragment sets"]
-    Resolve["EOF / capacity pressure<br/>expiry only in monotonic mode"] -.-> Reassembly
+    Resolve["EOF / capacity pressure<br/>monotonic expiry / opted-in first-led reuse"] -.-> Reassembly
     Reassembly -->|"unfragmented packets, complete datagrams,<br/>or fallback prefixes"| Ready["Retained ready packets<br/>held while fragments are unresolved"]
     Ready -.->|"backlog limit exceeded:<br/>resolve pending sets"| Reassembly
     Ready -->|"after pending sets are resolved"| Route["Sort by timestamp and ordinal<br/>validate packet metadata and route"]
@@ -395,8 +395,8 @@ Reassembly keeps a bounded history of completed datagrams. Fragment keys and UDP
 are compared before suppressing repeated complete datagrams or prefix fallback matching a recent
 completion.
 
-A reused IPv4 ID with different content can form a new datagram, even if its first
-fragment is identical. An incomplete new datagram whose observed fragments match a recent
+After completion, a reused IPv4 ID with different content can form a new datagram, even if its
+first fragment is identical. An incomplete new datagram whose observed fragments match a recent
 completion is indistinguishable from a duplicate and may be suppressed, as may a fully identical
 new datagram.
 
@@ -411,6 +411,36 @@ The canonical VLAN context is part of each fragment key: different tagged segmen
 assembled together, while PCP or DEI changes do not split a datagram. All reassembly state must
 remain bounded in memory.
 
+### Opt-in pending-ID recovery
+
+`--ipv4-fragment-reuse-gap-ms` (environment: `DPP_IPV4_FRAGMENT_REUSE_GAP_MS`) supplies an explicit
+capture-time interval for recovering an unresolved or rejected IPv4 fragment set. It requires
+`--full-fragments`, conflicts with `--monotonic-capture`, and is unset by default. CLI values take
+precedence over the environment; the supported range is `1..=9223372036854775` milliseconds.
+
+The routing stage retains one reassembler across batches. Each pending candidate records the
+maximum timestamp of admitted fragments, including duplicate and rejection events. A valid DNS
+first fragment here means a structurally valid IPv4/UDP first fragment with a port 53 endpoint;
+DNS semantic validation remains downstream. Interval-based recovery starts a new candidate only
+when its timestamp is strictly greater than the old maximum plus the configured interval.
+Identical first-fragment bytes can also initiate this recovery. Completed and non-DNS history
+alone cannot initiate it.
+
+Recovery removes the old segments and completion history. An old usable first-response prefix
+passes through the existing fallback path once; rejected candidates and candidates whose bytes
+still match a recent completion produce no prefix. A retained cutoff, equal to the old maximum
+plus the interval, rejects first fragments, tails, and non-DNS first fragments at or before it
+before they can change state. The existing pending, completed, and non-DNS records carry this
+cutoff, taking the maximum when boundaries meet. No additional history table is required.
+First-seen timestamps continue to govern the existing ordering policy. Saturated timestamps at
+the signed 64-bit limits cannot initiate recovery.
+
+This policy relies on the operator's hypothesis that capture timestamps and the chosen interval
+separate generations; it does not establish a universal datagram lifetime. Tail-before-first
+recovery across generations is not guaranteed. The cutoff is retained only while some bounded
+history for the key remains; after its eviction, old replay can be accepted again. Structural
+fragment validation and exported record formats retain their existing semantics.
+
 ## DNS validation
 
 ### Decoder paths and message boundaries

docs/rfc/0005-dual-path-pcap-parsing.md

@@ -115,6 +115,28 @@ With `--full-fragments`, incomplete responses may use prefix inference on capaci
 EOF. Elapsed capture-time timeout can also trigger fallback when `--monotonic-capture` is enabled.
 Fragmented queries require complete reassembly.
 
+`--ipv4-fragment-reuse-gap-ms <MS>` opts in to recovering an existing pending fragment set when
+a valid DNS first fragment arrives strictly more than `MS` after its maximum admitted timestamp.
+The maximum includes duplicates and rejection events, so rejected sets can also recover. The
+first fragment must be structurally valid IPv4/UDP with a port 53 endpoint; DNS semantic
+validation remains downstream. The option is unset by default, accepts `1..=9223372036854775`
+milliseconds, requires full reassembly, and conflicts with monotonic-capture mode.
+`DPP_IPV4_FRAGMENT_REUSE_GAP_MS` supplies the value when the CLI option is absent. Mode
+requirements are checked after CLI and environment resolution.
+
+Recovery retires old segments and completion history, and releases an old usable response prefix
+through the existing fallback at most once. Rejected sets and candidates whose observed bytes
+match a recent completion do not release a prefix. The old maximum plus the interval becomes a
+cutoff: first fragments, tails, and non-DNS first fragments at or before it are ignored before
+changing state. Existing bounded pending, completed, and non-DNS history carries the maximum
+retained cutoff. Completed or non-DNS records alone never initiate interval-based recovery.
+
+The interval is an operator's hypothesis about timestamps and generation separation, not a
+universal maximum datagram lifetime or an automatic expiry timer. Tail-before-first recovery
+across generations is not guaranteed; replay protection ends when all retained history for the
+key is evicted. Saturated signed 64-bit timestamps cannot initiate recovery. Without this option,
+the existing fragment policy is unchanged.
+
 The reassembler keys fragments by source, destination, protocol, and IPv4 identification. Payload
 bytes use the declared eight-octet offsets. A datagram reaches the existing UDP/DNS parser only
 when the first fragment, final fragment, and contiguous coverage are present. Its fragmented IP

src/app.rs

@@ -164,6 +164,11 @@ fn display_fragment_mode(args: &AppConfig) {
         info!(
             "IPv4 fragment reassembly: enabled (incomplete responses may use first-fragment matching)"
         );
+        if let Some(gap_ms) = args.ipv4_fragment_reuse_gap_ms {
+            info!(
+                "IPv4 ID reuse recovery gap: {gap_ms} ms (per-key forward capture timestamp assumption)"
+            );
+        }
     } else if args.allow_fragments {
         info!(
             "IPv4 response fragment prefix matching: enabled (no reassembly; full UDP/DNS content unverified)"
@@ -203,6 +208,8 @@ struct RunConfigSummary {
     output_format: OutputFormat,
     report_format: ReportFormat,
     match_timeout_ms: u64,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    ipv4_fragment_reuse_gap_ms: Option<u64>,
     monotonic_capture: bool,
     dns_wire_fast_path: bool,
     max_dns_compression_jumps: usize,
@@ -404,6 +411,7 @@ fn build_run_summary(
             output_format: args.format,
             report_format: args.report_format,
             match_timeout_ms: args.match_timeout_ms,
+            ipv4_fragment_reuse_gap_ms: args.ipv4_fragment_reuse_gap_ms,
             monotonic_capture: args.monotonic_capture,
             dns_wire_fast_path: args.dns_wire_fast_path,
             max_dns_compression_jumps: args.max_dns_compression_jumps,
@@ -626,7 +634,8 @@ pub(crate) fn run(args: AppConfig) -> Result<(), AppRunError> {
         .map_err(|source| AppRunError::DnsProcessorInit { source })?
         .with_max_dns_compression_jumps(args.max_dns_compression_jumps)
         .with_allow_fragments(args.allow_fragments)
-        .with_full_fragments(args.full_fragments),
+        .with_full_fragments(args.full_fragments)
+        .with_ipv4_fragment_reuse_gap_micros(args.ipv4_fragment_reuse_gap_micros()),
     );
     let mut packet_parser = match PacketParser::new_with_shutdown(
         &args.input_source,
@@ -779,6 +788,7 @@ mod tests {
             format: OutputFormat::Csv,
             report_format: crate::config::ReportFormat::Text,
             match_timeout_ms: crate::config::DEFAULT_MATCH_TIMEOUT_MS,
+            ipv4_fragment_reuse_gap_ms: None,
             monotonic_capture: false,
             zstd: false,
             v2: false,
@@ -829,6 +839,11 @@ mod tests {
         );
 
         let serialized = serde_json::to_value(&summary).expect("summary serializes");
+        assert!(
+            serialized["config"]
+                .get("ipv4_fragment_reuse_gap_ms")
+                .is_none()
+        );
         let config = serialized
             .get("config")
             .and_then(serde_json::Value::as_object)
@@ -857,6 +872,7 @@ mod tests {
         let mut config = test_config();
         config.allow_fragments = true;
         config.full_fragments = true;
+        config.ipv4_fragment_reuse_gap_ms = Some(120_000);
         let summary = build_run_summary(
             &config,
             config.execution_budget(),
@@ -872,6 +888,7 @@ mod tests {
         let serialized = serde_json::to_value(&summary).expect("summary serializes");
         assert_eq!(serialized["config"]["allow_fragments"], true);
         assert_eq!(serialized["config"]["full_fragments"], true);
+        assert_eq!(serialized["config"]["ipv4_fragment_reuse_gap_ms"], 120_000);
         assert_eq!(serialized["metrics"]["fragmented_response_prefix_count"], 2);
     }
 
@@ -999,6 +1016,7 @@ mod tests {
             format,
             report_format: crate::config::ReportFormat::Text,
             match_timeout_ms: crate::config::DEFAULT_MATCH_TIMEOUT_MS,
+            ipv4_fragment_reuse_gap_ms: None,
             monotonic_capture: false,
             zstd: false,
             v2: false,