Skip to content

fix: bound CSV and RTF materialization - #107

Open
jsk1004ha wants to merge 2 commits into
firecrawl:mainfrom
jsk1004ha:fix/bound-csv-rtf-materialization
Open

fix: bound CSV and RTF materialization#107
jsk1004ha wants to merge 2 commits into
firecrawl:mainfrom
jsk1004ha:fix/bound-csv-rtf-materialization

Conversation

@jsk1004ha

@jsk1004ha jsk1004ha commented Aug 19, 2026

Copy link
Copy Markdown

Summary

Fixes #104.

CSV and RTF bypass the archive package's decompression limits and could materialize an unbounded number of cells, copied text bytes, or separately allocated text runs. This adds a shared fixed budget for standalone frontends and returns the existing resourceLimit error before pathological input can grow the document model without bound.

What is bounded

  • standalone CSV/RTF input bytes before decoding or lexing;
  • text bytes copied into retained document-model strings;
  • content-bearing table cells;
  • separately allocated RTF/CSV text runs.

The limits are fixed alongside the existing archive, XML, expansion, and asset caps. CSV charges every parsed field before constructing its cell. RTF charges decoded text centrally, every emitted text run, generated ordered-list labels, and every closed table cell. Retry/recovery paths cannot swallow ResourceLimit.

Tests

  • budget overflow and stable limit names;
  • CSV cell and copied-text exhaustion;
  • RTF text-byte, text-run, and table-cell exhaustion;
  • full locked Rust test suite;
  • clippy across all targets with warnings denied;
  • rustfmt check.

Implementation was prepared with AI-assisted tooling; all listed checks ran on the submitted commit.


Summary by cubic

Bounds CSV and RTF materialization and standalone input size to prevent unbounded document growth. Previously, standalone CSV/RTF bypassed archive caps and could allocate unlimited cells, text bytes, and text runs; now both enforce fixed caps and return a ResourceLimit before the model grows.

  • Adds package/materialization::MaterializationBudget with defaults in package/limits and stable limit names: "max_standalone_input_bytes", "max_materialized_text_bytes", "max_materialized_cells", "max_materialized_text_runs".
  • CSV: validates input via the budget; charges one cell, copied text bytes, and one text run per field before constructing the cell; no longer preallocates row cells before charging.
  • RTF: validates input via the budget; charges decoded text bytes centrally, each emitted text run, generated ordered-list labels, and on cell close; preserves and returns the first ResourceLimit across recovery and paragraph boundaries.
  • Public API unchanged; parse signatures unchanged; parse_with_budget is internal.

Written for commit b5cf3a1. Summary will update on new commits.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 5 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread src/formats/rtf/mod.rs
Comment thread src/formats/csv.rs Outdated
Comment thread src/formats/rtf/mod.rs

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread .github/workflows/agent-fix-review.yml Outdated
Comment thread .github/workflows/agent-fix-review.yml Outdated
jsk1004ha added a commit to jsk1004ha/anydoc that referenced this pull request Aug 19, 2026
Trigger the temporary main-branch workflow that applies and verifies all review fixes for firecrawl#107.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread .github/workflows/pr-autofix-anydoc.yml Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/apply-verified-rtf-fix.yml">

<violation number="1">
P1: This push-triggered job gives a write-scoped `GITHUB_TOKEN` to code compiled and executed from the triggering branch. A contributor can add a build script, test, or procedural macro that uses the checkout credentials to push arbitrary repository changes; run verification with read-only permissions and perform the final commit from a separate trusted job or reviewed ref.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Add fixed standalone-input and materialization budgets for CSV and RTF, charge cells/text/text runs before retaining them, include generated list labels, preserve the first ResourceLimit across RTF boundaries, and add regression coverage.

Verified with rustfmt, cargo test --locked, and workspace-wide Clippy with warnings denied.
@jsk1004ha
jsk1004ha force-pushed the fix/bound-csv-rtf-materialization branch from 4215dfd to 6096ffa Compare August 19, 2026 08:48
…aterialization

# Conflicts:
#	src/formats/rtf/mod.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CSV/RTF: bound cell/text materialization to prevent memory exhaustion

1 participant