Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .jules/sentinel.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,7 @@
**Vulnerability:** The `_safe_upload_filename` function used `filename.replace`, `PurePosixPath`, and `re.sub` on unbounded client input, making it vulnerable to ReDoS or CPU/memory exhaustion (DoS) when fed extremely long strings.
**Learning:** Even fast standard library functions like `PurePosixPath` and string replacements can cause significant lag when chained on strings in the megabytes. String processing operations should always bound their inputs first if the input is untrusted and can be arbitrarily large.
**Prevention:** Cap the length of client-provided filename strings early by slicing them (e.g. `filename = filename[-512:]`) before doing more complex string parsing or regex replacements, especially when only the basename suffix is relevant.
## 2026-08-29 - [DoS Risk Mitigation] Form Field Max Length
**Vulnerability:** FastAPIs using `Form()` without a `max_length` can be exploited to cause memory exhaustion by uploading arbitrarily large fields, as `python-multipart` loads them fully into memory.
**Learning:** Even auxiliary configuration fields (like `language` and `mode`) can present critical resource exhaustion vectors if their length is unbounded.
**Prevention:** Always define a tight `max_length` (e.g. `max_length=50`) on string `Form()` parameters.
6 changes: 4 additions & 2 deletions src/newsdom_api/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,19 +205,21 @@ async def parse(
language: Annotated[
str,
Form(
max_length=50,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

πŸŸ₯ 폼 μ œν•œ μ „ λ©”λͺ¨λ¦¬ 고갈

max_lengthλŠ” multipart νŒŒμ‹±μ΄ λλ‚œ λ’€ κ²€μ¦λ©λ‹ˆλ‹€. μš”μ²­ ν•˜λ‚˜κ°€ μ œν•œ λ°–μ˜ μΆ”κ°€ ν•„λ“œ 1,000개둜 μ•½ 1GiBλ₯Ό μ μœ ν•  수 μžˆμŠ΅λ‹ˆλ‹€.

Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.

description=(
"MinerU language family or compatibility alias (e.g. `ch`, "
"`en`, `japan`, `korean`, `arabic`, `devanagari`)."
)
),
),
] = DEFAULT_LANGUAGE,
mode: Annotated[
str,
Form(
max_length=50,
description=(
"MinerU parsing mode: `auto` (born-digital text PDFs skip forced "
"OCR), `ocr` (force OCR), or `txt` (embedded text layer only)."
)
),
),
] = DEFAULT_MODE,
) -> ParseResponse:
Expand Down
20 changes: 20 additions & 0 deletions tests/test_parse_endpoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -555,3 +555,23 @@ def spy_unlink(self, missing_ok=False):
# We should have unlinked exactly one file, which should be in the temp directory
assert len(unlinked_paths) == 1
assert "tmp" in unlinked_paths[0].lower() or "temp" in unlinked_paths[0].lower()


def test_parse_endpoint_rejects_excessively_long_form_fields():
client = TestClient(app)
Comment on lines +560 to +561

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

πŸ“ Info: ν™˜κ²½ 제거둜 ν…ŒμŠ€νŠΈ μ˜λ―ΈλŠ” λ°”λ€Œμ§€ μ•ŠμŒ

app은 ν•¨μˆ˜ μ‹€ν–‰ 전에 μƒμ„±λ˜λ―€λ‘œ μ‚­μ œλœ setenvλŠ” μ›λž˜ μ μš©λ˜μ§€ μ•Šμ•˜λ‹€. λ™μΌν•œ ν™˜κ²½κ°’μ€ conftest.pyκ°€ ν…ŒμŠ€νŠΈ μˆ˜μ§‘ 전에 μ„€μ •ν•œλ‹€.

Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.


long_string = "a" * 100

response_lang = client.post(
"/parse",
files={"file": ("test.pdf", b"%PDF-1.4\n...", "application/pdf")},
data={"language": long_string, "mode": "auto"},
)
assert response_lang.status_code == 422

response_mode = client.post(
"/parse",
files={"file": ("test.pdf", b"%PDF-1.4\n...", "application/pdf")},
data={"language": "ch", "mode": long_string},
)
assert response_mode.status_code == 422
Loading