diff --git a/.jules/sentinel.md b/.jules/sentinel.md index 2b5d819c..46165c31 100644 --- a/.jules/sentinel.md +++ b/.jules/sentinel.md @@ -90,3 +90,7 @@ **Vulnerability:** The `_safe_upload_filename` function used `filename.replace`, `PurePosixPath`, and `re.sub` on unbounded client input, making it vulnerable to ReDoS or CPU/memory exhaustion (DoS) when fed extremely long strings. **Learning:** Even fast standard library functions like `PurePosixPath` and string replacements can cause significant lag when chained on strings in the megabytes. String processing operations should always bound their inputs first if the input is untrusted and can be arbitrarily large. **Prevention:** Cap the length of client-provided filename strings early by slicing them (e.g. `filename = filename[-512:]`) before doing more complex string parsing or regex replacements, especially when only the basename suffix is relevant. +## 2024-08-30 - Form Parameter DoS Risk Mitigation +**Vulnerability:** FastAPIs `Form` fields for `language` and `mode` lacked `max_length` constraints, allowing potential memory exhaustion DoS via oversized form values, since `python-multipart` loads form data into memory before route execution. +**Learning:** In FastAPI, textual `Form` fields must always specify `max_length` to prevent resource exhaustion attacks, as global upload limits do not restrict individual form field sizes handled by `python-multipart`. +**Prevention:** Add `max_length` parameter to all `Form` fields, e.g., `Form(max_length=50)`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 2398ea5c..63a13a7e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 > image placeholder until package, OpenAPI, image, provenance, and release > acceptance are aligned for an actual 0.3.0 publication. +### Security +- 폼 파라미터 DoS 보호: `language` 및 `mode` 입력 필드에 `max_length` 제한을 추가하여 메모리 고갈 공격을 방지함. + ### Changed - `/parse`를 언어 선택형 파서로 일반화: MinerU `-l japan`/`-m ocr` 하드코딩을 제거하고 optional form 필드 `language`(MinerU 3.4.4 공식 기본 `ch`, 공개 언어군/alias 검증)와 `mode`(`auto`/`ocr`/`txt`, 기본 `auto`)로 파라미터화. `mode=auto`는 born-digital PDF가 강제 OCR을 건너뛰도록 함. 기존 입력 `language=japan&mode=ocr`는 공식 규약대로 `ch`/`ocr`로 정규화됨. - OpenAPI 제목/설명, README, `ArticleNode.headline` 문서를 일반 문서용 (section heading) 표현으로 재구성하여 특정 언어/신문 가정을 소비자에게 노출하지 않도록 함. 응답 스키마 필드는 하위 호환을 위해 변경하지 않음. diff --git a/src/newsdom_api/main.py b/src/newsdom_api/main.py index f61aafc2..21416ada 100644 --- a/src/newsdom_api/main.py +++ b/src/newsdom_api/main.py @@ -208,7 +208,8 @@ async def parse( description=( "MinerU language family or compatibility alias (e.g. `ch`, " "`en`, `japan`, `korean`, `arabic`, `devanagari`)." - ) + ), + max_length=50, ), ] = DEFAULT_LANGUAGE, mode: Annotated[ @@ -217,7 +218,8 @@ async def parse( description=( "MinerU parsing mode: `auto` (born-digital text PDFs skip forced " "OCR), `ocr` (force OCR), or `txt` (embedded text layer only)." - ) + ), + max_length=50, ), ] = DEFAULT_MODE, ) -> ParseResponse: diff --git a/tests/test_parse_endpoint_security.py b/tests/test_parse_endpoint_security.py new file mode 100644 index 00000000..13081b2d --- /dev/null +++ b/tests/test_parse_endpoint_security.py @@ -0,0 +1,29 @@ +import pytest +from fastapi.testclient import TestClient +from newsdom_api.main import app, _runtime_settings +from newsdom_api.config import RuntimeSettings, AuthenticationMode + +def test_parse_form_field_max_length_rejection(): + """Verify that oversized form fields are rejected with 422 to prevent DoS.""" + + def override_settings(): + return RuntimeSettings(authentication_mode=AuthenticationMode.DISABLED) + + app.dependency_overrides[_runtime_settings] = override_settings + + try: + client = TestClient(app) + + pdf_content = b"%PDF-1.4\n1 0 obj\n<<>>\nendobj\ntrailer\n<<>>\n%%EOF" + + oversized_string = "a" * 51 + response = client.post( + "/parse", + files={"file": ("test.pdf", pdf_content, "application/pdf")}, + data={"language": oversized_string, "mode": "auto"}, + ) + + assert response.status_code == 422 + assert "detail" in response.json() + finally: + app.dependency_overrides.clear()