feat: prepare Doc2FAQ v0.4.0 — Review, Save & Export

CHANGELOG.md

@@ -8,6 +8,31 @@ All notable Doc2FAQ changes are recorded here. Version numbers follow semantic v
 
 - Show the full build version in the preview heading and footer, populated from `pom.xml` during the Maven build so future releases stay in sync.
 
+## 0.4.0 (unreleased)
+
+### Added
+
+- Cursor-responsive ambient background with reduced-motion and page-visibility support.
+- Document title help, separate editable download filename, and Save work labels that explain how to resume edits.
+- Clear document control that also removes the saved source and discards late imports, FAQ responses, and exports.
+- Original source filename defaults such as `TEst Data.pdf` to `testdata-Doc2FAQ.pdf`, preserved in saved work and supported by export response headers.
+
+- Versioned project files that save/reopen source text, title, FAQ edits/order, original excerpts, and review choices, including incomplete entries.
+- Optional browser autosave/restore with clear saved data controls; saving is off by default.
+- Exact source excerpts and extracted-text line references for both drafting modes, with edited-answer labels.
+- Similar-question review, keep/merge controls, and undo for removals/merges. Merges retain the original references.
+- PDF and DOCX downloads with title metadata, page numbers, Unicode font support, optional references, and validated input limits.
+- Safe content preview and configurable titles for all exports.
+- Project/schema/merge tests, source-reference tests, and export API tests for Unicode, pagination, validation, and unsupported PDF glyphs.
+
+### Changed
+
+- PDF uses embedded Lato Regular/Bold fonts and a centered title; PDF and Word exports omit the FAQ-count subtitle.
+
+- Distinct answers under the same topic or question are retained for manual review instead of silently discarded. Exact question/answer duplicates remain filtered; the draft cap remains 30.
+- Answer editing now supports up to 50,000 characters, consistent with imported text.
+- Pending draft results are invalidated when draft entries are edited, added, moved, removed, merged, or restored. Delayed file imports cannot replace newer source/project changes.
+
 ## 0.3.1
 
 ### Fixed

README.md

@@ -2,11 +2,24 @@
 
 **Turn documents into editable FAQ drafts — locally, with full control over the answers.**
 
-Doc2FAQ is a Java 21 / Spring Boot document-to-FAQ workbench. Paste text or upload TXT, Markdown, PDF, or DOCX, then extract existing questions or create source-grounded suggestions from ordinary document sections. Review, edit, reorder, and export your results as **Markdown, JSON, or HTML**.
+Doc2FAQ is a Java 21 / Spring Boot document-to-FAQ workbench. Paste text or upload TXT, Markdown, PDF, or DOCX, then extract existing questions or create source-grounded suggestions from ordinary document sections. Review source excerpts, edit and reorder your FAQs, save a project, and export as **Markdown, JSON, HTML, PDF, or Word (DOCX)**.
 
 Created by **Okan Tiktas** · [OKAN-SOFTWARE](https://github.com/OKAN-SOFTWARE).
 
-> **Source version: v0.3.1.** The version in `pom.xml` identifies this build. For published tags, release downloads, and announcements, see [GitHub Releases](https://github.com/OKAN-SOFTWARE/Doc2FAQ/releases). This source version does not by itself publish a GitHub Release.
+> **Development version: v0.4.0-SNAPSHOT.** The version in `pom.xml` identifies this build. For published tags, release downloads, and announcements, see [GitHub Releases](https://github.com/OKAN-SOFTWARE/Doc2FAQ/releases). This source version does not by itself publish a GitHub Release.
+
+## New in the v0.4.0 development build
+
+- **Save work / Open saved work:** a versioned JSON file preserves source text, title, mode, draft order, edits, original excerpts, and duplicate-review choices. Incomplete FAQs can be saved too. Project files are limited to 4 MB and validated before replacing the workspace.
+- **Optional browser autosave:** off by default. Enabling it stores the complete source and draft on this browser's origin and restores them on reload. Disabling autosave or using **Clear saved data** removes Doc2FAQ's saved data without clearing your open workspace. Storage failures leave the draft open; use **Save work** as a backup.
+- **Source references:** expand **Show source** to see the original section, extracted-text line range, and exact passage. Edited questions and answers are marked for rechecking. References use text lines, not PDF page numbers; manually added FAQs have no automatic reference.
+- **Duplicate review:** distinct passages are retained even if their questions cover the same topic. Similar questions are flagged using a word-overlap heuristic. Choose **Keep both** or **Merge into this FAQ**. Merging keeps both answers and all source references; **Undo removal / merge** restores the previous items until another FAQ edit, addition, reorder, or review choice. Exact duplicate question/answer pairs are still filtered. The maximum remains 30 FAQs.
+- **PDF and Word export:** edited content, a centered document title, readable page layout, metadata, and page numbers. Optionally include section/line references and edited-answer notices. PDF embeds Lato Regular and Bold (see `src/main/resources/fonts/LICENSE-Lato.txt`); if text contains a glyph that font cannot represent, export returns an error and suggests DOCX.
+- **Export preview:** inspect the final content before downloading. PDF and Word apply their own pagination. Markdown, JSON, and standalone HTML export remain available; binary exports use **Download**.
+
+- **Clear document:** removes the loaded source, FAQ draft, file names, pending results, and its saved browser copy. The autosave preference is retained. A late import or export cannot bring back the cleared PDF.
+- **Original filenames:** importing `TEst Data.pdf` pre-fills the document title and the download name `testdata-Doc2FAQ.pdf`. Edit **Download filename** before exporting; saved work preserves that choice. Names are sanitized for Windows and the extension follows the selected format.
+- **Interactive background:** gentle particles and a cursor-responsive glow, paused while the page is hidden. Reduced-motion preferences use a still background.
 
 ## What's fixed in v0.3.1
 
@@ -26,7 +39,7 @@ The document placeholder now uses neutral Q&A prompts, and Generate Mode's sampl
 - **Consistent 50,000-character limit** across document extraction, the browser, and both FAQ API endpoints.
 - **Windows CMD setup** for people who don't have Java or Maven installed, plus troubleshooting.
 
-**Privacy and accuracy:** The current Generate Mode is **deterministic and rule-based, not a generative AI model**. No API key, paid AI service, or external model is required. Its questions are suggestions; answers come from excerpts in your source document. Topic-based duplicate filtering can sometimes suppress distinct details within a topic. It cannot reliably interpret every layout or guarantee an accurate summary. Always review before publishing.
+**Privacy and accuracy:** The current Generate Mode is **deterministic and rule-based, not a generative AI model**. No API key, paid AI service, or external model is required. Its questions are suggestions; answers come from excerpts in your source document. Similar-question flags are suggestions and can include questions you want to keep. It cannot reliably interpret every layout or guarantee an accurate summary. Always review before publishing.
 
 For detailed changes, see [CHANGELOG.md](CHANGELOG.md). For the release checklist, see [docs/RELEASING.md](docs/RELEASING.md).
 
@@ -150,15 +163,15 @@ After updating your checkout to the latest `main`, run these commands from the d
 ```cmd
 mvn -B -ntp clean verify
 node --test src/test/js/*.test.cjs
-java -jar target/doc2faq-0.3.1.jar
+java -jar target/doc2faq-0.4.0-SNAPSHOT.jar
 ```
 
 The JavaScript checks require Node.js. Keep the terminal open while the app runs, and use **Ctrl+C** to stop it. Open **http://localhost:8080** and refresh the browser to load the updated frontend.
 
 If port 8080 is occupied, stop the previous Doc2FAQ process or use an alternate port:
 
 ```cmd
-java -jar target/doc2faq-0.3.1.jar --server.port=8081
+java -jar target/doc2faq-0.4.0-SNAPSHOT.jar --server.port=8081
 ```
 
 Then open **http://localhost:8081**. Merging source changes does not update an existing JAR or a running Java process; rebuild and restart to use the fixes. Published JARs must be built from the corresponding release commit.
@@ -170,8 +183,9 @@ The preview heading and footer display the full project version from `pom.xml`.
 1. Paste your document's text or choose a `.txt`, `.md`, `.pdf`, or `.docx` file.
 2. Select **Extract Mode** if your text already contains FAQ questions; select **Generate Mode** for a normal document with section headings.
 3. Click **Extract FAQ draft** or **Generate FAQ draft**, depending on the mode.
-4. Review and edit each question and answer. Use arrows to reorder, **Remove** to delete, and **Add FAQ** to add an entry.
-5. Choose **Markdown**, **JSON**, or **HTML**, then **Copy** or **Download**. Blank questions or answers must be completed before export.
+4. Expand **Show source** and review each question and answer. Reorder, remove, or add entries. Review similar questions and use Undo after a removal or merge.
+5. Set the document title printed on your FAQ and use **Save work** to keep the source and your edits. Use **Open saved work** to continue later. Optionally enable browser autosave.
+6. Choose **Markdown**, **JSON**, **HTML**, **PDF**, or **Word**, optionally include references, and inspect **Preview export**. Use **Copy** for text formats or **Download** for any format. Blank questions or answers must be completed before export.
 
 Example document for **Generate Mode**:
 
@@ -193,7 +207,7 @@ The team provides onboarding guides and a shared knowledge base.
 Members share guides on document organization and clear technical writing.
 ```
 
-Generate Mode preserves explicit FAQ pairs, then suggests questions for additional sections while filtering repeated common topics and boilerplate appendix headings. The answer text is sourced from the document. Generated question phrasing is heuristic; this is *not* legal advice or an automated contract interpretation service.
+Generate Mode preserves explicit FAQ pairs and suggests questions for additional sections. Exact duplicate pairs and boilerplate appendix headings are filtered; distinct passages remain available for review. The answer text is sourced from the document. Generated question phrasing is heuristic; this is *not* legal advice or an automated contract interpretation service.
 
 ## Formats and limits
 
@@ -203,6 +217,8 @@ Generate Mode preserves explicit FAQ pairs, then suggests questions for addition
 | PDF | Text extraction using Apache PDFBox | 5 MB upload, 50 pages, 50,000 extracted characters |
 | DOCX | Text extraction using Apache POI | 5 MB upload, 50,000 extracted characters |
 | FAQ drafts | Local rules based on source text | Maximum 30 suggested/extracted pairs |
+| Project files | Validated JSON, read in the browser | 4 MB, 30 FAQs, 50,000 source characters |
+| Exports | Text formats in browser; PDF/DOCX on the server | Title 200 characters; question 500; answer 50,000; combined FAQ text 300,000 |
 
 The server checks file signatures, not just extensions. Empty, corrupted, password-protected, and unsupported documents are rejected with clear errors. **Scanned/image-only PDFs require OCR**, which this version does not provide. Complex tables and multi-column layouts can lead to imperfect text order; inspect the extracted text before drafting.
 
@@ -214,6 +230,8 @@ The server checks file signatures, not just extensions. Empty, corrupted, passwo
 | POST | `/api/documents/extract` | Multipart PDF/DOCX to extracted text |
 | POST | `/api/faqs/preview` | Extract explicit questions and answers |
 | POST | `/api/faqs/generate` | Suggest source-grounded FAQs using local rules |
+| POST | `/api/faqs/export/pdf` | Validated FAQ JSON to downloadable PDF |
+| POST | `/api/faqs/export/docx` | Validated FAQ JSON to downloadable Word document |
 
 Both FAQ endpoints take JSON input:
 
@@ -226,7 +244,12 @@ Response format:
 ```json
 {
   "items": [
-    {"question": "What services are included?", "answer": "Website monitoring and software maintenance."}
+    {
+      "question": "What services are included?",
+      "answer": "Website monitoring and software maintenance.",
+      "source": {"section": "Services", "startLine": 1, "endLine": 2,
+                 "excerpt": "## Services\nWebsite monitoring and software maintenance."}
+    }
   ],
   "count": 1,
   "notice": "..."
@@ -235,14 +258,32 @@ Response format:
 
 The `/api/faqs/preview` endpoint deliberately returns zero results if there are no complete, explicit Q&A pairs. The `/api/faqs/generate` endpoint can produce a conservative draft from ordinary text, even without headings.
 
+Export endpoints accept this shape (1–30 nonblank entries; references are optional):
+
+```json
+{
+  "title": "Team FAQ",
+  "includeSources": true,
+  "filename": "team-Doc2FAQ.pdf",
+  "items": [{
+    "question": "Where is support?",
+    "answer": "Ask in the team channel.",
+    "edited": false,
+    "sources": [{"section": "Support", "startLine": 1, "endLine": 2}]
+  }]
+}
+```
+
+Export references are supplied by the caller. They identify passages for human review; the export API does not independently verify their origin. Both download responses use `Cache-Control: no-store`. The optional `filename` controls the attachment name; the endpoint enforces the PDF/DOCX extension. Exports omit the FAQ-count subtitle.
+
 ## Tests
 
 ```bash
 mvn -B -ntp clean verify
 node --test src/test/js/*.test.cjs
 ```
 
-GitHub Actions automatically runs the Java and JavaScript checks on pushes and pull requests. Tests cover both modes, whitespace-heavy parsing, stale responses and errors, distinct client/provider topics, FAQ appendix/title exclusions, document import errors, API validation, output limits, and export safety. The frontend tests require Node.js and have no npm dependencies.
+GitHub Actions automatically runs the Java and JavaScript checks on pushes and pull requests. Tests cover both modes, exact source references, preservation of distinct passages, stale responses, document import errors, project round trips and invalid files, merge behavior, API validation, Unicode exports, PDF pagination, and export safety. The frontend tests require Node.js and have no npm dependencies.
 
 ## Troubleshooting
 
@@ -258,14 +299,14 @@ GitHub Actions automatically runs the Java and JavaScript checks on pushes and p
 
 ## Security and privacy
 
-Doc2FAQ currently has **no accounts, billing, rate limiting, or permanent draft storage**. PDF/DOCX files are processed for the request and are not intentionally saved by the app, though the web server may use temporary multipart buffering. TXT/Markdown are read in the browser and submitted as text when you generate a draft. Editing and exporting happen in the browser; drafts disappear when the page is reloaded.
+Doc2FAQ currently has **no accounts, billing, rate limiting, or server-side draft storage**. PDF/DOCX files are processed for the request and are not intentionally saved by the app, though the web server may use temporary multipart buffering. TXT/Markdown are read in the browser and submitted as text when you generate a draft. Editing, project saves, and text-format exports happen in the browser. PDF/DOCX export sends the edited FAQ content and reference labels to this server for an in-memory download. With autosave off, unsaved drafts disappear on reload. Downloaded project files and optional browser autosave contain source text; clear saved data on shared devices when finished.
 
 This is a **local development application**, not a hardened internet-facing service. Before offering paid or public document processing, add authentication, rate limiting, isolated document parsing, malware scanning, HTTPS, audit/monitoring, and retention policies. Do not process confidential material on an untrusted server.
 
 ## License
 
 MIT · Copyright © 2026 **Okan Tiktas**.
 
-## Planned enhancements
+## Later releases
 
-- PDF and DOCX FAQ export with page layout, fonts, document metadata, and validation tests (not included in this version).
+Optional AI generation with explicit opt-in and token/cost limits, OCR, and account/subscription workflows are outside the v0.4.0 development scope. See [docs/RELEASING.md](docs/RELEASING.md) for the validation and release process.