Skip to content

Harden note structure repair and add P0 evaluation workflow - #9

Merged
Cat-blizzard merged 1 commit into
Cat-blizzard:mainfrom
hongzuoj-pixel:fix/p0-structure-eval-hardening
Oct 8, 2026
Merged

Cat-blizzard merged 1 commit into
Cat-blizzard:mainfrom
hongzuoj-pixel:fix/p0-structure-eval-hardening

Conversation

@hongzuoj-pixel

Copy link
Copy Markdown
Contributor

Summary

This PR implements the current P0 quality and evaluation work on top of upstream commit 3fca464.

  • Add a lecture-note structure contract and deterministic document framing.
  • Add guarded one-shot structure repair that rejects truncated or regressing output, preserves source markers and image targets, and re-runs image/figure grounding after an accepted repair.
  • Add a fixed-sample evaluation harness, synthetic sample deck, human-review rubric, and Markdown/DOCX/PDF layout verification.
  • Harden benchmark path handling and fail closed when required artifacts are missing or invalid.
  • Clarify the boundary between coverage signals, heuristic quality checks, and manual review in the GUI and documentation.
  • Expand CI to install GUI dependencies and run the complete test suite.

Safety and compatibility

  • The branch is based directly on Cat-blizzard/SlideNote@3fca464; it does not include the stale history from the fork's main.
  • Evaluation case IDs and output paths are validated before deletion or writing.
  • Existing source markers, image references, and coverage evidence cannot regress during structure repair.
  • Text line endings are normalized with .gitattributes to avoid repository-wide formatting noise.

Validation

  • python -m pytest -q: 354 passed, 0 skipped
  • python scripts/smoke_first_run.py: passed
  • python -m pip check: no broken requirements
  • GUI import smoke test: passed
  • git diff --check: passed
  • Secret-pattern scan: clean

Remaining real-world validation

The included synthetic/local benchmark verifies the engineering workflow, but it does not prove that LLM-generated notes are accurate for arbitrary real courseware. A production-quality claim still requires fixed-model before/after runs on several real lecture decks, manual fact/source/figure checks, and actual DOCX/PDF export review.

@Cat-blizzard
Cat-blizzard merged commit fb27dba into Cat-blizzard:main Oct 8, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants