[Aliussman]

Description

Implements periodic report generation API endpoints and async worker execution for monthly, quarterly, and annual department reporting in FireForm as outlined in #557 (part of contract tracker #541).

Key Additions:

• Endpoints:
  • POST /api/v1/reports/generate: Asynchronously kicks off periodic report generation; returns 202 Accepted with report_id, job_id, and poll_url.
  • GET /api/v1/reports/{report_id}: Polling and result retrieval. Returns 202 Accepted with retry_after_seconds while generating; returns 200 OK with PDF binary or JSON payload (PeriodicReport schema) upon completion; returns 404 Not Found if missing.
• Contract & Schemas:
  • Added contracts/path/reporting.yaml and contracts/schemas/reporting.yaml, mounted in contracts/openapi.yaml.
  • Added Pydantic schemas in app/api/schemas/reporting.py and updated AsyncJobResponse in app/api/schemas/common.py.
• Database Repositories:
  • Added create_report, get_report, update_report, get_incidents_in_date_range (with strict UTC timezone awareness), get_forms_for_incidents, and get_extractions_by_ids in app/db/repositories.py.
• Business Logic & PDF Generation:
  • Implemented app/services/reporting.py with period boundary computation & validation, future period rejection (422 Unprocessable Entity with code FUTURE_PERIOD), metrics aggregation (incident types, statuses, forms generated, completeness score, processing time), compliance metrics (NERIS submission rate, delay, overdue count), and ReportLab-based PDF generation.
• Background Worker:
  • Implemented Celery task generate_periodic_report in app/tasks/report.py and registered it in app/core/celery.py.
• Dependencies:
  • Added reportlab to requirements.txt for PDF rendering.

Fixes #557

Type of change

• New feature (non-breaking change which adds functionality)

How Has This Been Tested?

• Added unit and integration tests in tests/test_v1_reports.py (18 tests):
  • Async generation request acceptance (POST /api/v1/reports/generate -> 202).
  • Monthly, quarterly, and annual period calculations.
  • Rejection of future periods (422 Unprocessable Entity with FUTURE_PERIOD).
  • Validation of invalid month and quarter values.
  • Polling status check (GET /api/v1/reports/{id} -> 202).
  • Completed PDF generation and binary download (Content-Type: application/pdf).
  • Completed JSON report retrieval matching PeriodicReport schema.
  • Handling of non-existent report IDs (404 Not Found).
  • Queue failure handling when Redis is unavailable (503 Service Unavailable).
  • Empty incident range handling.
  • Multi-incident aggregations across quarterly boundaries.
  • Direct Celery task execution with session patching.
• Full test suite executed locally: PYTHONPATH=. pytest → 331 passed.
• Linter executed: python -m ruff check app/ tests/ → All checks passed!

Test Configuration:

• Python version: 3.13.5
• Docker/Compose version: N/A (tested against SQLite test engine and mocked queue)
• OS: Ubuntu Linux (x86_64)

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
• The Lint workflow check passes on this PR ✅
• The Tests workflow check passes on this PR ✅

[marcvergees]

Hi @Aliussman, we should be having a frontend implementation for that feature. Feel free to create issue and PR.

Thanks,
Marc

[vharkins1]

Hey @Aliussman, thanks for putting this together, it's a big one and the tests look really thorough!

Before I review it for merge there are a few things. Like Marc said, this needs a frontend side, so could you open an issue + PR in fireform-frontend and link them here? It's also conflicting with development in router.py now, so it'll need a rebase.

[Aliussman]

Thanks @marcvergees and @vharkins1! Both points are done:

• Rebased onto the latest development. The router.py conflict is resolved, keeping the new profiles / llm_config routers. The rebase also brought in the new test_contracts.py, which found a broken $ref to an OutputFormat enum that was missing from contracts/schemas/enums.yaml. I've added it, and the full suite passes (431 tests).
• Frontend: issue [FEAT]: Add periodic report generation interface fireform-frontend#58, PR feat: add periodic report generation interface fireform-frontend#60. It adds a Reports tab that requests a report, polls until it's ready, then downloads the PDF or shows the JSON report.

[vharkins1]

Hey @Aliussman, thanks for turning this around so fast! The rebase and the frontend PR both look great.

One bigger change though. Right now the report reads from the Incident, Form and Extraction tables, but nothing on development writes to those (they're from the Pipeline C contract that got pulled in #645), so every report comes back empty.

Could you switch it to build from FormSubmission instead? That's where every filled form lives (template_id, input_text, output_pdf_path, created_at). So for a period: total forms filled, a breakdown by template, and a list of the submissions with links to their PDFs. The incident-only parts (categories, NERIS rate, injuries) can come out for now, and the reporting contract should match the new shape.

The frontend can mostly stay as it is, it just needs the JSON view updated for the new fields. Thanks, this is solid work!

Since this changes the scope of the issue I'd also like @marcvergees and @chetanr25 to look at this.

[marcvergees]

@Aliussman, let us know when you're done with that.

[marcvergees]

Hi @vharkins1 and @Aliussman, I've checking what you guys have been discussing recently. I checked out both front and back of your stuff Ali. I don't know why but having a bunch of incidents in my local db, the pdf report is completely.

[Aliussman]

Thanks @vharkins1 and @marcvergees for the feedback. I understand the change in direction.

I'll update the reporting implementation to use FormSubmission as the data source, since that's where the current form submissions are stored. I'll revise the report to include the total submissions, breakdown by template, and submission details with PDF links, while removing the incident-specific metrics for now.

I'll also update the reporting contract and the frontend JSON view to reflect the revised response structure. I'll investigate the PDF behaviour you shared as part of the changes and report back once I've validated everything.

[Aliussman]

Hi @vharkins1 @marcvergees,

Thank you for the detailed feedback! I have refactored the reporting service and updated the OpenAPI contract as requested:

1. Switched Data Source to FormSubmission & Template:

   • Queries FormSubmission joined with Template across the selected date range.
   • Generates summary statistics:
     • total_submissions: total count of form submissions in the period.
     • by_template: mapping of template name to submission count (sorted descending).
     • groups: monthly breakdown for quarterly and annual periods.
     • submissions: detailed list with submission ID, template ID & name, input text preview, output PDF path, and preview URL.
   • Removed incident-only fields (neris_submission_rate, average_submission_delay_days, overdue_incidents, by_type, by_status, avg_completeness_score, avg_processing_time_seconds).

2. Updated OpenAPI Contracts & Pydantic Schemas:

   • Updated contracts/schemas/reporting.yaml and contracts/path/reporting.yaml.
   • Updated app/api/schemas/reporting.py.
   • Contract tests (test_contracts.py) and reporting unit tests (test_v1_reports.py) are fully passing (454/454 tests passing across the backend test suite).

3. ReportLab PDF & Frontend:

   • PDF export now includes executive summary metrics, template usage breakdown, and submission records.
   • Associated frontend PR has also been updated to render the new JSON schema and provide a direct Preview button for each submission PDF.

The PR has been rebased cleanly onto latest development. Ready for review