[DevOps]: Local replication and testing workflow for NERSC archive ingestor after DB-backed state migration

[tomvothecoder]

Is your feature request related to a problem?

After #189 changes the NERSC archive ingestor to use database-backed state, SimBoard still needs a clean way to locally replicate and test the ingestion workflow that runs at NERSC Spin.

Right now, local validation of the deployed NERSC ingestion path is awkward because it spans multiple concerns:

1. The ingestor script scans a performance archive and submits path-based ingestion requests.
2. The backend must be able to read the same archive path the ingestor submits.
3. Production execution happens through a Spin CronJob, but local developers mainly need a repeatable one-shot test harness.
4. Current local docs do not clearly define a supported workflow for exercising the NERSC ingestion runner end-to-end after the DB-backed state model lands.

Without a supported local workflow, changes to the ingestor and path-based ingestion behavior are harder to verify before deployment.

Describe the solution you'd like

Add a documented local development workflow for the post-#189 NERSC ingestion runner.

Proposed scope:

1. Provide a local one-shot execution path for app.scripts.ingestion.nersc_archive_ingestor using the same production-oriented contract introduced by [Enhancement]: Database-backed ingestion state management #189.
2. Define how local developers should provide:
   • SimBoard API base URL
   • service-account token
   • shared archive path visible to both the runner and backend
   • machine name
3. Add developer-facing commands and/or helper targets to make dry-run and real-run testing straightforward.
4. Document a repeatable verification flow covering:
   • first ingest succeeds
   • second ingest is idempotent / no-op
   • adding a new execution directory causes delta ingestion
5. Update NERSC/Spin docs only as needed to clarify how the local workflow differs from the Rancher CronJob deployment wrapper.

This should target the final DB-backed-state design from #189, not preserve the current file-backed STATE_PATH workflow.

Describe alternatives you've considered

• Reproducing the full Rancher/Kubernetes CronJob locally.
  • Likely too heavy for routine development and not necessary for most ingestion changes.
• Building the local workflow now against the current file-backed state implementation.
  • This would likely create throwaway work that must be rewritten after [Enhancement]: Database-backed ingestion state management #189 lands.
• Relying only on unit tests.
  • Useful, but not enough for validating local end-to-end path ingestion behavior.

Additional context

• Follow-up to #189.
• Current production deployment/runbook: docs/deploy/spin.md
• Current runner implementation: backend/app/scripts/ingestion/nersc_archive_ingestor.py
• Goal is local replication of the NERSC ingestion workflow with minimal dev friction, not full platform emulation.