Skip to content

Add container image documentation and generic Slurm/HTCondor submission scripts - #160

Draft
afroch wants to merge 15 commits into
mainfrom
container-docs-slurm-scripts
Draft

Add container image documentation and generic Slurm/HTCondor submission scripts#160
afroch wants to merge 15 commits into
mainfrom
container-docs-slurm-scripts

Conversation

@afroch

@afroch afroch commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

This pull request introduces the following changes

  • Document the CI-built container image (upp:latest / upp:<tag>) in the setup docs, with apptainer and docker usage, including the CVMFS-unpacked path on unpacked.cern.ch
  • Add a "Running on HPC" docs page covering interactive and batch usage of the container image on Slurm and HTCondor
  • Ship generic batch submission scripts (scripts/batch/) that submit one job per component/region/split, deriving the job list from the preprocessing config; the scheduler (Slurm or HTCondor) is auto-detected
  • Default to the CVMFS-unpacked image when available, falling back to the registry
  • Add a list_components console script that enumerates the components defined in a config
  • Document how to obtain the scripts without a local clone (copied from the image itself)

Conformity

- Document the CI-built container image (apptainer/docker) in the setup docs
- Add a 'Running on HPC' docs page covering interactive and batch usage
- Ship generic Slurm scripts (submit.sh/batch.sh/run_stage.sh) that derive
  the job list from the preprocessing config
- Add a list_components console script to enumerate config components
@afroch afroch added documentation Improvements or additions to documentation enhancement New feature or request AI-assisted AI coding assistants were used in the development of the changes in this merge request. labels Aug 4, 2026
@afroch afroch self-assigned this Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 95.34%. Comparing base (281e9ab) to head (e905f2f).

Files with missing lines Patch % Lines
upp/utils/list_components.py 95.45% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #160   +/-   ##
=======================================
  Coverage   95.34%   95.34%           
=======================================
  Files          24       25    +1     
  Lines        2318     2340   +22     
=======================================
+ Hits         2210     2231   +21     
- Misses        108      109    +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@afroch afroch changed the title Add container image documentation and generic Slurm submission scripts Add container image documentation and generic Slurm/HTCondor submission scripts Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI-assisted AI coding assistants were used in the development of the changes in this merge request. documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant