diff --git a/changelog.md b/changelog.md index 10ab98d822..dc2b0dd39c 100644 --- a/changelog.md +++ b/changelog.md @@ -63,6 +63,7 @@ v1.0.0 - [2925](https://github.com/RuminantFarmSystems/RuFaS/pull/2925) - [minor change] [NoInputChange] [NoOutputChange] Fix the `graph_and_report` option in report_generation.py. - [2907](https://github.com/RuminantFarmSystems/RuFaS/pull/2907) - [minor change] [NoInputChange] [OutputChange] Fix the FarmGrownFeed Emissions unit issue. The mirror issue of [Fix FarmGrownFeed Emissions Unit on test #2908](https://github.com/RuminantFarmSystems/MASM/pull/2908) to update `dev`. - [2934](https://github.com/RuminantFarmSystems/RuFaS/pull/2934) - [minor change] [NoInputChange] [NoOutputChange] [Animal][Reproduction] Refactor `Reproduction.execute_cow_ed_protocol()`. +- [2840](https://github.com/RuminantFarmSystems/RuFaS/pull/2840) - [minor change] [Docs] [NoInputChange] [NoOutputChange] Adds page documenting workflow to acheive version update. ### v1.0.0 diff --git a/docs/_src/_wiki/Scientific-Review-Process.rst b/docs/_src/_wiki/Scientific-Review-Process.rst new file mode 100644 index 0000000000..6e1f5d1bd8 --- /dev/null +++ b/docs/_src/_wiki/Scientific-Review-Process.rst @@ -0,0 +1,96 @@ +Model Review Process +==================== + +This document defines the process for reviewing the RuFaS model prior to a new version release. It complements the branching strategy, versioning policy, and maintainer workflow documentation. + +There are two main phases of the model review process: + +- Preparation and Methodology Review +- Evaluation and Reporting + +Preparation and Methodology Review +--------------------- + +These steps may begin as soon as a new version release has been initiated and before merging development changes into the ``test`` branch. + + +Update Evaluation Farm Dataset +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +- Ensure the evaluation farm input files are up to date and run without error on ``dev`` or on ``test`` after merging ``dev`` to ``test``. + +Update Desired Outputs +^^^^^^^^^^^^^^^^^^^^^^ + +Update all outputs required for evaluation, including: + +- RuFaS output filters +- Post-processing scripts +- Derived or summary outputs + +Update Benchmarks +^^^^^^^^^^^^^^^^^ + +- Review and update benchmark datasets and expectations as needed. +- Ensure benchmarks reflect current model scope and scientific understanding. + +Evaluation and Reporting +--------------------- + +These steps occur after merging into the ``test`` branch. + +Run Model Simulations +^^^^^^^^^^^^^^^^^^^^^ + +- Run evaluation farms using the updated model. +- Compile summarized results including comparison with previous results and benchmarks. +- Responsibility: Scientific Director + + +Review Results Summary +^^^^^^^^^^^^^^^^^^^^^^ + +- Share results with Maintainer SMEs. +- At least one SME from each module must review before proceeding. + +Iterate +^^^^^^^ + +- Repeat simulation and evaluation steps until results are acceptable. + + +Reporting and Review +-------------------- + +Prepare Report +^^^^^^^^^^^^^^ + +- Summarize: + + - Key results + - Comparisons to benchmarks and prior versions + - Notable changes + +Review Report +^^^^^^^^^^^^^ + +- Similar to above, at least one SME from each module should review the report. +- Additional review from external or Contributor SMEs is recommended. + +Approval Criteria +----------------- + +Proceed toward release when: + +- Each module is represented in both results review and report review. +- Model performance is acceptable relative to: + + - Benchmarks + - Prior versions + - Scientific expectations + +Notes +----- + +- This process is iterative and may be adapted as needed. +- Automation of datasets, outputs, and benchmarking workflows is encouraged to improve reproducibility and efficiency. diff --git a/docs/_src/_wiki/Version-Scientific-Evaluation.rst b/docs/_src/_wiki/Version-Scientific-Evaluation.rst new file mode 100644 index 0000000000..9e269af0b4 --- /dev/null +++ b/docs/_src/_wiki/Version-Scientific-Evaluation.rst @@ -0,0 +1,92 @@ +Maintainer Team Workflow and Communication during a Version Update +=================================================================== + +This page describes the process that RuFaS Maintainers will use to communicate +and execute a version update. + +In addition to the updated model and scientific documentation in the repository +codebase, additional deliverables of this process for **Major** and **Minor** +updates are: + +- A version release note that includes: + + - A high-level summary of the outcomes of the SME Review + - A summary of the changelog + +- A detailed technical report of the SME Review + + + +Before Merging ``dev`` to ``test`` +------------------------------ + +**Major** and **Minor** updates: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + +1. RuFaS Maintainers meet to discuss what work **will and will not** be + included in the release and the proposed timeline. + +2. The PML reviews and approves pending work to be included and excluded + from the version update, with additional Maintainer input as needed. + +3. The PML and team leaders create a GitProject for the version release and + add pending issues/PRs that must be completed before the version update + (before ``dev → test``). If a Maintainer identifies additional work that would benefit + the version release after the project is created, issues/PRs should be added to + the version release project or discussed with the PML at the members discretion. + +4. SME Maintainers review the scientific evaluation process and criteria from the + previous version release and update as necessary. + + +**Patch** Updates +^^^^^^^^^^^^^^^^^^ + +Patch updates do not require a full project. Bug fixes required in the patch should be collected +as sub-issues under a **Patch version update parent issue**. + + +After ``test`` is updated +------------------------- + +RuFaS SME Maintainers evaluate integrated model performance: +1. The Scientific Director runs Evaluation Farm Scenarios on ``test`` and compiles their outputs. +2. SME Maintainers review model outputs Evaluation Farms on ``test``. +2. Outputs are collected and summarized for the SME Review process + (link to upcoming SME Review page). +3. SME Maintainers conduct the SME Review led by the Scientific Director. + +During this stage: + +- Maintainers identify and fix any bugs or model errors surfaced during + the SME Review following the processes outlined in the branch management + guidelines for version updates. + +- The Scientific Director prepares: + + - A high-level statement of model outcomes + - A technical summary of the review process + +These summaries are shared with SME Maintainers for input. + + +Release Approval and Publication +-------------------------------- + +After SME Maintainers have had an opportunity to review and provide +feedback on the evaluation report and technical summaries: + +1. RuFaS Directors vote on the release: + + - Director of Software Engineering + - Director of Strategic Partnerships and Governance + - Scientific Director + +2. The high-level summary of the SME review is combined with the release + note. + +3. The technical summary of the SME review is combined with release notes + and added to the RuFaS GitHub Pages documentation. + +4. The Director of Software Engineering generates the official release. \ No newline at end of file diff --git a/docs/_src/_wiki/Version-Update-Workflow.rst b/docs/_src/_wiki/Version-Update-Workflow.rst new file mode 100644 index 0000000000..1f2e839253 --- /dev/null +++ b/docs/_src/_wiki/Version-Update-Workflow.rst @@ -0,0 +1,55 @@ +Maintainer Team Workflow and Communication during a Version Update +=================================================================== + +This page describes the process that RuFaS Maintainers will use to communicate +and execute a version update. + +In addition to the updated model and scientific documentation in the repository +codebase, additional deliverables of this process for **Major** and **Minor** +updates are: + +- A detailed technical report of the scientific review by SME Maintainers +- A version release note that includes: + - A high-level summary of the outcomes of the SME Review + - A summary of the changelog + + +Before Merging ``dev`` to ``test`` +------------------------------ + +**Major** and **Minor** updates: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + +1. RuFaS Maintainers meet to discuss what work **will and will not** be + included in the release and the proposed timeline. + +2. The PML reviews and approves pending work to be included and excluded + from the version update, with additional Maintainer input as needed. + +3. The PML and team leaders create a GitProject for the version release and + add pending issues/PRs that must be completed before the version update + (before ``dev → test``). If a Maintainer identifies additional work that would benefit + the version release after the project is created, issues/PRs should be added to + the version release project or discussed with the PML at the members discretion. + +4. SME Maintainers review the scientific evaluation process and criteria from the + previous version release and update as necessary. + + +**Patch** Updates +^^^^^^^^^^^^^^^^^^ + +Patch updates do not require a full project. Bug fixes required in the patch should be collected +as sub-issues under a **Patch version update parent issue**. + + +After ``test`` is updated +------------------------- +- During the testing phase, SME Maintainers will evaluate integrated model performance. +(link to Scientific Evaluation Process). +- When the model performance has been verified by at least one SME Maintainer per Module, +the Scientific Director will prepare a Scientific Report summarizing the model performance and changes in scientific methodology. +- The Director of Software Engineering will lead preparation of a set of Release Notes to summarize model updates. +- When the testing phase is complete, both the Scientific Report and Release notes are added to the project documentation on GitHub +and shared with the PML for approval. diff --git a/docs/_src/index.rst b/docs/_src/index.rst index 3bfc9b42fc..3fbc7943de 100644 --- a/docs/_src/index.rst +++ b/docs/_src/index.rst @@ -68,6 +68,7 @@ Team Culture _wiki/RuFaS-Versioning-Policy _wiki/SCRUM-at-RuFaS + _wiki/Version-Update-Workflows Release Notes -------------