Add explicit version bootstrapping

[douglasjacobsen]

This merge adds the ability for requires_utility to define an explict version that should be used instead of just a range. Additionally, now any variables for the utility can be passed into the requires_utility directive as a kwarg to make it more easily set.

For example, the spack utility can be required with something like:

requires_utility("spack", path="/spack-path", hash="1234abc")

which will control the hash and the installation path for the installed spack.

[gemini-code-assist]

Code Review

This pull request introduces support for validating exact versions of external dependencies using VCS (Git) history and fallback command output during the bootstrapping process. It also adds corresponding unit tests and updates workspace utility configuration hashing. The review feedback highlights several key improvements: moving the nested helper function _hashable_val out of loops to avoid redundant redefinitions, safely handling cases where obj.variables or configuration blocks are None to prevent runtime errors, ensuring that relative executable paths do not cause Git commands to run in the current working directory, and tightening the exact version substring check with a regular expression to prevent false positives.

[ramble-pr-bot]

Ramble Performance Test Metrics

Results produced with commit: 0e5bade

Test Name	Outcome	Duration (s)	Most Recent Run (s)	Last 5 Avg (s)
test_analyze_large_file	passed	2.1473	2.0570 (6d999da)	2.0671
test_large_template_expansion	passed	1.1836	1.1796 (6d999da)	1.1899
test_many_experiments	passed	24.1530	23.5008 (6d999da)	24.1369
test_many_objects_defaults	passed	12.6898	11.9471 (6d999da)	12.5569
test_matrix_filter_perf	passed	1.0884	1.0345 (6d999da)	1.0943

[codecov]

Codecov Report

❌ Patch coverage is 93.92625% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.11%. Comparing base (5802679) to head (0e5bade).
⚠️ Report is 12 commits behind head on develop.

Files with missing lines	Patch %	Lines
lib/ramble/ramble/test/test_utility_base.py	92.77%	20 Missing ⚠️
...os/builtin/base_classes/utility-base/base_class.py	94.11%	3 Missing ⚠️
lib/ramble/ramble/cmd/workspace.py	66.66%	2 Missing ⚠️
lib/ramble/ramble/test/test_base_classes_extra.py	97.91%	2 Missing ⚠️
...uiltin/base_classes/application-base/base_class.py	96.77%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1747      +/-   ##
===========================================
+ Coverage    94.04%   94.11%   +0.07%     
===========================================
  Files          370      371       +1     
  Lines        37330    37768     +438     
===========================================
+ Hits         35106    35546     +440     
+ Misses        2224     2222       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for validating exact versions of external dependencies using VCS (Git) history during bootstrapping. It adds a VCS-based exact match check, updates version validation and availability checks to support exact versions, and adds corresponding unit tests. The review feedback highlights two improvement opportunities: avoiding the use of try...except BaseException: pass in tests to prevent swallowing unexpected failures, and logging swallowed exceptions in the VCS check helper instead of silently ignoring them.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for exact version validation of external utility dependencies in Ramble, including checking versions via VCS (Git) history and fallback command outputs. It also adds corresponding unit tests and updates workspace utility hashing. The review feedback highlights a few critical issues: first, the _hashable_val helper in workspace.py needs to be recursive to prevent TypeError on nested structures; second, there is an inconsistency in how obj.variables is checked (using hasattr instead of checking for None), which will cause a crash during bootstrapping and in the new test; and finally, exact_version should be explicitly cast to a string in utility-base/base_class.py to avoid a TypeError when parsing non-string values from YAML.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for validating exact versions of external dependencies, including checking exact matches via VCS (Git) history. It updates the bootstrapping process in application-base to detect explicit version requests (e.g., commits, tags, branches) and pass them to version validation, warning instead of failing if validation fails on an explicit request. Additionally, it adds helper functions for hashing nested configurations and includes comprehensive unit tests. The reviewer suggests adding an optional path parameter to validate_versions and is_available to allow searching for executables in a user-specified directory.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for validating exact versions of bootstrapped utilities using VCS (Git) history. It adds a helper to check exact matches via VCS, updates validate_versions and is_available to support exact_version checks, and adjusts the bootstrapping process to warn rather than fail when an explicit version is requested but validation fails. It also includes comprehensive tests for these new features. Feedback suggests using the -- double-dash separator in the git rev-parse command to prevent potential option injection vulnerabilities from user-provided version strings.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for validating exact versions of external dependencies using VCS (specifically Git) and explicit version requests (such as commit, branch, tag, or sha). It updates _bootstrap_utilities to extract and pass exact version constraints, and enhances validate_versions and is_available in the utility base class to support exact version checks. Comprehensive unit tests are also added to verify these changes. A review comment points out a redundant check for obj.variables being None at line 2712, as it is already initialized earlier in the same loop iteration.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for validating exact versions of external dependencies (utilities) using VCS (specifically Git) in Ramble. It adds a helper method _check_exact_match_via_vcs to check if an executable matches the requested exact version via Git history, allowing the validation to pass even if the version cannot be parsed from the executable's output. It also updates the bootstrapping process to track explicit version requests, handle missing variables gracefully, and downgrade validation failures to warnings when an explicit version is requested. Additionally, new unit tests are added to verify these changes. Feedback is provided to optimize performance in validate_versions by skipping the execution of the version command subprocess when the exact version is already confirmed via VCS and no minimum or maximum version constraints are specified.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces exact version validation for external dependencies using VCS (Git) history, updates utility bootstrapping to support exact version checks, and adds comprehensive unit tests. Feedback on the changes includes addressing a bug in git rev-parse where the -- separator causes the revision to be incorrectly treated as a pathspec, and extending the _hashable_val helper to handle tuples containing nested mutable structures.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for validating exact versions of external dependencies/utilities using VCS (Git) history. It adds a _check_exact_match_via_vcs helper to verify exact version matches via Git commits, updates validate_versions and is_available to handle exact version checks, and adjusts the bootstrapping process to warn rather than fail when an explicit version is requested but validation fails. Additionally, it adds comprehensive tests for these features and fixes a potential issue with unhashable utility configurations. The reviewer pointed out that checking is_git.returncode == 0 is insufficient for git rev-parse --is-inside-work-tree because it can return 0 even when not in a working tree (e.g., inside a .git directory), and suggested verifying that the stdout is explicitly "true".

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for validating exact versions of external dependencies and utilities using VCS (Git) history and fallback output matching. It updates the utility bootstrapping process in the application base class to detect explicit version requests (such as commits, tags, or branches) and pass them to the utility's availability and validation checks. Additionally, it updates the validation logic to issue a warning instead of terminating when validation fails on an explicitly requested version. The changes also include helper functions for hashing nested configurations in workspace commands and comprehensive unit tests for the new validation logic. I have no feedback to provide.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for exact version validation of bootstrapped utilities using VCS (Git) history, updates workspace configuration hashing, and adds comprehensive tests. Feedback on the changes suggests ensuring backward compatibility with Python 3.5/3.6 by avoiding capture_output and text in subprocess.run, resolving symlinks before performing VCS checks, and sorting dictionary keys by their string representation in _hashable_val to prevent type errors with mixed-type keys.

[douglasjacobsen]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for validating exact versions of external dependencies using VCS (Git) history, updating both the application and utility base classes to handle explicit version requests. It also adds comprehensive unit tests for these new validation paths and fixes various documentation formatting issues. The review feedback highlights a critical backward-compatibility concern: custom or third-party utilities overriding is_available or validate_versions with older signatures may raise a TypeError when called with the new exact_version keyword argument. It is recommended to wrap these calls in try...except TypeError blocks to gracefully fall back to the older signatures.