Skip to content

Use module-scoped fixtures for new_upgrades/test_classparameter.py#21650

Draft
tpapaioa wants to merge 1 commit into
SatelliteQE:masterfrom
tpapaioa:new_upgrades_classparameter_module_scope
Draft

Use module-scoped fixtures for new_upgrades/test_classparameter.py#21650
tpapaioa wants to merge 1 commit into
SatelliteQE:masterfrom
tpapaioa:new_upgrades_classparameter_module_scope

Conversation

@tpapaioa
Copy link
Copy Markdown
Contributor

@tpapaioa tpapaioa commented May 23, 2026

Problem Statement

  • Endeavour Upgrade test suites take a long time to run, with a large number of Satellites deployed
  • For example, just the puppet class parameter tests in recent 6.19-to-stream upgrade-scenario CI jobs for team Endeavour take ~170 minutes total for 9 tests deploying 9 separate Satellites. They are currently all run sequentially, but even with maximum concurrency of 4 xdist workers running 9 tests, that would involve 3 separate Satellite deployments (4 + 4 + 1).

Solution

Refactor the puppet class parameter tests to use module-scoped fixtures, where all of the puppet imports and initial class parameter setup is done first on a single Satellite, which is then used by all of the tests, whether run in parallel or sequentially by xdist. (See the tests and fixtures in tests/new_upgrades/test_pulp.py for an example of how this is already done.)

$ pytest tests/new_upgrades/test_classparameter.py
[...]
=== 9 passed, 36 warnings in 2206.99s (0:36:46) ===

Related Issues

SAT-40414

Summary by Sourcery

Refactor puppet class parameter upgrade tests to use module-scoped fixtures and shared setup to reduce Satellite deployments and speed up execution.

Enhancements:

  • Introduce a module-scoped satellite fixture for puppet upgrade tests to reuse a single Satellite instance across tests.
  • Centralize valid smart class parameter definitions in a shared constant for reuse and clarity.
  • Simplify smart class parameter setup by configuring all parameter types in a single setup flow and swapping the Satellite version once per module.

Tests:

  • Rework puppet class parameter upgrade tests to consume module-scoped fixtures and parametrized smart class parameter data instead of per-test setup with xdist-specific splaying.

@tpapaioa tpapaioa self-assigned this May 23, 2026
@tpapaioa tpapaioa added Upgrades Issues and PRs related to upgrades CherryPick PR needs CherryPick to previous branches 6.16.z 6.17.z 6.18.z Introduced in or relating directly to Satellite 6.18 6.19.z labels May 23, 2026
@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented May 23, 2026

Reviewer's Guide

Refactors the puppet class parameter upgrade tests to use a module-scoped shared Satellite fixture and a single setup run, while parameterizing the assertions over a static data table to improve runtime and parallel behavior.

File-Level Changes

Change Details Files
Refactor class parameter test setup to run once per module and reuse a single upgraded Satellite across all parameter validations.
  • Introduced a module-scoped fixture that uses module_puppet_upgrade_shared_satellite and upgrade_action to create a custom environment and configure all smart class parameters in a single loop.
  • Replaced per-test Box-based test_data object with yielding the configured Satellite instance directly from the module-level fixture.
  • Moved smart class parameter type/value definitions into a module-level VALID_SC_PARAMS_DATA constant used by both setup and tests.
  • After upgrade readiness, swapped the Satellite nailgun to the target upgrade version and reset the session before yielding for tests.
tests/new_upgrades/test_classparameter.py
Update the verification test to consume the new module-scoped fixture and pytest parametrization over the shared data set instead of per-test setup.
  • Replaced the request-parametrized fixture with pytest.mark.parametrize over the VALID_SC_PARAMS_DATA list, passing (index, param) tuples into the test.
  • Adjusted test_post_puppet_class_parameter_data_and_type to derive the smart class parameter name from the index and assert against the parametrized data instead of recomputing it.
  • Simplified value validation by reusing _validate_value with the parametrized data object.
tests/new_upgrades/test_classparameter.py
Add a module-scoped shared Satellite fixture for puppet upgrade tests to coordinate a single Satellite instance across tests in the module.
  • Created module_puppet_upgrade_shared_satellite fixture that checks out a shared Satellite instance under the key 'module_puppet_upgrade'.
  • Wrapped enable_puppet_satellite and shared_checkin in SharedResource contexts, signaling readiness at appropriate times and yielding the Satellite between them.
  • Documented that tests using this fixture should be marked with pytest.mark.puppet_upgrades and preserved the existing puppet_upgrade_shared_satellite fixture for other tests.
tests/new_upgrades/conftest.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@tpapaioa
Copy link
Copy Markdown
Contributor Author

tpapaioa commented May 23, 2026

trigger: test-robottelo
pytest: tests/new_upgrades/test_classparameter.py
env:
  ROBOTTELO_upgrade__from_version: '6.19'
  ROBOTTELO_upgrade__to_version: stream
  ROBOTTELO_server__inventory_filter: ''
  ROBOTTELO_server__xdist_behavior: run-on-one

@tpapaioa tpapaioa requested review from lpramuk and synkd May 23, 2026 02:09
@Satellite-QE
Copy link
Copy Markdown
Collaborator

PRT Result

Build Number: 15573
Build Status: SUCCESS
PRT Comment: pytest tests/new_upgrades/test_classparameter.py --external-logging
Test Result : ================= 9 passed, 44 warnings in 2140.10s (0:35:40) ==================

@Satellite-QE Satellite-QE added the PRT-Passed Indicates that latest PRT run is passed for the PR label May 23, 2026
@tpapaioa tpapaioa marked this pull request as ready for review May 23, 2026 04:13
@tpapaioa tpapaioa requested a review from a team as a code owner May 23, 2026 04:13
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • The test_post_puppet_class_parameter_data_and_type parametrization can be simplified by using enumerate(VALID_SC_PARAMS_DATA, start=1) directly and generating more descriptive ids (e.g. based on sc_type) rather than range(...), which will also make test output easier to read.
  • The module-scoped setup fixture calls private attributes/methods (_swap_nailgun, _session); if there is a public helper or API to reset the client/session for the upgraded Satellite, it would be more robust to use that instead of relying on private internals.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `test_post_puppet_class_parameter_data_and_type` parametrization can be simplified by using `enumerate(VALID_SC_PARAMS_DATA, start=1)` directly and generating more descriptive `ids` (e.g. based on `sc_type`) rather than `range(...)`, which will also make test output easier to read.
- The module-scoped setup fixture calls private attributes/methods (`_swap_nailgun`, `_session`); if there is a public helper or API to reset the client/session for the upgraded Satellite, it would be more robust to use that instead of relying on private internals.

## Individual Comments

### Comment 1
<location path="tests/new_upgrades/test_classparameter.py" line_range="94-96" />
<code_context>


 @pytest.mark.puppet_upgrades
+@pytest.mark.parametrize(
+    'sc_data',
+    [p for p in enumerate(VALID_SC_PARAMS_DATA)],
+    ids=range(1, len(VALID_SC_PARAMS_DATA) + 1),
+)
</code_context>
<issue_to_address>
**suggestion (testing):** Parametrization over `enumerate(VALID_SC_PARAMS_DATA)` could be made clearer and more descriptive for test readability

`sc_data` is currently an `(index, my_param)` tuple from `enumerate(VALID_SC_PARAMS_DATA)`, with numeric `ids` from `range(1, ...)`. This works but makes parameter handling and failure output harder to interpret.

Consider either:
- Parametrizing two arguments explicitly:
  ```python
  @pytest.mark.parametrize(
      "index, sc_param",
      [(i, data) for i, data in enumerate(VALID_SC_PARAMS_DATA)],
      ids=[f"{i+1}-{data['sc_type']}" for i, data in enumerate(VALID_SC_PARAMS_DATA)],
  )
  ```
- Or using `pytest.param` with explicit ids based on `sc_type`.

Both approaches make CI failures self-describing without mapping numeric ids back to the constant list.

Suggested implementation:

```python
@pytest.mark.puppet_upgrades
@pytest.mark.parametrize(
    "index, sc_param",
    [(i, data) for i, data in enumerate(VALID_SC_PARAMS_DATA)],
    ids=[f"{i+1}-{data['sc_type']}" for i, data in enumerate(VALID_SC_PARAMS_DATA)],
)

```

1. Update the corresponding test function signature to accept `index` and `sc_param` instead of the previous single `sc_data` argument (if it existed), e.g. `def test_...(index, sc_param, ...)`.
2. Adjust any test body references that used `sc_data` to instead use `index` or `sc_param` as appropriate, for example:
   - If you previously unpacked `sc_data` as `(idx, param)`, you can now use `index` and `sc_param` directly.
   - Any logging or assertion messages can now include `sc_param["sc_type"]` to take advantage of the more descriptive parametrization.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +94 to +96
@pytest.mark.parametrize(
'sc_data',
[p for p in enumerate(VALID_SC_PARAMS_DATA)],
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Parametrization over enumerate(VALID_SC_PARAMS_DATA) could be made clearer and more descriptive for test readability

sc_data is currently an (index, my_param) tuple from enumerate(VALID_SC_PARAMS_DATA), with numeric ids from range(1, ...). This works but makes parameter handling and failure output harder to interpret.

Consider either:

  • Parametrizing two arguments explicitly:
    @pytest.mark.parametrize(
        "index, sc_param",
        [(i, data) for i, data in enumerate(VALID_SC_PARAMS_DATA)],
        ids=[f"{i+1}-{data['sc_type']}" for i, data in enumerate(VALID_SC_PARAMS_DATA)],
    )
  • Or using pytest.param with explicit ids based on sc_type.

Both approaches make CI failures self-describing without mapping numeric ids back to the constant list.

Suggested implementation:

@pytest.mark.puppet_upgrades
@pytest.mark.parametrize(
    "index, sc_param",
    [(i, data) for i, data in enumerate(VALID_SC_PARAMS_DATA)],
    ids=[f"{i+1}-{data['sc_type']}" for i, data in enumerate(VALID_SC_PARAMS_DATA)],
)
  1. Update the corresponding test function signature to accept index and sc_param instead of the previous single sc_data argument (if it existed), e.g. def test_...(index, sc_param, ...).
  2. Adjust any test body references that used sc_data to instead use index or sc_param as appropriate, for example:
    • If you previously unpacked sc_data as (idx, param), you can now use index and sc_param directly.
    • Any logging or assertion messages can now include sc_param["sc_type"] to take advantage of the more descriptive parametrization.

@tpapaioa tpapaioa marked this pull request as draft May 24, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.16.z 6.17.z 6.18.z Introduced in or relating directly to Satellite 6.18 6.19.z CherryPick PR needs CherryPick to previous branches PRT-Passed Indicates that latest PRT run is passed for the PR Upgrades Issues and PRs related to upgrades

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants