Skip to content

use multithreading instead of multiprocessing for spawning runners - #642

Merged
cbartz merged 3 commits into
mainfrom
remove-multiprocessing
Oct 15, 2025
Merged

cbartz merged 3 commits into
mainfrom
remove-multiprocessing

Conversation

@cbartz

@cbartz cbartz commented Oct 1, 2025 •

Copy link
Copy Markdown
Collaborator

Applicable spec:

Overview

Use multithreading instead of multiprocessing

Rationale

multiprocessing is more costly in terms of memory. The charm might spawn way more runners than currently in the future once it is able to spawn runners for different openstack flavors,images, etc. In order to support this usecase, we remove the multiprocessing part.

Checklist

  • The charm style guide was applied.
  • The contributing guide was applied.
  • The changes are compliant with ISD054 - Managing Charm Complexity
  • The documentation for charmhub is updated.
  • The PR is tagged with appropriate label (urgent, trivial, complex).
  • The changelog is updated with changes that affects the users of the charm.
  • The application version number is updated in github-runner-manager/pyproject.toml.

No direct user impact (except fewer resource usage).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR switches from multiprocessing to multithreading for spawning runners to reduce memory usage. The change anticipates future scaling needs where the charm may spawn many more runners across different OpenStack flavors and images.

  • Replaced multiprocessing.Pool with multiprocessing.pool.ThreadPool
  • Updated version number from 0.6.2 to 0.6.3

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
github-runner-manager/src/github_runner_manager/manager/runner_manager.py Switched from Process Pool to Thread Pool for runner spawning
github-runner-manager/pyproject.toml Bumped version number to reflect the change

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@cbartz
cbartz enabled auto-merge (squash) October 7, 2025 09:12
@cbartz
cbartz merged commit a94c6f5 into main Oct 15, 2025
84 of 109 checks passed
@cbartz
cbartz deleted the remove-multiprocessing branch October 15, 2025 11:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants