Skip to content

Avoid blocking Git subprocesses in async server jobs #832

Description

@GautamSharma99

Issue description

The server executes synchronous Git operations from native async functions. Because APScheduler runs coroutine jobs directly on the
application’s asyncio event loop, these calls can temporarily block all other work handled by that worker.

Exact code path

At startup, the FastAPI lifespan creates and starts an AsyncIOScheduler:

  • packages/server/src/repowise/server/app.py:209
  • packages/server/src/repowise/server/app.py:210

Every 15 minutes, APScheduler invokes the async polling_fallback() function. It loops through every registered repository and calls
subprocess.run() synchronously:

  • packages/server/src/repowise/server/scheduler.py:72
  • packages/server/src/repowise/server/scheduler.py:94
  • packages/server/src/repowise/server/scheduler.py:110

Each invocation has a 10-second timeout. Since repositories are processed sequentially, the worst-case blocking time grows linearly:

number of repositories × 10 seconds

The same underlying problem exists in the background job executor:

  • execute_job() awaits _incremental_page_regen() at packages/server/src/repowise/server/job_executor.py:407.
  • _incremental_page_regen() synchronously runs git rev-parse at packages/server/src/repowise/server/job_executor.py:742.
  • It then runs synchronous GitPython change detection and file parsing on the event-loop thread.

The final state-stamping path also invokes synchronous git rev-parse through _read_head_sha():

  • packages/server/src/repowise/server/job_executor.py:487
  • packages/server/src/repowise/server/job_executor.py:497
  • packages/server/src/repowise/server/job_executor.py:598

Why this matters

While one of these operations is blocked, the same server worker cannot promptly:

  • Handle API requests.
  • Deliver SSE job-progress events.
  • Process cancellation requests.
  • Run other scheduled jobs.
  • Advance other background generation tasks.

Git normally responds quickly, but slow filesystems, repository locks, hooks, network-mounted worktrees, or unhealthy repositories can make
the pause noticeable. Polling multiple repositories compounds the delay.

Expected behavior

Git inspection and synchronous change detection should not block the asyncio event loop. Other requests and background tasks should continue
running while a repository is inspected.

Proposed implementation direction

Use one of the repository’s async-safe execution approaches:

  • Run isolated blocking operations with await asyncio.to_thread(...).
  • Prefer asyncio.create_subprocess_exec() for Git commands, with explicit timeout, termination, and process cleanup.
  • Offload the synchronous ChangeDetector work as a unit rather than moving only git rev-parse off the event loop.

The existing behavior for timeouts, non-zero exit codes, and missing repositories should remain unchanged.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions