Skip to content

fix(simulation): prevent concurrent prepare threads - #64

Open
MA1503 wants to merge 1 commit into
nikmcfly:mainfrom
MA1503:fix/concurrent-prepare-guard
Open

fix(simulation): prevent concurrent prepare threads#64
MA1503 wants to merge 1 commit into
nikmcfly:mainfrom
MA1503:fix/concurrent-prepare-guard

Conversation

@MA1503

@MA1503 MA1503 commented Jul 24, 2026

Copy link
Copy Markdown

Symptom

Opening Step 2 (environment setup) in the frontend while a preparation is
already running spawns another full persona-generation thread. With repeated
visits, several generators run at once — they split the Ollama throughput
between them and race on the same profile output files. The UI shows
progress counters that jump around (observed: three concurrent generators at
counter bands 30 / 139 / 159 of 252).

Cause

Two gaps in backend/app/api/simulation.py:

  1. /prepare/status reports not_started while a prepare is running.
    The status endpoint only checks whether the output files already exist
    (_check_simulation_prepared). While generation is still in flight those
    files are absent, so with a simulation_id and no task_id the endpoint
    returns "status": "not_started" — telling the frontend that no
    preparation has begun.

  2. /prepare has no duplicate guard. It creates a task and immediately
    starts threading.Thread(target=run_prepare) with no record of an
    in-flight prepare, so nothing stops a second (or third) call from starting
    another thread for the same simulation.

Combined, a client that polls /prepare/status, sees not_started, and calls
/prepare again keeps launching new generation threads.

Fix

A module-level registry _active_prepares maps simulation_id → task_id:

  • /prepare refuses to start a second prepare while one is active for the
    same simulation and returns the existing task_id (status preparing)
    instead of spawning another thread. A stale/finished entry is cleared so a
    genuine re-prepare still works.
  • The background thread is wrapped so the registry entry is removed in a
    finally block — on success and on failure.
  • /prepare/status now checks the registry: when a task is PENDING or
    PROCESSING it reports preparing with the task's real progress and
    message, instead of the misleading not_started.

After a backend restart the registry is empty, so a stale preparing value in
state.json cannot block a fresh prepare.

Repro

  1. Start a prepare for a simulation (Step 2).
  2. While persona generation is running, open Step 2 again one or more times
    (or poll /prepare/status with only simulation_id, see not_started,
    and call /prepare again).
  3. Before the fix: multiple persona-generation threads run concurrently,
    counters jump, profile files are overwritten by racing writers.
    After the fix: subsequent /prepare calls return the existing task and no
    second thread starts; /prepare/status reports preparing with real
    progress throughout.

🤖 Generated with Claude Code

The prepare status endpoint reported "not_started" while a prepare was
actively running (it only checks output files, not the live task), and
/prepare had no duplicate guard — every visit to Step 2 spawned another
full persona-generation thread. Observed: three concurrent generators
(counter bands 30/139/159 of 252) splitting Ollama throughput three ways
and racing on the same profile files, with jumping counters in the UI.

Fix: a module-level registry of active prepare tasks per simulation.
/prepare refuses duplicates and returns the existing task_id;
/prepare/status reports active tasks as "preparing" with real progress.
The registry entry is cleared when the thread ends (success and failure).
After a backend restart the registry is empty, so a stale "preparing"
state does not block a fresh prepare.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant