Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

---

## [6.1.0] – branch: `fix/singleton-multi-target`

### Fixed

- **Singleton-Guard bei Multi-Target-Jobs zielspezifisch (closes #147):** `ExecutionStartEndpoint` prüfte beim Singleton-Guard mit `hasRunningExecution()` job-weit (`WHERE cronjob_id = :id AND finished_at IS NULL`), ohne den `target`-Wert zu berücksichtigen. Bei Jobs mit mehreren Targets (z.B. di, dm, dv, dvme) blockierte der erste gestartete Target alle weiteren, obwohl jeder Target eine unabhängige Ausführung auf einem anderen System darstellt. Folge: Pro Scheduler-Tick (und pro „Jetzt ausführen"-Aufruf) erschien in der Ausführungshistorie nur ein einziger Eintrag, obwohl alle Targets laufen sollten. Fix: Neue Methode `hasRunningExecutionForTarget()` (`WHERE cronjob_id = :id AND target = :target AND finished_at IS NULL`). Der Singleton-Guard und der Retry-Pending-Check verwenden nun `hasPendingRetryForTarget()` statt `hasPendingRetry()`, sodass der Guard ausschließlich dann greift, wenn **dasselbe Target** bereits läuft oder auf einen Retry wartet.

---

## [6.0.0] – branch: `feature/v6.0.0`

### Added
Expand Down
17 changes: 11 additions & 6 deletions TECHNICAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,12 +249,17 @@ runs regardless and a best-effort finish report is sent.

**Singleton mode:**

When a job has the `singleton` flag set, the `ExecutionStartEndpoint` queries
`execution_log` for any row with the same `cronjob_id` that has `finished_at IS NULL`
(i.e. still running). If one is found, the agent returns `409 Conflict` instead of
inserting a new log row. The wrapper detects the `409` HTTP status code and exits 0
immediately — no execution record is created and no failure is reported. This is
transparent to the user unless they inspect the agent log.
When a job has the `singleton` flag set, the `ExecutionStartEndpoint` checks
`execution_log` for any row with the same `cronjob_id` **and the same `target`** that has
`finished_at IS NULL` (i.e. still running for that specific target). If one is found, the
agent returns `409 Conflict` instead of inserting a new log row. The wrapper detects the
`409` HTTP status code and exits 0 immediately — no execution record is created and no
failure is reported. This is transparent to the user unless they inspect the agent log.

The check is **target-specific**: for jobs with multiple targets (e.g. di, dm, dv, dvme),
each target is treated as an independent execution. A running instance on target A does not
block a new instance on target B. The singleton guard only fires when the same target would
overlap itself.

**Dependencies:** `bash 4+`, `curl`, `openssl`, `php`

Expand Down
2 changes: 1 addition & 1 deletion agent/VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
6.0.0
6.1.0
27 changes: 26 additions & 1 deletion agent/src/Endpoints/ExecutionStartEndpoint.php
Original file line number Diff line number Diff line change
Expand Up @@ -187,9 +187,10 @@ public function handle(array $params): void
return;
}

if ($this->isSingleton($jobId) && ($this->hasRunningExecution($jobId) || (!$isRetryExecution && $this->hasPendingRetry($jobId)))) {
if ($this->isSingleton($jobId) && ($this->hasRunningExecutionForTarget($jobId, $effectiveTargetForRetry) || (!$isRetryExecution && $this->hasPendingRetryForTarget($jobId, $effectiveTargetForRetry)))) {
$this->logger->info('ExecutionStartEndpoint: singleton job is busy (running or retry pending) – skipping', [
'job_id' => $jobId,
'target' => $effectiveTargetForRetry,
]);
jsonResponse(409, [
'error' => 'Conflict',
Expand Down Expand Up @@ -510,6 +511,30 @@ private function hasRunningExecution(int $jobId): bool
return $stmt->fetchColumn() !== false;
}

/**
* Return true when an execution for the given (job, target) pair is still running.
*
* Unlike hasRunningExecution(), this method matches the exact target so the
* singleton guard allows multiple independent targets of the same job to run
* concurrently — only the same target is blocked from overlapping itself.
*
* @param int $jobId The job ID to check.
* @param string $target Effective execution target (e.g. "local", SSH alias).
*
* @return bool
*
* @throws \PDOException On database errors.
*/
private function hasRunningExecutionForTarget(int $jobId, string $target): bool
{
$stmt = $this->pdo->prepare(
'SELECT 1 FROM execution_log WHERE cronjob_id = :id AND target = :target AND finished_at IS NULL LIMIT 1'
);
$stmt->execute([':id' => $jobId, ':target' => $target]);

return $stmt->fetchColumn() !== false;
}

/**
* Return true when a retry is pending for the given job in job_retry_state.
*
Expand Down
56 changes: 47 additions & 9 deletions tests/Integration/Endpoints/ExecutionStartEndpointTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -236,26 +236,64 @@ public function singletonJobWithRunningExecutionReturns409(): void
}

#[Test]
public function singletonJobWithPendingRetryReturns409(): void
public function singletonJobWithPendingRetryOnSameTargetReturns409(): void
{
$jobId = $this->seedJob(['singleton' => 1]);
$executionId = $this->seedRunningExecution($jobId);
// Mark it as finished so singleton's running-check passes
$executionId = $this->seedRunningExecution($jobId, ['target' => 'local']);
// Mark it as finished so the running-check passes
$this->pdo->prepare('UPDATE execution_log SET finished_at = NOW(), exit_code = 1 WHERE id = :id')
->execute([':id' => $executionId]);
// Plant a retry state on a DIFFERENT target so hasPendingRetryForTarget() won't flag this as a retry invocation
$this->seedRetryState($jobId, $executionId, ['target' => 'other-target']);
// Retry is pending on the SAME target ('local') → must block
$this->seedRetryState($jobId, $executionId, ['target' => 'local']);

$this->callHandle($this->makeEndpoint(), [
'job_id' => $jobId,
'started_at' => '2026-01-15T10:00:00Z',
'target' => 'local',
'is_retry_invocation' => false,
'job_id' => $jobId,
'started_at' => '2026-01-15T10:00:00Z',
'target' => 'local',
'is_retry_invocation' => false,
]);

$this->assertStatus(409);
}

#[Test]
public function singletonJobWithRunningExecutionOnDifferentTargetAllowsStart(): void
{
// target A (ssh-server-1) is running; target B (local) must not be blocked
$jobId = $this->seedJob(['singleton' => 1]);
$this->seedRunningExecution($jobId, ['target' => 'ssh-server-1']);

$this->callHandle($this->makeEndpoint(), [
'job_id' => $jobId,
'started_at' => '2026-01-15T10:00:00Z',
'target' => 'local',
]);

$this->assertStatus(201);
// Two rows: the seeded running one + the newly created one
$this->assertSame(2, $this->countExecutions($jobId));
}

#[Test]
public function singletonJobWithPendingRetryOnDifferentTargetAllowsStart(): void
{
// Retry pending on ssh-server-1 must not block a new run on local
$jobId = $this->seedJob(['singleton' => 1]);
$executionId = $this->seedRunningExecution($jobId, ['target' => 'ssh-server-1']);
$this->pdo->prepare('UPDATE execution_log SET finished_at = NOW(), exit_code = 1 WHERE id = :id')
->execute([':id' => $executionId]);
$this->seedRetryState($jobId, $executionId, ['target' => 'ssh-server-1']);

$this->callHandle($this->makeEndpoint(), [
'job_id' => $jobId,
'started_at' => '2026-01-15T10:00:00Z',
'target' => 'local',
'is_retry_invocation' => false,
]);

$this->assertStatus(201);
}

// =========================================================================
// 5. Retry-pending guard (non-singleton)
// =========================================================================
Expand Down
2 changes: 1 addition & 1 deletion web/VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
6.0.0
6.1.0
Loading